GKE Autopilot Node Compromise via Race Condition

Date: Apr 1, 2021 05:48PM

Vulnerability Details

GKE Autopilot provides many default security controls to protect the node, including protections against privileged pods and limitations on hostPath volume mounts.

There is a race condition during provisioning of the Autopilot OPA policies that allows an attacker to bypass the intended controls by simply creating their malicious resource while the OPA policies are being provisioned. To demonstrate, use the following steps:

  1. Download the file deploy.yaml
  2. Create an Autopilot cluster
  3. While the cluster is being created, run the following commands (replacing $name):
while true; do
  gcloud container clusters get-credentials $name && break;
done
while true; do
  timeout 1 kubectl apply -f deploy.yaml && break;
done
kubectl exec -it deploy/priv -- nsenter --mount=/proc/1/ns/mnt -- /bin/bash

Observe that the privileged deployment initially fails to create for the following reasons:

After a few minutes have passed, the deployment will succeed and allow the user to exec into their new privileged pod.

Attack Scenario

The Autopilot documentation describes the motivation for its security controls as:

In order for GKE to offer management of the nodes and provide you with a more streamlined operational experience, there are a few restrictions and limitations when compared to GKE Standard. Some of these limitations are security best practices, while others allow Autopilot clusters to be safely managed.

A user with access to create pods and persistent volumes in the Autopilot cluster could bypass the security controls in the cluster and gain privileged access to the managed Kubernetes nodes. Using this access, they could read all secrets in the cluster (including those outside their provisioned access) or explore the attack surface of the Autopilot control plane. Additionally, this user can retrieve a service account token from the node metadata service for the default compute service account.

Timeline