GKE + Workload Identity Setup

Setting up Workload Identity in GKE workloads was one of the first things I had to deal with in GKE. I primarily worked on EKS until then, where we moved from using IAM users with IAM Access Keys to IRSA (IAM Roles for Service Accounts). Just like AWS, at first I started with the GCP Service Account Key, which has the same drawbacks as AWS IAM Access Keys or any similar methods i.e. the keys need to be rotated to ensure safety and even so it poses the risk of falling into the wrong hands. So we found out about Workload Identity in GKE and implemented it. This is guide on how it’s done.

GCP Service Account#

We’ll need a GCP Service Account with the necessary permissions. For testing we’ll assign roles/storage.bucketViewer for list access to storage buckets

Create a GCP Service Account

gcloud iam service-accounts create GSA_NAME --display-name "GSA for Workload Identity" --project PROJECT_ID

Assign roles/storage.bucketViewer to the service account

gcloud projects add-iam-policy-binding PROJECT_ID --member="serviceAccount:GSA_NAME@PROJECT_ID.iam.gserviceaccount.com" --role="roles/storage.bucketViewer" --project PROJECT_ID

GKE Cluster#

Now that the GSA is ready, we’ll need to make sure to enable Workload Identity in the GKE Cluster we have

gcloud container clusters update CLUSTER_NAME --location=LOCATION --workload-pool=PROJECT_ID.svc.id.goog

NOTE
If you have a GKE Autopilot Cluster, the Workload Identity will be enabled by default with namespace - PROJECT_ID.svc.id.goog

To enable it in an existing nodepool

gcloud container node-pools update NODEPOOL_NAME --cluster=CLUSTER_NAME --location=LOCATION --workload-metadata=GKE_METADATA

NOTE
If the nodepools are created after Workload Identity is enabled in the cluster, they will also have it enabled

GKE Workload#

The GSA and GKE Cluster are setup correctly now. Next we need to let the Kubernetes workload list the storage buckets.

Bind the GSA to the KSA (Kubernetes Service Account)

gcloud iam service-accounts add-iam-policy-binding --role="roles/iam.workloadIdentityUser" --member="serviceAccount:PROJECT_ID.svc.id.goog[K8S_NAMESPACE/K8S_SA]" GSA_NAME@PROJECT_ID.iam.gserviceaccount.com --project PROJECT_ID

Create yaml for a sample kubernetes pod and a service account. The annotation iam.gke.io/gcp-service-account: YOUR-GCP-SA@YOUR-PROJECT.iam.gserviceaccount.com lets GKE mount a token into the pod, which lets us access the GCP resources by impersonating the account

1
apiVersion: v1
2
kind: ServiceAccount
3
metadata:
4
  name: K8S_SA
5
  namespace: K8S_NAMESPACE
6
  annotations:
7
    iam.gke.io/gcp-service-account: GSA_NAME@PROJECT_ID.iam.gserviceaccount.com
8
---
9
apiVersion: v1
10
kind: Pod
11
metadata:
12
  name: gcloud-cli-pod
13
  namespace: K8S_NAMESPACE
14
spec:
15
  serviceAccountName: K8S_SA
16
  containers:
17
  - name: gcloud-cli
18
    image: gcr.io/google.com/cloudsdktool/google-cloud-cli:stable
19
    command: ["sleep"]
20
    args: ["infinity"]

Use kubectl to create the resources

kubectl apply -f workload-identity.yaml

Once the pod is up, exec into the pod

kubectl -n K8S_NAMESPACE exec gcloud-cli-pod -it -- bash

Try to list the storage buckets in the GCP Project

gcloud storage buckets list

Et voilà! Now you can follow these steps to avoid using GSA Keys.

NOTE
By default Google Cloud SDK supports this method, so any code can be run from inside the pod utilizing Workload Identity without any additional configuration