IRSA#

1. Get the OIDC url from the EKS cluster

export OIDC_PROVIDER=$(aws eks describe-cluster --name EKS_CLUSTER_NAME --query "cluster.identity.oidc.issuer" --output text | cut -d'/' -f3-)

2. Create the IAM Trust Policy JSON. Replace with the correct values:
   • ACCOUNT_ID - AWS Account ID

cat <<EOF > trust-policy.json
{
   "Version": "2012-10-17",
   "Statement": [
      {
        "Sid": "",
        "Effect": "Allow",
        "Principal": {
           "Federated": "arn:aws:iam::ACCOUNT_ID:oidc-provider/${OIDC_PROVIDER}"
        },
        "Action": "sts:AssumeRoleWithWebIdentity",
        "Condition": {
           "StringEquals": {
              "${OIDC_PROVIDER}:sub": "system:serviceaccount:external-secrets:aws-secrets-sa"
           }
        }
      }
   ]
}
EOF

3. Create the IAM Role and note the Arn

export IAM_ROLE_ARN=$(aws iam create-role --role-name external-secrets-role --assume-role-policy-document file://trust-policy.json --query "Role.Arn" --output text)

4. Create IAM Policy JSON. This policy allows the external-secrets operator to get the secret value from AWS Secrets Manager

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "secretsmanager:GetResourcePolicy",
        "secretsmanager:GetSecretValue",
        "secretsmanager:DescribeSecret",
        "secretsmanager:ListSecretVersionIds"
      ],
      "Resource": "*"
    }
  ]
}

5. Create IAM Policy and get the Arn

export IAM_POLICY_ARN=$(aws iam create-policy --policy-name external-secrets-policy --policy-document file://iam-policy.json --query "Policy.Arn" --output text)

6. Attach the IAM Policy to the IAM Role we created

aws iam attach-role-policy --policy-arn external-secrets-policy --role-name external-secrets-role

Kubernetes Setup#

Now we’ll need to create 2 custom k8s resources - ClusterSecretStore and ExternalSecret. ExternalSecret is the resource that creates the k8s secret by accessing the AWS Secrets Manager / Vault using the ClusterSecretStore. ClusterSecretStore is a cluster-wide resource and the ExternalSecret is a namespaced resource.

1. Create the ClusterSecretStore resource. The yaml below also consists of the k8s service account that has access to the IAM Role we created earlier. Replace IAM_ROLE_ARN with the value we obtained earlier.

1
apiVersion: v1
2
kind: ServiceAccount
3
metadata:
4
  annotations:
5
    eks.amazonaws.com/role-arn: IAM_ROLE_ARN
6
  name: aws-secrets-sa
7
  namespace: external-secrets
8
---
9
apiVersion: external-secrets.io/v1
10
kind: ClusterSecretStore
11
metadata:
12
  name: aws-secrets
13
spec:
14
  provider:
15
    aws:
16
      service: SecretsManager
17
      region: us-east-1
18
      auth:
19
        jwt:
20
          serviceAccountRef:
21
            name: aws-secrets-sa
22
            namespace: external-secrets

2. Create the ExternalSecret resource. This creates a k8s secret called ExternalSecret in the namespace default, with values from the AWS Secret aws-secret. This is set to check and refresh the values of the k8s secret every 3 minutes. So any changes made in the AWS Secrets Manager will be reflected in the cluster secret too.

1
apiVersion: external-secrets.io/v1
2
kind: ExternalSecret
3
metadata:
4
  name: aws-secret
5
  namespace: default
6
spec:
7
  refreshInterval: 3m0s
8
  secretStoreRef:
9
    kind: ClusterSecretStore
10
    name: aws-secrets
11
  dataFrom:
12
  - extract:
13
      key: aws-secret
14
  target:
15
    creationPolicy: Owner
16
    name: aws-secret

3. The k8s secret with the name should be available

kubectl -n default describe secrets aws-secret

Vault Kubernetes Auth#

We will be creating a Vault Policy, Vault Kubernetes Role with access to the Secrets stored in the Vault server assigned to the Kubernetes Service Account used by the External Secrets Operator.

1. Exec into the vault-0 pod

kubectl -n vault exec -it vault-0 -- sh

2. Login to the vault server

vault login

3. Create a Vault policy with access to the Vault secrets from which we want to create k8s secrets.

vault policy write external-secrets-policy - <<EOF
path "vault-secrets/*" {
  capabilities = ["read", "list"]
}
EOF

4. Create a Vault Kubernetes Role that binds the above policy to the Kubernetes Service Account used by the External Secrets Operator.

vault write auth/kubernetes/role/external-secrets-role bound_service_account_names=external-secrets bound_service_account_namespaces=external-secrets policies=external-secrets-policy ttl=24h

Kubernetes Setup#

Now we’ll need to create the ClusterSecretStore and ExternalSecret resources for Vault.

1. Create the ClusterSecretStore resource.

1
apiVersion: external-secrets.io/v1
2
kind: ClusterSecretStore
3
metadata:
4
  name: hashicorp-vault
5
spec:
6
  provider:
7
    vault:
8
      auth:
9
        kubernetes:
10
          mountPath: kubernetes
11
          role: external-secrets-role
12
      path: vault-secrets
13
      server: http://vault.vault:8200
14
      version: v2

2. Create the ExternalSecret resource.

1
apiVersion: external-secrets.io/v1
2
kind: ExternalSecret
3
metadata:
4
  name: vault-secret
5
  namespace: default
6
spec:
7
  dataFrom:
8
  - extract:
9
      conversionStrategy: Default
10
      decodingStrategy: None
11
      key: vault-secrets/data/vault-secret-name
12
      metadataPolicy: None
13
  refreshInterval: 2m0s
14
  secretStoreRef:
15
    kind: ClusterSecretStore
16
    name: hashicorp-vault
17
  target:
18
    creationPolicy: Owner
19
    deletionPolicy: Retain
20
    name: vault-secret

3. The k8s secret with the name should be available

kubectl -n default describe secrets vault-secret

And there you have it! We have successfully set up the External Secrets Operator with AWS Secrets Manager and HashiCorp Vault.