Cert-Manager + Route53 Setup

After setting up ingress-nginx with an ACM certificate in EKS clusters, we also needed to set up ingress-nginx in GKE and AKS clusters, which was easy since we just needed to use the default ingress-nginx helm chart to deploy to whichever cluster as required. But TLS was an issue.

Solution - since the domain is managed in Amazon Route 53, we can use cert-manager with DNS01 challenge to get TLS working along with Route53 in any cluster, regardless of the cloud provider. Here we’ll see the steps required for this setup. DNS01 challenge essentially involves you proving you own the domain by creating a TXT record, which is then checked by the CA (here it will be Let’s Encrypt) and once verified you’re given the TLS cert. For more details - Ref

AWS setup#

Create the IAM user to be used by cert-manager

aws iam create-user --user-name route53-certmanager-user

Use the following policy json that allows cert-manager to access the route53 zone

1
{
2
  "Version": "2012-10-17",
3
  "Statement": [
4
    {
5
      "Effect": "Allow",
6
      "Action": "route53:GetChange",
7
      "Resource": "arn:aws:route53:::change/*"
8
    },
9
    {
10
      "Effect": "Allow",
11
      "Action": [
12
        "route53:ChangeResourceRecordSets",
13
        "route53:ListResourceRecordSets"
14
      ],
15
      "Resource": "arn:aws:route53:::hostedzone/*",
16
      "Condition": {
17
        "ForAllValues:StringEquals": {
18
          "route53:ChangeResourceRecordSetsRecordTypes": ["TXT"]
19
        }
20
      }
21
    },
22
    {
23
      "Effect": "Allow",
24
      "Action": "route53:ListHostedZonesByName",
25
      "Resource": "*"
26
    }
27
  ]
28
}

Create and attach policy to the IAM User

aws iam put-user-policy --user-name route53-certmanager-user --policy-name route53-certmanager-policy --policy-document file://cert-manager/route53-policy.json

Create IAM Secret keys for the IAM user. Note the fields - AccessKeyId and SecretAccessKey

aws iam create-access-key --user-name route53-certmanager-user

Cert-Manager setup#

For ease of management let’s use helm to deploy cert-manager.

Run the following command to deploy it in your K8s cluster:

helm install \
  cert-manager oci://quay.io/jetstack/charts/cert-manager \
  --version v1.18.2 \
  --namespace cert-manager \
  --create-namespace \
  --set crds.enabled=true

Replace AWS_ACCESS_KEY_ID_BASE64 and AWS_SECRET_ACCESS_KEY_BASE64 in the following K8s secrets yaml with the base64 encoded values of AccessKeyId and SecretAccessKey we created earlier. Note that the secret should be created in the same namespace where cert-manager is deployed - here it’s cert-manager.

1
apiVersion: v1
2
kind: Secret
3
metadata:
4
  name: route53-credentials-secret
5
  namespace: cert-manager
6
data:
7
  aws-access-key-id: AWS_ACCESS_KEY_ID_BASE64
8
  aws-secret-access-key: AWS_SECRET_ACCESS_KEY_BASE64
9
type: Opaque

Create the secret

kubectl apply -f route53-credentials-secret.yaml

Replace ROUTE53_DOMAIN_NAME with the Route53 domain name. Example: yourdomain.com

1
apiVersion: cert-manager.io/v1
2
kind: ClusterIssuer
3
metadata:
4
  name: letsencrypt-prod
5
spec:
6
  acme:
7
    privateKeySecretRef:
8
      name: letsencrypt-prod
9
    server: https://acme-v02.api.letsencrypt.org/directory
10
    solvers:
11
    - dns01:
12
        route53:
13
          accessKeyIDSecretRef:
14
            key: aws-access-key-id
15
            name: route53-credentials-secret
16
          region: us-east-1
17
          secretAccessKeySecretRef:
18
            key: aws-secret-access-key
19
            name: route53-credentials-secret
20
      selector:
21
        dnsZones:
22
        - ROUTE53_DOMAIN_NAME

Create the ClusterIssuer. This will be used in our ingress resource to request the TLS certificate.

kubectl apply -f clusterissuer.yaml

Testing Ingress-Nginx with TLS#

Now we can deploy the sample workload to test this. Replace SAMPLE_DOMAIN with your domain as required

1
apiVersion: v1
2
kind: Pod
3
metadata:
4
  name: nginx
5
  labels:
6
    app: nginx
7
spec:
8
  containers:
9
    - name: nginx
10
      image: nginx:latest
11
      ports:
12
        - containerPort: 80
13
---
14
apiVersion: v1
15
kind: Service
16
metadata:
17
  name: nginx-service
18
spec:
19
  selector:
20
    app: nginx
21
  ports:
22
    - protocol: TCP
23
      port: 80
24
      targetPort: 80
25
  type: ClusterIP
26
---
27
apiVersion: networking.k8s.io/v1
28
kind: Ingress
29
metadata:
30
  name: nginx-ingress
31
  annotations:
32
    cert-manager.io/cluster-issuer: letsencrypt-prod
33
spec:
34
  ingressClassName: nginx  # This must match the Ingress Controller's class name
35
  tls:
36
  - hosts:
37
    - SAMPLE_DOMAIN
38
    secretName: domain-cert-secret
39
  rules:
40
    - host: SAMPLE_DOMAIN  # Replace with your actual domain
41
      http:
42
        paths:
43
          - path: /
44
            pathType: Prefix
45
            backend:
46
              service:
47
                name: nginx-service
48
                port:
49
                  number: 80

Get the ingress-nginx loadbalancer IP (if applicable). This is usually applicable in most clusters.

kubectl -n ingress-nginx get svc ingress-nginx-controller -o jsonpath='{.status.loadBalancer.ingress[0].ip}'

OR - Get the ingress-nginx loadbalancer hostname (if applicable). This is usually applicable in EKS.

kubectl -n ingress-nginx get svc ingress-nginx-controller -o jsonpath='{.status.loadBalancer.ingress[0].hostname}'

Create Route53 record for SAMPLE_DOMAIN with the value as either the loadbalancer IP or loadbalancer hostname.
Wait for a few seconds for the DNS01 challenge and DNS propagation to be over.
Try to access https://SAMPLE_DOMAIN. Et voilà! The endpoint is accessible with working TLS