How to Automatically Update AWS ECR Token in Kubernetes with CronJobs
The ECR token expires every 12 hours. Learn to automatically use Kubernetes CronJobs to update the ECR token in your clusters or namespaces.
β LHB Community
Do you use AWS Elastic Container Registry (ECR) to store and manage your Docker images?
In that case, you may have encountered a problem: the ECR token expires every 12 hours, and you need to renew it manually or programmatically to pull images from ECR.
This can be a hassle, especially if multiple clusters or namespaces need access to ECR.
Fortunately, there is a solution: you can use Kubernetes CronJobs to update the ECR token in your clusters or namespaces automatically.
In this article, I will show you how to set up a CronJob that runs every 10 hours and updates the ECR token in a secret that your pods or deployments can use to pull images from ECR.
This way, you donβt have to worry about token expiration and can focus on your application development. Letβs get started!
Prerequisites
Before you begin, make sure you have:
- An AWS account: You need an AWS account to create and use an ECR repository. You can sign up for free here if you don't have one.
- An ECR repository: You need an ECR repository to store and manage your Docker images. If you donβt have one, you can create one using the AWS Console or the AWS CLI.
- AWS credentials (access key and secret access key): You need AWS credentials to authenticate with ECR and get a token. You can create a new IAM user or use an existing one that has the AmazonEC2ContainerRegistryReadOnly policy attached.
- A Kubernetes cluster: You need a Kubernetes cluster to run your CronJob and your pods or deployments that pull images from ECR. You can use any Kubernetes provider or platform, such as Amazon EKS, Minikube, etc.
- kubectl: You need kubectl to interact with your Kubernetes cluster and create resources. You can install kubectl on your local machine or use a cloud shell.
Once you have all the prerequisites ready, you can proceed to the next section, where you will set up a CronJob to update the ECR token secret.
Letβs get started!
Step 1: Create a secret, a configmap, and a service account for the CronJob
The first step is to create a secret, a configmap, and a service account that the CronJob will use to access AWS and update the ECR token secret.
I will use the default namespace for this example, but you can use any namespace you want.
To create the secret, the configmap, and the service account, save the following content as ecr-registry-helper.yaml
:
apiVersion: v1
kind: Secret
metadata:
name: ecr-registry-helper-secrets
namespace: default
stringData:
AWS_SECRET_ACCESS_KEY: "" # Replace with your AWS secret access key
AWS_ACCESS_KEY_ID: "" # Replace with your AWS access key ID
AWS_ACCOUNT: "" # Replace with your AWS account ID
---
apiVersion: v1
kind: ConfigMap
metadata:
name: ecr-registry-helper-cm
namespace: default
data:
AWS_REGION: "eu-central-1" # Replace with your ECR region
DOCKER_SECRET_NAME: regcred # Replace with your desired ECR token secret name
---
apiVersion: v1
kind: ServiceAccount
metadata:
name: sa-default
namespace: default
Then, apply the file using:
kubectl apply -f ecr-registry-helper.yaml
You can verify that the secret, the configmap, and the service account are created by running:
kubectl get secret ecr-registry-helper-secrets -n default
kubectl get configmap ecr-registry-helper-cm -n default
kubectl get serviceaccount sa-default -n default
Step 2: Create a role and a role binding for the CronJob
The next step is to create a role and a role binding that grants the CronJob permission to delete and create secrets in the desired namespace.
I will use the default namespace and the regcred
secret name for this example, but you can use any namespace and secret name you want.
To create the role and the role binding, save the following content as ecr-registry-helper-rbac.yaml
:
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
namespace: default
name: role-full-access-to-secrets
rules:
- apiGroups: [""]
resources: ["secrets"]
resourceNames: ["regcred"] # Replace with your desired ECR token secret name
verbs: ["delete"]
- apiGroups: [""]
resources: ["secrets"]
verbs: ["create"]
---
kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: default-role-binding
namespace: default
subjects:
- kind: ServiceAccount
name: sa-default # Replace with your service account name if different
namespace: default
apiGroup: ""
roleRef:
kind: Role
name: role-full-access-to-secrets # Replace with your role name if different
apiGroup: ""
Then, apply the file using:
kubectl apply -f ecr-registry-helper-rbac.yaml
You can verify that the role and the role binding are created by running:
kubectl get role role-full-access-to-secrets -n default
kubectl get rolebinding default-role-binding -n default
Step 3: Create a CronJob that runs a script to update the ECR token secret
The final step is to create a CronJob that runs a script every 10 hours (or any other interval you prefer) to update the ECR token secret. The script uses AWS CLI to get a new ECR token and then uses kubectl to delete and create the secret with the token.
To create the CronJob, save the following content as ecr-registry-helper-cronjob.yaml
:
apiVersion: batch/v1
kind: CronJob
metadata:
name: ecr-registry-helper
namespace: default
spec:
schedule: "0 */10 * * *" # Replace with your desired schedule
successfulJobsHistoryLimit: 2
suspend: false
jobTemplate:
spec:
template:
spec:
serviceAccountName: sa-default # Replace with your service account name if different
containers:
- name: ecr-registry-helper
image: omarxs/awskctl:v1.0
imagePullPolicy: IfNotPresent
envFrom:
- secretRef:
name: ecr-registry-helper-secrets # Replace with your secret name if different
- configMapRef:
name: ecr-registry-helper-cm # Replace with your configmap name if different
command:
- /bin/bash
- -c
- |-
ECR_TOKEN="$(aws ecr get-login-password --region ${AWS_REGION})"
NAMESPACE_NAME=default # Replace with your desired namespace
kubectl delete secret --ignore-not-found $DOCKER_SECRET_NAME -n $NAMESPACE_NAME
kubectl create secret docker-registry $DOCKER_SECRET_NAME --docker-server=https://${AWS_ACCOUNT}.dkr.ecr.${AWS_REGION}.amazonaws.com --docker-username=AWS --docker-password=${ECR_TOKEN} --namespace=$NAMESPACE_NAME
echo "Secret was successfully updated at $(date)"
restartPolicy: Never
Then, apply the file using:
kubectl apply -f ecr-registry-helper-cronjob.yaml
You can verify that the CronJob is created by running:
kubectl get cronjob ecr-registry-helper -n default
Step 4: Test and verify the CronJob
The last step is to test and verify that the CronJob is working as expected and regularly updating the ECR token secret.
You can do this by manually triggering the CronJob and checking the logs of the CronJob pods.
To manually trigger the CronJob, run the following command:
kubectl create job --from=cronjob/ecr-registry-helper ecr-registry-helper-manual # Replace with your CronJob name if different
This will create a one-time job that runs the script and updates the ECR token secret. You can check the status and logs of the job by running:
kubectl get job ecr-registry-helper-manual -n default # Replace with your job name and namespace if different
kubectl logs -n default -f job/ecr-registry-helper-manual # Replace with your job name and namespace if different
You should see something like:
Secret was successfully updated at Sat Jan 1 00:00:00 UTC 2023 # Replace with your actual date and time
You can delete the job after verifying it by running:
kubectl delete job ecr-registry-helper-manual -n default # Replace with your job name and namespace if different
You can also use the secret to pull images from ECR by creating a pod or a deployment that references the secret. For example, you can save the following content as test-pod.yaml
:
apiVersion: v1
kind: Pod
metadata:
name: test-pod
namespace: default
spec:
containers:
- name: test-container
image: <your-account>.dkr.ecr.<your-region>.amazonaws.com/<your-image>:<your-tag> # Replace with your ECR image
imagePullPolicy: Always
imagePullSecrets:
- name: regcred # Replace with your secret name if different
Then, apply the file using:
kubectl apply -f test-pod.yaml
You can verify that the pod is running and pulling the image from ECR by running:
kubectl get pod test-pod -n default
kubectl describe pod test-pod -n default
Alternative Step: Run the whole manifest at once
If you want to run the whole manifest at once, you can combine the files ecr-registry-helper.yaml
, ecr-registry-helper-rbac.yaml
, and ecr-registry-helper-cronjob.yaml
into one file, such as ecr-registry-helper-all.yaml
.
Make sure to replace the values with your own.
apiVersion: v1
kind: Secret
metadata:
name: ecr-registry-helper-secrets
namespace: default
stringData:
AWS_SECRET_ACCESS_KEY: "" # Replace with your AWS secret access key
AWS_ACCESS_KEY_ID: "" # Replace with your AWS access key ID
AWS_ACCOUNT: "" # Replace with your AWS account ID
---
apiVersion: v1
kind: ConfigMap
metadata:
name: ecr-registry-helper-cm
namespace: default
data:
AWS_REGION: "eu-central-1" # Replace with your ECR region
DOCKER_SECRET_NAME: regcred # Replace with your desired ECR token secret name
---
apiVersion: v1
kind: ServiceAccount
metadata:
name: sa-default
namespace: default
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
namespace: default
name: role-full-access-to-secrets
rules:
- apiGroups: [""]
resources: ["secrets"]
resourceNames: ["regcred"] # Replace with your desired ECR token secret name
verbs: ["delete"]
- apiGroups: [""]
resources: ["secrets"]
verbs: ["create"]
---
kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: default-role-binding
namespace: default
subjects:
- kind: ServiceAccount
name: sa-default # Replace with your service account name if different
namespace: default
apiGroup: ""
roleRef:
kind: Role
name: role-full-access-to-secrets # Replace with your role name if different
apiGroup: ""
---
apiVersion: batch/v1
kind: CronJob
metadata:
name: ecr-registry-helper
namespace: default
spec:
schedule: "0 */10 * * *" # Replace with your desired schedule
successfulJobsHistoryLimit: 2
suspend: false
jobTemplate:
spec:
template:
spec:
serviceAccountName: sa-default # Replace with your service account name if different
containers:
- name: ecr-registry-helper
image: omarxs/awskctl:v1.0
imagePullPolicy: IfNotPresent
envFrom:
- secretRef:
name: ecr-registry-helper-secrets # Replace with your secret name if different
- configMapRef:
name: ecr-registry-helper-cm # Replace with your configmap name if different
command:
- /bin/bash
- -c
- |-
ECR_TOKEN="$(aws ecr get-login-password --region ${AWS_REGION})"
NAMESPACE_NAME=default # Replace with your desired namespace
kubectl delete secret --ignore-not-found $DOCKER_SECRET_NAME -n $NAMESPACE_NAME
kubectl create secret docker-registry $DOCKER_SECRET_NAME --docker-server=https://${AWS_ACCOUNT}.dkr.ecr.${AWS_REGION}.amazonaws.com --docker-username=AWS --docker-password=${ECR_TOKEN} --namespace=$NAMESPACE_NAME
echo "Secret was successfully updated at $(date)"
restartPolicy: Never
To apply the file, run:
kubectl apply -f ecr-registry-helper-all.yaml
This will create all the resources needed for the CronJob in one go. You can verify that they are created by running:
kubectl get secret ecr-registry-helper-secrets -n default
kubectl get configmap ecr-registry-helper-cm -n default
kubectl get serviceaccount sa-default -n default
kubectl get role role-full-access-to-secrets -n default
kubectl get rolebinding default-role-binding -n default
kubectl get cronjob ecr-registry-helper -n default
Conclusion
In this article, you have learned how to set up a CronJob to update the ECR token secret in Kubernetes with CronJobs automatically. This way, you donβt have to worry about token expiration and can focus on our application development.
You have also learned how to test and verify that the CronJob is working as expected and regularly update the ECR token secret.
I hope you found this article useful and informative. If you have any questions or feedback, please feel free to leave a comment below. Thank you for reading!
LHB Community is made of readers like you who like to contribute to the portal by writing helpful Linux tutorials.