Integrating AWS ECR(Elastic Container Registry) with Kubernetes
Tags: kubernetes, aws, python
Published: 2020-05-31 16:02:29

I run my own kubernetes cluster spun up using Rancher on AWS lightsail which is an alternative to DigitalOcean. Though Lightsail is part of AWS, its not tightly as integrated as the rest of AWS. The ECR docker image token(or password) expires every 12 hours, and everytime you want to pull or push you have to renew it. To use it with kubernetes you need someway to update the secret automatically every 12 hours.
Getting ECR to work with it is like as same as any other non AWS(or EKS) cluster. You may read further if you want to integrate it with your DIY or other non AWS kubernetes clusters.
I dockerized a lightweight python script to run as a cron job, which will fetch a new login token every 6 hours(based on your deployment.yaml).
First create a secret that holds your AWS credentials with
```bash
kubectl create secret -n ecr-kube-helper generic ecr-kube-helper-ecr-secret --from-literal=REGION=[AWS_REGION] --from-literal=ID=[AWS_KEY_ID] --from-literal=SECRET=[AWS_SECRET]
```
Lets begin by creating a service account.
```yaml
apiVersion: v1
kind: Namespace
metadata:
  name: ecr-kube-helper
  labels:
    name: ecr-kube-helper
---
apiVersion: v1
kind: ServiceAccount
metadata:
  namespace: ecr-kube-helper
  name: svac-ecr-kube-helper
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: role-ecr-kube-helper
  namespace: ecr-kube-helper
rules:
  - apiGroups: [""]
    resources: ["secrets"]
    verbs: ["get","delete", "create"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  namespace: ecr-kube-helper
  name: rb-ecr-kube-helper
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: role-ecr-kube-helper
subjects:
  - kind: ServiceAccount
    name: svac-ecr-kube-helper
    namespace: ecr-kube-helper
```
Then deploy it with a CronJob deployment, but be sure to change the environment values.
```yaml
apiVersion: batch/v1beta1
kind: CronJob
metadata:
  namespace: ecr-kube-helper
  name: cron-ecr-kube-helper
  labels:
    app: cron-ecr-kube-helper
spec:
  schedule: "0 */6 * * *"
  successfulJobsHistoryLimit: 3
  failedJobsHistoryLimit: 5
  jobTemplate:
    spec:
      template:
        spec:
          restartPolicy: OnFailure
          serviceAccountName: svac-ecr-kube-helper
          volumes:
            - name: svac-ecr-kube-helper-token-dr9bg
              secret:
                secretName: svac-ecr-kube-helper-token-dr9bg
          containers:
            - name: pod-ecr-kube-helper
              image: anaganisk/ecr-kube-helper:1.0.0
              imagePullPolicy: IfNotPresent
              volumeMounts:
                - mountPath: /var/run/secrets/kubernetes.io/serviceaccount
                  name: svac-ecr-kube-helper-token-dr9bg
              env:
                - name: AWS_DEFAULT_REGION
                  valueFrom:
                    secretKeyRef:
                      # AWS credientials secret
                      name: ecr-kube-helper-ecr-secret
                      key: REGION
                - name: AWS_ACCESS_KEY_ID
                  valueFrom:
                    secretKeyRef:
                      # AWS credientials secret
                      name: ecr-kube-helper-ecr-secret
                      key: ID
                - name: AWS_SECRET_ACCESS_KEY
                  valueFrom:
                    secretKeyRef:
                      # AWS credientials secret
                      name: ecr-kube-helper-ecr-secret
                      key: SECRET
                - name: LOGLEVEL
                  value: INFO
                - name: TARGET_SECRET
                  value: xxxSecretxxx
                - name: TARGET_ECR
                  value: "xxxECR_REPOxxx"
                - name: TARGET_NAMESPACE
                  value: "ecr-kube-helper"
                - name: TARGET_EMAIL
                  value: "docker@example.com"

```
And you’re done. Let it handle the rest.
The helper can only update one ECR ID for now so, if you want to use it with multiple ECRs you may have to create multiple cronjobs. One ECR ID may have multiple repositories
for example ECR_ID.dkr.ecr.ap-south-1.amazonaws.com/repository

Github Page https://anaganisk.github.io/ecr-kube-helper/