Secure builds on Kubernetes with Kaniko (AWS ECR)
Tags: aws, kaniko, kubernetes, gitlab
Published: 2023-08-23 10:27:52


![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/sjt899tfmjkbcjq7psvl.png)


Kaniko is a suitable choice for scenarios where security, isolation, compatibility with container orchestration platforms, and efficient resource utilization are important factors. It provides an alternative to traditional Docker builds, particularly in environments where running Docker daemons might not be optimal or secure.

All the files are included in [gitlab](https://gitlab.com/angjelo/ecr-kaniko-builds)

Prerequisites:

- Kubernetes Cluster
- Helm Installed
- gitlab-registration token
- IAM User from AWS with full ECR ACCESS

Lets start by adding the AWS credentials as variables into our repository

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/tt5fibd3d19afsjr0ou9.png)


The helm repository will enable us to use the gitlab-runner chart
```bash
helm repo add gitlab https://charts.gitlab.io
```

###Gitlab runner config yaml

A deeper overview of the gitlab configuration is located in the official repo ["https://gitlab.com/gitlab-org/charts/gitlab-runner/-/blob/main/values.yaml"] 

```yaml
gitlabUrl: https://gitlab.com
runnerRegistrationToken: $REGISTRATION_TOKEN
concurrent: 2
rbac:
  create: true
  name: gitlab-runner
runners:
  tags: kube,aws
  runUntagged: true
  imagePullPolicy: if-not-present
```

- `gitalb-runner/values.yml` contains the values to overwrite in the helm repostiry
- `tags: kube,aws` can be anything that makes sense to you
- `runnerRegistrationToken` should be obtained from the CI/CD section.

Create a namespace for gitlab: `kubectl create ns gitlab`

### Using helm to instal gitlab-runner
```bash
helm install --namespace gitlab gitlab-runner -f values.yml gitlab/gitlab-runner
```

`kubectl get pods -n gitlab` should have active gitlab pods

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/ouxriq04axja9vzr9vfr.png)

Now if you navigate to the runners section in your gitlab, there should be a new runner registered:

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/0nny75i0l9pk08fpwi7v.png)

 

For the project build you can use the `Dockerfile` & `index.html` from my gitlab-repoisitory

### .gitlab-ci.yml

To further understand kaniko please reference to the documentation [kaniko Gitlab](https://docs.gitlab.com/ee/ci/docker/using_kaniko.html)

```yaml
stages:
  - build

variables:
  AWS_REGION: eu-north-1
  REGISTRY: "${AWS_ACCOUNT_ID}.dkr.ecr.eu-north-1.amazonaws.com"

build:
  stage: build
  image:
    name: gcr.io/kaniko-project/executor:debug
    entrypoint: [""]
  script:
    - mkdir -p /kaniko/.docker
    - echo "{\"credsStore\":\"ecr-login\",\"credHelpers\":{\"$REGISTRY/contoso\":\"ecr-login\"}}" > /kaniko/.docker/config.json
    - >-
      /kaniko/executor
      --context "${CI_PROJECT_DIR}" 
      --dockerfile "${CI_PROJECT_DIR}/Dockerfile" 
      --build-arg AWS_REGION=$AWS_REGION 
      --build-arg AWS_ACCESS_KEY_ID=$AWS_ACCESS_KEY_ID 
      --build-arg AWS_SECRET_ACCESS_KEY=$AWS_SECRET_ACCESS_KEY 
      --destination "${REGISTRY}/contoso:${CI_COMMIT_TAG:-$CI_COMMIT_REF_SLUG}"
  tags:
    - aws,kube
```
`"{\"credsStore\":\"ecr-login\",\"credHelpers\":{\"$REGISTRY/contoso\":\"ecr-login\"}}"`:

- `credsStore`: This specifies the Docker credentials store to be used for authentication. In this case, it's set to "ecr-login", in which the Amazon Elastic Container Registry (ECR) authentication mechanism is being used.

- `credHelpers`: Is an object that defines custom credential helpers for specific Docker registries. In this case, a custom helper is being set up for the $REGISTRY/contoso registry.

Once we push gitlab-ci.yml and trigger the pipeline we see the build is running, it Could take up to 30 secs for the runner to pick up the job

You can see the full pipeline[pipeline log](https://gitlab.com/angjelo/ecr-kaniko-builds/-/jobs/4929396368):  
![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/z9b7zn3tqrsqjqh5zh1c.png)

Navigate to the ECR repository to confirm the image build

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/yvtd6lm2th29sd36jr97.png)

Now that you have the pipeline all working start experimenting and see what you can do :)