Helm 3 - CI/CD with Azure DevOps using Azure Container Registry (ACR) and Azure Kubernetes Service (AKS)
Tags: kubernetes, aks, azuredevops, helm
Published: 2021-02-26 23:06:59

---
title: Helm 3 - CI/CD with Azure DevOps using Azure Container Registry (ACR) and Azure Kubernetes Service (AKS)
published: true
date: 2021-01-30 00:00:00 UTC
tags: 
- kubernetes
- aks
- azuredevops
- helm
canonical_url: https://www.visualstudiogeeks.com/devops/helm/deploying-helm-chart-with-azdo
---

I have been bit late to Kubernetes world, but ever since I have started using it, I have been part of a great teams building awesome applications and using Helm package manager. With Helm, you package your Kubernetes application as charts, which are then stored in Helm chart repo. Helm also has a templating engine allowing you to set values in your charts dynamically allowing you to manage your applications more easily. Azure Container Registry (ACR) currently supports publishing Helm 3 charts to ACR and it is currently in preview.

In this post we will see how we can publish a sample Helm chart to ACR and also deploy the application to Azure Kubernetes Service (AKS) by consuming the published chart from ACR. We will also use ACR’s repository scoped tokens - a preview feature which offer great benefits.<!--more-->

We will cover following things in this post.

**1) Prerequisites**

- ACR 
  - Setting up the ACR to use repository scoped tokens
  - Create scope-maps
  - Create a token
- AKS 
  - Create AKS instance on Azure

For CI and CD, I am going to use Azure DevOps YAML multi-stage pipelines in this post and our pipeline will be divided as below.

**2) CI Stage**

- Get the latest chart from GitHub
- Publish to ACR

**3) CD Stage**

- Pull the latest version of chart from ACR
- Deploy the pulled to AKS
- View the deployed service in Azure DevOps

![](https://www.visualstudiogeeks.com/images/screenshots/utkarsh/2021-01-30-deploying-helm-chart-with-azdo/featureimage.png)

The code I am using is on [![GitHub User](https://img.shields.io/badge/github-azure--vote--helm--chart-brightgreen?logo=github,github-actions)](https://github.com/utkarshpoc/azure-vote-helm-chart)

## 1. Prerequisites

You can authenticate with ACR in multiple ways.

1. **Using a service principal** - You can create separate service principals. This gives access to whole ACR and all the repositories inside that ACR instance. More info [here](https://docs.microsoft.com/en-gb/azure/container-registry/container-registry-auth-service-principal)

2. **Using managed identity** - Using user-assigned or system-assigned managed identity for easily consuming charts on an Azure VM. More info [here](https://docs.microsoft.com/en-gb/azure/container-registry/container-registry-authentication-managed-identity)

3. **Using image pull secret** - For your Kubernetes (K8s) cluster (including unmanaged AKS instance), you can also define a image pull secret, which then lets K8s cluster do pull images and run the application. More info on how you can do this with AKS is [here](https://docs.microsoft.com/en-gb/azure/container-registry/container-registry-auth-kubernetes)

4. **Using a repository scoped token** - This feature of ACR is currently in _preview_ and only available on **Premium** container registry service tier. This allows you to define scope maps and repository specific tokens for your ACR.

### Setting up the ACR to use repository scoped tokens

I am going to use **Tokens** to authenticate ourselves with ACR. The biggest advantage is that, as a registry owner, you can have a single instance of ACR across your organisation and provide access to certain teams only selected repositories. There are other interesting scenarios highlighted in the [documentation](https://docs.microsoft.com/en-gb/azure/container-registry/container-registry-repository-scoped-permissions), mainly…

- Allow IoT devices with individual tokens to pull an image from a repository
- Provide an external organisation with permissions to a specific repository
- Limit repository access to different user groups in your organisation. For example, provide write and read access to developers who build images that target specific repositories, and read access to teams that deploy from those repositories.

If you have an existing ACR instance (which in my case I do), the first step is to upgrade your ACR instance to use `Premium` tier.

![](https://www.visualstudiogeeks.com/images/screenshots/utkarsh/2021-01-30-deploying-helm-chart-with-azdo/acr-upgrade.jpg)

Then select `Premium` and click `Save`.

![](https://www.visualstudiogeeks.com/images/screenshots/utkarsh/2021-01-30-deploying-helm-chart-with-azdo/acr-upgrade-premium.jpg)

### Creating scope maps

Once, you are on Premium tier, you will be able to to create a tokens. But before we can create tokens, let us define two scope maps. Scope maps when associated with tokens sets permissions on what can be done on ACR.

1. **chartpull** - a scope limiting read permission to `helmdemo/vote-app` repo. I am using `content/read` and `metadata/read` permissions.
2. **chartpush** - a scope limiting write permission inside `helmdemo/vote-app` repo. I am using `content/write` and `content/read` permissions.

The idea here is that, development team pushes the charts to ACR and some other team is consuming the charts/application. But you can extend this idea to let users external to your organisation to pull images only from specific repositories.

So to create scope map, go to `Scope maps` under `Repository permissions` sections and then click `+ Add`. In my case, I am limiting these scopes only to `helmdemo` repo.

![](https://www.visualstudiogeeks.com/images/screenshots/utkarsh/2021-01-30-deploying-helm-chart-with-azdo/scope-map.jpg)

I follow the same process to create another scope named `chartpull` scope with `content/read`, `metadata/read` scopes.

> - Notice that in `chartpush` scope we also allow `content/read` permission. This is because, you need `content/read` scope along with `content/write` scope if you are going to push charts to ACR (for the repo defined in the scope).
> - You can also use Azure CLI to do the same. Use `az acr token` command. Refer the [documentation](https://docs.microsoft.com/en-us/cli/azure/acr/token?view=azure-cli-latest) for more information.

### Creating token

Once you defined scopes, you are ready to create tokens. Go to `Tokens` in `Repository permissions` section and click `+ Add`. Then name the token (e.g. `helmdemopull`) and select the scope map you created above. In my case I am creating two tokens, one for pull and another one for push.

![](https://www.visualstudiogeeks.com/images/screenshots/utkarsh/2021-01-30-deploying-helm-chart-with-azdo/create-token.jpg)

You can repeat the process to create another token. Refreshing the screen you will see something like this below.

![](https://www.visualstudiogeeks.com/images/screenshots/utkarsh/2021-01-30-deploying-helm-chart-with-azdo/tokens.jpg)

You will notice that these tokens (`helmdemopull` and `helmdemopush`) do not yet have passwords generated. Lets create passwords. Click on the token, click the icon under `Actions` column for either `password1` or `password2`. You will also have an option to set an expiration date for the password if you planning to allow token to be valid only for specific days.

![](https://www.visualstudiogeeks.com/images/screenshots/utkarsh/2021-01-30-deploying-helm-chart-with-azdo/password1.jpg)

## 2. CI Stage

### Get the latest chart from GitHub

To get the latest source from the GitHub repo, in our YAML pipeline we add a `resources` section and point to the repo on GitHub. We will also need to pass the `endpoint` parameter with the name of the GitHub service connection. More info about how to do this is [here](https://docs.microsoft.com/en-us/azure/devops/pipelines/yaml-schema?view=azure-devops&tabs=schema,parameter-schema#repository-resource) and [here](https://docs.microsoft.com/en-us/azure/devops/pipelines/yaml-schema?view=azure-devops&tabs=schema,parameter-schema#repository-resource).

```yml
resources:
 repositories:
   - repository: helmrepo
     type: github
     name: utkarshpoc/azure-vote-helm-chart
     endpoint: github
```

### Publish chart to ACR

Before we write steps to publish to ACR, we need to ensure we store the tokens we generated in the pipeline as variables. Here I am adding tokens as variables using the variables UI of the pipeline and marking passwords as secret.

![](https://www.visualstudiogeeks.com/images/screenshots/utkarsh/2021-01-30-deploying-helm-chart-with-azdo/add-variables.jpg)

> You can also define these in Azure Pipelines [Library](https://docs.microsoft.com/en-us/azure/devops/pipelines/library/?view=azure-devops) if you intend to use these tokens in multiple pipelines.

I also have few static and non-secret variables in YAML itself so that they are source controlled using `variables` as below.

```yml
variables:
  acr.name: acrdemoutkarsh
  acr.repo.name: helmdemo/vote-app
```

The steps involved for publishing our Helm chart are simple.

1. Installing Helm 3 on the agent
2. Login to the ACR using Helm
3. Save and push the chart

#### 1. Installing Helm 3 on the agent

You can do this using OOB `HelmInstaller` task. The YAML is as below. Here I am using `latest` for `helmVersion`, but you can stick specific to a version (3.x and above).

```yml
- task: HelmInstaller@0
  displayName: install helm
  inputs:
    helmVersion: 'latest'
    installKubectl: false
```

### 2. Login to the ACR using Helm

We can use simple `script` step to login to the registry.

```yml
- script: |
    helm registry login $(acr.name).azurecr.io --username $(acr.push.username) --password $(acr.push.password)
  displayName: login to acr using helm
```

But if you run this, at the time of writing you will get an error as below

```txt
Error: this feature has been marked as experimental and is not enabled by default. 
Please set HELM_EXPERIMENTAL_OCI=1 in your environment to use this feature
```

As the error states, this is because, publishing Helm charts which follow Open Container Initiative (OCI) standard is experimental and you are required to set `HELM_EXPERIMENTAL_OCI` to `1`. We can do this by updating our `variables` section in the YAML. The updated yaml looks as below.

```yml
variables:
  acr.name: acrdemoutkarsh
  acr.repo.name: helmdemo/vote-app
  HELM_EXPERIMENTAL_OCI: 1
```

#### 3. Save and push the chart to ACR

Next step is to save this chart locally and create an alias for the chart with our ACR registry URL. So our YAML looks as below.

```yml
- script: |
    helm chart save $(build.sourcesdirectory)/src/azure-vote-helm-chart/ $(acr.name).azurecr.io/$(acr.repo.name):latest
  displayName: save the chart and set the alias
```

And finally, we push the Helm chart to ACR using `helm chart push` command.

```yml
- script: |
    helm chart push $(acr.name).azurecr.io/$(acr.repo.name):latest
  displayName: push the chart to acr
```

So our CI stage is now complete.

## 2. CD Stage

### Pull the latest version of chart from ACR

The first step is to pull the latest version of the chart (one with tag `latest`) to our agent machine and extract it in a folder. We can do that in pipeline as below. Notice I have added a new stage name `cd` and its dependent of stage `ci`.

```yml
- stage: cd
    displayName: CD
    dependsOn: ci
    jobs:
    - deployment: helm_publish_aks
      displayName: deploy to aks
      environment: 
        name: PROD
        resourceName: helmdemo
        resourceType: Kubernetes
      strategy:
        runOnce:
          deploy:
            steps:
            - task: HelmInstaller@0
              displayName: install helm
              inputs:
                helmVersion: 'latest'
                installKubectl: false

            - script: |
                echo "$(acr.pull.password)" | helm registry login $(acr.name).azurecr.io --username $(acr.pull.username) --password-stdin
              displayName: login to acr using helm

            - bash: |
                  helm chart pull $(acr.name).azurecr.io/$(acr.repo.name):latest
              displayName: get helm chart on agent
            
            - bash: |
                helm chart export $(acr.name).azurecr.io/$(acr.repo.name):latest --destination $(build.stagingdirectory)
              displayName: export the chart to folder
```

> Because each job in Azure DevOps run in a separate agent, I have to ensure agent has Helm tool, so install Helm tool again in the first step. Also, note that I am using token with scope-map permission set only to pull charts.

### Deploy the chart to AKS

Now that the latest chart is pulled from our ACR and is available on agent, only step remaining is to deploy to AKS. We can do that using `HelmDeploy` task.

```yml
- task: HelmDeploy@0
  displayName: deploy chart to aks
  inputs:
    connectionType: 'Azure Resource Manager'
    azureSubscription: '$(azure.service.connection)'
    azureResourceGroup: 'demos'
    kubernetesCluster: 'aksdemoutkarsh'
    namespace: 'helmdemo'
    command: 'upgrade'
    chartType: 'FilePath'
    chartPath: '$(build.stagingdirectory)/azure-vote/'
    releaseName: 'helmdemo'
    arguments: '--create-namespace --install'
```

### View the deployed service in Azure DevOps

If you have added your [Kubernetes cluster as a resource](https://docs.microsoft.com/en-us/azure/devops/pipelines/process/environments-kubernetes?view=azure-devops) in Azure DevOps environment, you can now view deployed services directly from Azure DevOps.

![](https://www.visualstudiogeeks.com/images/screenshots/utkarsh/2021-01-30-deploying-helm-chart-with-azdo/pipeline-env.jpg)

You can also view deployed services in YAML and also see any ports exposed.

![](https://www.visualstudiogeeks.com/images/screenshots/utkarsh/2021-01-30-deploying-helm-chart-with-azdo/pipeline-service.jpg)

Browsing the exposed port, you will see application.

![](https://www.visualstudiogeeks.com/images/screenshots/utkarsh/2021-01-30-deploying-helm-chart-with-azdo/browse-app.jpg)

### Conclusion

That is it! If you have read this far, thank you 🙏🏼. As we saw, with the support of tokens and OCI artifacts, ACR is one of the best in class container registry. Also, with deep integration with AKS (features like browsing services, logs, yaml), Azure DevOps brings in lots of productivity for your teams. Hope you found this post informative. If so, please share and tweet!