When the Captain Turns the Helm: a 7 Steps Migration
Tags: devops, helm, migration, legacy
Published: 2022-04-10 13:09:00

🚢 Shall we continue with sea allegories? I made an amazing picture to attract attention.

Large organizations are often quite conservative when it comes to upgrading to the latest software versions. 

Of course, it is very important to be sure that everything works clearly and correctly, especially considering Helm's wide use in CI. Fear of migration is not so unfounded. We're not sailors onboard the RMS Titanic [who didn't have binoculars](https://www.thevintagenews.com/2017/01/05/the-titanic-crew-had-no-binoculars-which-might-have-helped-them-spot-the-iceberg/), are we? You should turn the helm, understanding which way the ship will turn! But first things first! 🧊

Helm v2 became legacy in late 2020, but there are companies that still use version 2 in their practices these days. 

> There are not many short and clear summaries on the Internet about migrating to Helm v3. Well, in this note, the author will try to break everything down into points.

### ⚓ Stage 0. Backup

Backup all the existing releases! It’s important to take a backup of all the releases before starting with the migration. Though the first step of migration won’t delete Helm v2’s release information, it’s always better to take a backup.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/qs0wekzr003msmh8t20j.png)

By default, Helm v2 stores its releases' data as ConfigMaps in `kube-system` namespace of the cluster. This command is to list Helm v2 releases:

```
$ kubectl get configmap -n kube-system -l "OWNER=TILLER"
```

Helm v3 _changed_ the default release information storage to Secrets in the namespace of the release. How to list Helm v3 releases:

```
$ kubectl get secret --namespace my-namespace | grep sh.helm.release

```
Using [helm-backup plugin](https://github.com/maorfr/helm-backup) by Maor maorfr Friedman. The helm-backup is a Helm v2 plugin which can take backup of releases and also has the ability to restore them. Here is how it achieves the backup.

It finds the storage method used by Tiller, then backs up the ConfigMaps along with release names as a `.tar` file. While doing the restore, it first applies the ConfigMaps. Then, it tries to find the release with `STATUS=DEPLOYED` label, gets the manifest (YAML) for that release and then applies it.

Another way is to just backup all the ConfigMaps which hold the release information. The easiest way to do it is to run the following command: 

```
$ kubectl get configmaps \
    --namespace "kube-system" \
    --selector "OWNER=TILLER" \
    --output "yaml" > helm2-backup-cm.yaml
```

### ⚓ Stage 1: Converting releases

When we install or update a chart, it creates one Helm release in the cluster. 

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/kqsplqe32vo7vcqebtem.png)

It is stored as a `ConfigMap` within the cluster in case of Helm v2. This makes it possible to do the rollback and keeps a history. The format in which Helm v3 stores the release information is different from v2; v3 uses Secrets predominately as a storage. The 2to3 plugin written by Martin Hickey will do the work of converting these releases to the new format. 

⚠️ An important note about access to the [tiller namespace for required RBAC roles](https://v2.helm.sh/docs/rbac/). If Tillerless setup, then a service account with the proper cluster wide RBAC roles will need to be used. If not used, forbidden errors will be thrown when trying to access restricted resources.

Install 2to3 plugin:

```
$ helm3 plugin install https://github.com/helm/helm-2to3
```

### ⚓ Stage 2. Dumping Helm v2 (Tiller) releases:

It is good to have this list for your confidence.

```
$ helm2 tiller run -- helm2 ls --max=0 | sed -n '1!p' | awk 'FNR > 1 { print $1 }' > releases.log
$ kubectl -n kube-system get configmap -l OWNER=TILLER,STATUS=DEPLOYED | awk 'FNR > 1 { print $1 }' | cut -f1 -d"." > releases.log
```

### ⚓ Stage 3. Launching an automated migration with a bash script:

```bash
#!/bin/bash

helm3_cmd="helm3"
if [[ -x "$(which helm 2>/dev/null)" ]]; then
  helm2_releases="$(helm ls --all --short)"
else
  echo "'helm' is not installed or not present in PATH. Using kubectl to get list of releases."
  # …
fi

echo -e "Found the following releases:\n${helm2_releases}\n"

for release in ${helm2_releases}; do
  ${helm3_cmd} 2to3 convert --dry-run "${release}"
  read -p "Convert '${release}'? [Y/n] " user_choice
  if [[ "${user_choice}" == "Y" ]]; then
    echo "Converting '${release}'."
    ${helm3_cmd} 2to3 convert "${release}"
    echo "Converted '${release}' successfully."
  else
    echo "Skipping conversion of '${release}'."
  fi
done
```
The original [script](https://gitlab.com/-/snippets/1953002) written by Bhavin Gandhi.

### ⚓ Stage 4. Resolving Kubernetes API validation problem

Helm v3 validates all of your rendered templates against the Kubernetes OpenAPI schema. This approach supposes your rendered templates _are validated before they are sent to the Kubernetes API_. That means any invalid fields will trigger a non-zero exit code and fail the release. Lack of fields will lead to the same.

The labels and annotations [which had not been added to migrated Kubernetes objects] should be added manually:

```
$ kubectl -n ${NAMESPACE} label deployment -l "app.kubernetes.io/instance=${RELEASE}" "app.kubernetes.io/managed-by=Helm"
$ kubectl -n ${NAMESPACE} annotate deployment -l "app.kubernetes.io/instance=${RELEASE}" "meta.helm.sh/release-name=${RELEASE}" "meta.helm.sh/release-namespace=${NAMESPACE}"
```

⚠️ Example of needed labels & annotations which are valid for stage 4:

```yaml
metadata:
  labels:
    app.kubernetes.io/instance: {{ .Release.Name }}
    app.kubernetes.io/managed-by: Helm
  annotations:
    meta.helm.sh/release-name: {{ .Release.Name }}
    meta.helm.sh/release-namespace: {{ .Release.Namespace }}
```

If before we relied entirely on the version 2 of Helm, then we have Tiller installed in our cluster, so we may omit `--tiller-out-cluster flag`. The safest way of course to start with `--dry-run` flag (for simulation only).

### ⚓ Stage 5. Checking out whether it was successful

Helm v3 now behaves similarly to `kubectl`. With Helm v2, you could query all releases across all namespaces simply by typing `helm2 ls`. With Helm v3, running `helm3 list` will only show releases in the default namespace, just like `kubectl get pods` would. 

To list releases in any non-default namespace, you must supply the `-n` argument and the namespace, just as you would when using kubectl. 

```
$ helm3 list -n <namespace>
```

### ⚓ Stage 6. Helm v2 cleanup
As we did not specify `--delete-v2-releases flag`, Helm v2 releases information was left in-tact, it can be deleted with helm3 2to3 cleanup later on.

### ⚓ Stage 7. Bye-bye Tiller

After all, we may remove Tiller from our cluster:

```
$ kubectl delete deployment tiller-deploy --namespace kube-system
```

### 🔱 Note A. Check your CIs: CLI commands renamed

Some CLI commands have been renamed in Helm v3

`helm2 delete` is renamed to `helm3 uninstall`, but the first one is still retained as an alias to `helm3 uninstall`. Also, in Helm v2, the `--purge` flag removes the release configs. In Helm v3, this feature _is enabled by default_. To retain the previous behavior, you can use the command `helm3 uninstall --keep-history`.

`helm2 inspect` command is renamed to `helm3 show`.
`helm2 fetch` command is renamed to `helm3 pull`.

### 🔱 Note B. Check your YAMLs

When chart's Kubernetes `apiVersion` is set to `v2` it signifies that it is using `v2` _schema that is introduced in Helm v3_. Therefore, `apiVersion: v2` is not rendered in Helm v2 because of this. If you need more compatibility, stay on `v1` here.

Please see my [post](https://www.linkedin.com/posts/roman-balashevich_helm2-vs-helm3-part-1-activity-6916727858786635776-HfK-) on LinkedIn for references. Pic source: Padok 🇫🇷

Painless migration to you!
Happy charting!