Exploring the HelmChart custom resource in k3s
Tags: kubernetes, k3s, helm, rancher
Published: 2019-09-08 18:24:40

[k3s](https://k3s.io/) is a lightweight [Kubernetes](https://kubernetes.io/) distribution developed by [Rancher](https://rancher.com/) which is suitable for edge, <abbr title="Internet of Things">IoT</abbr>, and resource-constrained computing environments.

[Helm](https://helm.sh/) is a package manager for Kubernetes which enables users to find, share, and use software built for Kubernetes.

Traditionally, Helm is comprised of a client (`helm`) and server-side component, [Tiller](https://helm.sh/docs/glossary/#tiller), which manages the deployment and lifecycle of Helm charts. In k3s, Tiller is replaced by [Helm Controller](https://github.com/rancher/helm-controller). Helm Controller defines a new `HelmChart` [custom resource definition](https://kubernetes.io/docs/concepts/extend-kubernetes/api-extension/custom-resources/), or <abbr title="Custom Resource Definition">CRD</abbr>, for managing Helm charts.

You can verify the `HelmChart` resource is present on a k3s cluster by using `kubectl` to list the available API resources.

```console
 $ sudo k3s kubectl api-resources --api-group=helm.cattle.io
NAME         SHORTNAMES   APIGROUP         NAMESPACED   KIND
helmcharts                helm.cattle.io   true         HelmChart
```

## The HelmChart resource

Below is an example of a `HelmChart` resource definition.

```yaml
---
apiVersion: helm.cattle.io/v1
kind: HelmChart
metadata: # Kubernetes object metadata
  name: # Chart name (required)
  namespace: kube-system # Namespace Helm Controller is monitoring to deploy charts (required)
spec: # resource specification
```

The `apiVersion` and `kind` fields together identify this as a HelmChart resource to be managed by Helm Controller.

The `name` field under `metadata` specifies a name for the HelmChart resource. `namespace` specifies the [Kubernetes namespace](https://kubernetes.io/docs/concepts/overview/working-with-objects/namespaces/) in which Helm Controller is monitoring CRDs. For k3s, this must be `kube-system` as Helm Controller is only configured to watch this namespace for new HelmChart resources.

Lastly, the `spec`, or *specification*, field is used to specify the desired state of the HelmChart resource.

### HelmChart resource `spec`

The HelmChart resource supports several configuration [parameters](https://github.com/rancher/helm-controller/blob/3e223ca9dc94607ea9b7deb7f632551230c4db32/pkg/apis/helm.cattle.io/v1/types.go#L20-L25). These parameters correspond to `helm install` command line arguments and are passed to Helm during chart deployment.

* `chart` (string: null) - Chart reference or a URL.
* `targetNamespace` (string: null) - Namespace to install the Chart into. Defaults to the namespace used by the k3s controller (i.e., kube-system) if omitted.
* `version` (string: null) The exact Chart version to install. If this is not specified, the latest version is installed.
* `repo` (string: null) - Chart repository URL where to locate the requested Chart.
* `set` (string: null) - A dictionary/map consisting of configuration values to provide to Helm. Keys should be specified as strings. Values may be either Integers or Strings. If more complex values are required, use `valuesContent`.
* `valuesContent` (string: null) - A list of configuration values to provide to Helm. This should be a multi-line string.

Below is an example of a HelmChart resource definition with all available configuration values defined.

```yaml
---
apiVersion: helm.cattle.io/v1
kind: HelmChart
metadata:
  name: example-helmchart
  namespace: kube-system
spec:
  chart: test-helmchart
  targetNamespace: test-namespace
  version: 0.0.1
  repo: https://example.com/helm-charts
  set:
    key1: "value1"
    key2: "value2"
  valuesContent: |-
    config:
      application:
        admin_username: admin
        admin_password: insecure_password
      server:
        port: 8080
        tls_enabled: false
```

The corresponding CLI arguments for this would be:

```console
$ helm install test-helmchart --namespace test-namespace \
                              --version 0.0.1 \
                              --repo https://example.com/helm-charts \
                              --set-string key1=value1 \
                              --set-string key2=value2 \
                              --values values.yaml
```

Where values.yaml contains the content present under the `valuesContent` field.

*Note: Both `set` and `valuesContent` may be used to configure the chart. `set` only supports strings or integers as values. If other data types are needed, use `valuesContent`.*

## Creating a HelmChart

We will now walk through the process of creating a HelmChart resource definition to deploy [Node-RED](https://nodered.org/) – a flow-based programming for the Internet of Things.

The following steps were performed on a Raspberry Pi 4 running the [Raspian](https://en.wikipedia.org/wiki/Raspbian) operating system. You may need to modify the commands if using a different environment.

First, create a file named `node-red-helmchart.yaml` using your default editor.

```console
$ editor node-red-helmchart.yaml
```

Copy and paste the following block of text into the file.

```yaml
---
apiVersion: helm.cattle.io/v1
kind: HelmChart
metadata:
  name: node-red
  namespace: kube-system
spec:
  chart: stable/node-red
  targetNamespace: node-red
  set:
    image.tag: rpi
  valuesContent: |-
    ingress:
      enabled: "true"
      annotations:
        kubernetes.io/ingress.class: traefik
        traefik.ingress.kubernetes.io/rule-type: PathPrefixStrip
      hosts:
        - raspberrypi1.local
      path: /node-red/
```

*Note: Be sure to modify the `ingress.hosts` field to contain the hostname and/or IP address of one or more node in your Kubernetes cluster.*

The deployment of the chart has been customized by modifying several of the available [configuration values](https://github.com/helm/charts/tree/master/stable/node-red#configuration). Particularly, Ingress support has been enabled which exposes Node-RED to hosts outside of the Kubernetes cluster at http://raspberrypi1.local/node-red/.

## Installing a HelmChart

There are two methods to install a HelmChart in k3s.

1. Install the resource definition using `kubectl apply`.
2. Copy the resource definition to the server's manifest directory at `/var/lib/rancher/k3s/server/manifests/`.

### Installing via kubectl

Instruct Kubernetes to install the specified resource definition using `kubectl apply`.

```console
$ sudo k3s kubectl apply --filename node-red-helmchart.yaml
helmchart.helm.cattle.io/node-red created
```

This is the standard method of provisioning resources within Kubernetes.

### Installing using Auto-Deploying Manifests

k3s includes a feature called [Auto-Deploying Manifests](https://rancher.com/docs/k3s/latest/en/configuration/#auto-deploying-manifests) which allows for the automated deployment of Helm charts (and Kubernetes manifests) placed in k3s manifest directory (`/var/lib/rancher/k3s/server/manifests`). Helm Controller continually watches this directory for new CRDs. When it detects a new resource has been added, it will automatically deploy the chart. Therefore, you can initiate the installation by simply copying `node-red-helmchart.yaml` into the server's manifest directory.

```console
$ sudo cp node-red-helmchart.yaml /var/lib/rancher/k3s/server/manifests/node-red-helmchart.yaml
```

### Installation process under the hood

Regardless of the installation method used, when a chart is deployed Helm Controller creates a Kubernetes [Job](https://kubernetes.io/docs/concepts/workloads/controllers/jobs-run-to-completion/) which installs the chart into the cluster using `helm install`.

Execute `kubectl get jobs` in the kube-system namespace and search for Jobs associated with the deployed chart.

```console
$ sudo k3s kubectl --namespace kube-system get jobs \
                   --selector helmcharts.helm.cattle.io/chart=node-red
NAME                    COMPLETIONS   DURATION   AGE
helm-install-node-red   0/1           21s        23s
```

You can view the command the Job is executing by re-running `kubectl get jobs`, and appending additional arguments to select and display Job-specific information using kubectl's [custom-column](https://kubernetes.io/docs/reference/kubectl/overview/#custom-columns) output.

```console
$ export CUSTOM_COLUMN_OUTPUT="COMMAND:.spec.template.spec.containers[0].name,INSTALL_ARGS:.spec.template.spec.containers[0].args"
$ sudo k3s kubectl --namespace kube-system get job helm-install-node-red \
                   --output=custom-columns=$CUSTOM_COLUMN_OUTPUT
COMMAND   INSTALL_ARGS
helm      [install --name node-red stable/node-red --namespace node-red --set-string image.tag=rpi]
```

The Job installs the chart by executing `helm install` with the list of installation arguments provided in the HelmChart resource, as noted earlier.

### Verifying the chart installation

Verify the chart has been successfully deployed by ensuring the associated resources have been created within the node-red namespace.

```console
$ sudo kubectl --namespace node-red get ingress,service,pods
NAME                          HOSTS                ADDRESS      PORTS   AGE
ingress.extensions/node-red   raspberrypi1.local   10.0.0.133   80      2m2s

NAME               TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)    AGE
service/node-red   ClusterIP   10.43.135.177   <none>        1880/TCP   2m2s

NAME                            READY   STATUS    RESTARTS   AGE
pod/node-red-7c5c564cb6-sz8b8   1/1     Running   0          2m1s
```

## Updating a HelmChart

The configuration for a chart may be updated using `kubectl apply`.

```console
$ sudo k3s kubectl apply --filename node-red-helmchart.yaml
```

## Deleting a HelmChart

To delete a Helm Chart, use `kubectl delete helmchart`.

```console
$ sudo k3s kubectl --namespace kube-system delete helmchart node-red
```

## Conclusion

You have learned about the HelmChart custom resource available in K3s, how to construct a HelmChart resource definition, install it on the cluster, and perform post-deployment operations (update/delete).

That's it for this walkthrough.

I have a written program, `k3s-helmchart-generate`, which simplifies the process of building HelmChart manifests. You can find a link to the repo on GitHub below.

{% github blake/k3s-helmchart-generate no-readme %}