HULL Tutorial 03: Integrating ConfigMaps and Secrets
Published: 2022-05-31 11:24:38

## Preparation

As a reminder, the goal of this tutorial series is to demonstrate how to build Helm charts based on the [HULL library chart](https://github.com/vidispine/hull) by recreating the functionality of the original `kubernetes-dashboard` Helm chart with a HULL based chart from scratch. When you have followed the previous part of this tutorial [on setting up a HULL base chart](https://dev.to/gre9ory/setup-a-helm-chart-based-on-hull-44i6-temp-slug-7198775) you have created a for now unconfigured Helm chart named `kubernetes-dashboard-hull` in the `02_setup` subfolder of your working directory (we assume that's `~/kubernetes-dashboard-hull` here). You can alternatively download the current chart state [here](https://github.com/vidispine/hull-tutorials/tree/test/dev-to/hull/02_setup) and continue from there. Also you should have checked out and extracted the `kubernetes-dashboard` Helm chart to `kubernetes-dashboard` in your working directory because examining it will be frequently required.

Start by inspecting the current state of the project with:

```sh
cd ~/kubernetes-dashboard-hull && ls 02_setup/kubernetes-dashboard*
```

which should be giving you:

```yml
02_setup/kubernetes-dashboard:
Chart.lock  Chart.yaml  README.md  charts  templates  values.yaml

02_setup/kubernetes-dashboard-hull:
Chart.lock  Chart.yaml  charts  templates  values.yaml
```

To continue you should make a copy for our new tutorial Part and leave the current state result untouched:

```sh
cp -R 02_setup/ 03_configmaps_and_secrets
```

Ok, now you can change to our freshly copied `kubernetes-dashboard-hull` chart directory:

```sh
cd 03_configmaps_and_secrets/kubernetes-dashboard-hull
```

and start adding things to our Helm chart!

## Analyzing the existing ConfigMaps

It is time to start moving the objects from the original to our new chart and focus on ConfigMaps and Secrets as a first step.

When checking the `kubernetes-dashboard`'s `templates` folder with:

```sh
find ../kubernetes-dashboard/templates -type f -iregex '.*\(configmap\|secret\).*' | sort
```

you can spot one ConfigMap and one Secret file each in there from the response:

```yml
../kubernetes-dashboard/templates/configmap.yaml
../kubernetes-dashboard/templates/secret.yaml
```

Note that file naming of templates does not actually mean anything in terms of what may be contained in them but for our exercise the result above points in the right direction.

You can proceed with getting the contents of the `templates/configmap.yaml` ConfigMap:

```sh
cat ../kubernetes-dashboard/templates/configmap.yaml
```

shows this:

```yml
# Copyright 2017 The Kubernetes Authors.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
#     http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

apiVersion: v1
kind: ConfigMap
metadata:
  labels:
    {{- include "kubernetes-dashboard.labels" . | nindent 4 }}
    {{- if .Values.commonLabels }}
    {{- include "common.tplvalues.render" ( dict "value" .Values.commonLabels "context" $ ) | nindent 4 }}
    {{- end }}
  annotations:
    {{- if .Values.commonAnnotations }}
    {{- include "common.tplvalues.render" ( dict "value" .Values.commonAnnotations "context" $ ) | nindent 4 }}
    {{- end }}
  name: kubernetes-dashboard-settings
data:
{{- with .Values.settings }}
  _global: {{ toJson . | quote }}
{{- end }}
{{- with .Values.pinnedCRDs }}
  _pinnedCRD: {{ toJson . | quote }}
{{- end }}
```

Starting from the top, first you can see that some labels are generated by an include of  `kubernetes-dashboard.labels` helper function. In [classic Helm charts such helper functions](https://helm.sh/docs/chart_template_guide/named_templates/#partials-and-_-files) are found usually in one or more `_*.tpl` helper files. The intention of the leading underscore in the name and the `.tpl` ending is to indicate these files do not contain objects that can be rendered but only helper functions.

To get an overview of the helper functions you may display the `_*.tpl` file contents because we will need to get back to them frequently. Cat the files contents with:

```sh
cat ../kubernetes-dashboard/templates/_*.tpl
```

to print the helper functions which are for the most part the typical assortment of a Helm charts helper functions:

```yml
# Copyright 2017 The Kubernetes Authors.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
#     http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

{{/* vim: set filetype=mustache: */}}
{{/*
Expand the name of the chart.
*/}}
{{- define "kubernetes-dashboard.name" -}}
{{- default .Chart.Name .Values.nameOverride | trunc 63 | trimSuffix "-" -}}
{{- end -}}

{{/*
Create a default fully qualified app name.
We truncate at 63 chars because some Kubernetes name fields are limited to this (by the DNS naming spec).
If release name contains chart name it will be used as a full name.
*/}}
{{- define "kubernetes-dashboard.fullname" -}}
{{- if .Values.fullnameOverride -}}
{{- .Values.fullnameOverride | trunc 63 | trimSuffix "-" -}}
{{- else -}}
{{- $name := default .Chart.Name .Values.nameOverride -}}
{{- if contains $name .Release.Name -}}
{{- .Release.Name | trunc 63 | trimSuffix "-" -}}
{{- else -}}
{{- printf "%s-%s" .Release.Name $name | trunc 63 | trimSuffix "-" -}}
{{- end -}}
{{- end -}}
{{- end -}}

{{/*
Create chart name and version as used by the chart label.
*/}}
{{- define "kubernetes-dashboard.chart" -}}
{{- printf "%s-%s" .Chart.Name .Chart.Version | replace "+" "_" | trunc 63 | trimSuffix "-" -}}
{{- end -}}

{{/*
Common labels
*/}}
{{- define "kubernetes-dashboard.labels" -}}
app.kubernetes.io/name: {{ include "kubernetes-dashboard.name" . }}
helm.sh/chart: {{ include "kubernetes-dashboard.chart" . }}
app.kubernetes.io/instance: {{ .Release.Name }}
{{- if .Chart.AppVersion }}
app.kubernetes.io/version: {{ .Chart.AppVersion | quote }}
{{- end }}
app.kubernetes.io/managed-by: {{ .Release.Service }}
{{- end -}}

{{/*
Common label selectors
*/}}
{{- define "kubernetes-dashboard.matchLabels" -}}
app.kubernetes.io/name: {{ include "kubernetes-dashboard.name" . }}
app.kubernetes.io/instance: {{ .Release.Name }}
{{- end -}}

{{/*
Name of the service account to use
*/}}
{{- define "kubernetes-dashboard.serviceAccountName" -}}
{{- if .Values.serviceAccount.create -}}
    {{ default (include "kubernetes-dashboard.fullname" .) .Values.serviceAccount.name }}
{{- else -}}
    {{ default "default" .Values.serviceAccount.name }}
{{- end -}}
{{- end -}}
# Copyright 2017 The Kubernetes Authors.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
#     http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

{{/* vim: set filetype=mustache: */}}
{{/*
Renders a value that contains template.
Usage:
{{ include "common.tplvalues.render" ( dict "value" .Values.path.to.the.Value "context" $) }}
*/}}
{{- define "common.tplvalues.render" -}}
    {{- if typeIs "string" .value }}
        {{- tpl .value .context }}
    {{- else }}
        {{- tpl (.value | toYaml) .context }}
    {{- end }}
{{- end -}}
```

As already demonstrated in the previous part of this tutorial series, all HULL-created objects do automatically get the standard Kubernetes metadata fields populated. There is no further necessity to work on the standard metadata of our objects for the conversion, HULL automatically creates metadata information that is equivalent to what happens in the `kubernetes-dashboard.labels` function that is part of every objects metadata label definition:

```yml
{{/*
Common labels
*/}}
{{- define "kubernetes-dashboard.labels" -}}
app.kubernetes.io/name: {{ include "kubernetes-dashboard.name" . }}
helm.sh/chart: {{ include "kubernetes-dashboard.chart" . }}
app.kubernetes.io/instance: {{ .Release.Name }}
{{- if .Chart.AppVersion }}
app.kubernetes.io/version: {{ .Chart.AppVersion | quote }}
{{- end }}
app.kubernetes.io/managed-by: {{ .Release.Service }}
{{- end -}}
```

So that is covered. After the standard labels have been set here:

```yml
    {{- include "kubernetes-dashboard.labels" . | nindent 4 }}
```

there comes an optional block where additional common labels may be set:

```yml
    {{- if .Values.commonLabels }}
    {{- include "common.tplvalues.render" ( dict "value" .Values.commonLabels "context" $ ) | nindent 4 }}
    {{- end }}
```

The definition of `common.tplvalues.render` can be found above where you checked the helper functions, anyway here it is again:

```yml
# Copyright 2017 The Kubernetes Authors.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
#     http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

{{/* vim: set filetype=mustache: */}}
{{/*
Renders a value that contains template.
Usage:
{{ include "common.tplvalues.render" ( dict "value" .Values.path.to.the.Value "context" $) }}
*/}}
{{- define "common.tplvalues.render" -}}
    {{- if typeIs "string" .value }}
        {{- tpl .value .context }}
    {{- else }}
        {{- tpl (.value | toYaml) .context }}
    {{- end }}
{{- end -}}
```

What happens here is pretty simple actually: when a string is passed as `value` the value is subjected to the `tpl` function, for anything else (dictionaries, arrays) the Go object is converted to a YAML block before being subjected to `tpl`.

At this point you may check the reference to `.Values.commonLabels` which is defined in the `values.yaml` with cat and a little context around the hits:

```sh
cat ../kubernetes-dashboard/values.yaml | grep "commonLabels" -B 1 -A 4
```

which reveals:

```yml

## @param commonLabels Labels to add to all deployed objects
##
commonLabels: {}
## @param commonAnnotations Annotations to add to all deployed objects
##
commonAnnotations: {}

```

So with all pieces put together, with `commonLabels` being a dictionary the function would insert any key value pairs that may be defined in `commonLabels` into the `labels` section of the ConfigMap. With HULL, further metadata such as the `commonLabels` here can be added at the global level, the object type level or at object instance level. You will see this in use in a minute.

In terms of the `annotations` it is the same procedure here with the `commonAnnotations` as with the `commonLabels` so no need to check this any closer.

Now focus on the ConfigMaps `data` section:

```yml
data:
{{- with .Values.settings }}
  _global: {{ toJson . | quote }}
{{- end }}
{{- with .Values.pinnedCRDs }}
  _pinnedCRD: {{ toJson . | quote }}
{{- end }}
```

For storing the `_global` and `_pinnedCRD` data into the ConfigMap the YAML based data from the `values.yaml` fields `.Values.settings` and `.Values.pinnedCRDs` is converted to JSON. How to achieve that with HULL and without any templates is what will be explored next. This wraps up the inspection of this ConfigMap and the conversion to a HULL specification can begin.

## Creating ConfigMaps with HULL

Time to get down to business by creating the first (and in our case only) ConfigMap in the HULL charts `values.yaml`. 

So to get started, copy or type in the following:

```sh
echo 'hull:
  objects:
    configmap:
      settings:
        data:
          _global:
            inline: ""
          _pinnedCRD:
            inline: ""' >> values.yaml
```

to create the ConfigMap with empty entries in `values.yaml`. You can check the initial output with:

```sh
helm template .
```

now containing the `settings` ConfigMap:

```yml
---
# Source: kubernetes-dashboard/templates/hull.yaml
apiVersion: v1
kind: ServiceAccount
metadata:
  annotations: {}
  labels:
    app.kubernetes.io/component: default
    app.kubernetes.io/instance: release-name
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/name: kubernetes-dashboard
    app.kubernetes.io/part-of: undefined
    app.kubernetes.io/version: 2.5.0
    helm.sh/chart: kubernetes-dashboard-5.2.0
  name: release-name-kubernetes-dashboard-default
---
# Source: kubernetes-dashboard/templates/hull.yaml
apiVersion: v1
data:
  _global: ""
  _pinnedCRD: ""
kind: ConfigMap
metadata:
  annotations: {}
  labels:
    app.kubernetes.io/component: settings
    app.kubernetes.io/instance: release-name
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/name: kubernetes-dashboard
    app.kubernetes.io/part-of: undefined
    app.kubernetes.io/version: 2.5.0
    helm.sh/chart: kubernetes-dashboard-5.2.0
  name: release-name-kubernetes-dashboard-settings
---
# Source: kubernetes-dashboard/templates/hull.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  annotations: {}
  labels:
    app.kubernetes.io/component: default
    app.kubernetes.io/instance: release-name
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/name: kubernetes-dashboard
    app.kubernetes.io/part-of: undefined
    app.kubernetes.io/version: 2.5.0
    helm.sh/chart: kubernetes-dashboard-5.2.0
  name: release-name-kubernetes-dashboard-default
rules: []
---
# Source: kubernetes-dashboard/templates/hull.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  annotations: {}
  labels:
    app.kubernetes.io/component: default
    app.kubernetes.io/instance: release-name
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/name: kubernetes-dashboard
    app.kubernetes.io/part-of: undefined
    app.kubernetes.io/version: 2.5.0
    helm.sh/chart: kubernetes-dashboard-5.2.0
  name: release-name-kubernetes-dashboard-default
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: release-name-kubernetes-dashboard-default
subjects:
- kind: ServiceAccount
  name: release-name-kubernetes-dashboard-default
```

Right now the `settings` ConfigMaps data entries are intentionally just empty, this is the relevant excerpt from above output:

```yml
---
# Source: kubernetes-dashboard/templates/hull.yaml
apiVersion: v1
data:
  _global: ""
  _pinnedCRD: ""
kind: ConfigMap
metadata:
  annotations: {}
  labels:
    app.kubernetes.io/component: settings
    app.kubernetes.io/instance: release-name
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/name: kubernetes-dashboard
    app.kubernetes.io/part-of: undefined
    app.kubernetes.io/version: 2.5.0
    helm.sh/chart: kubernetes-dashboard-5.2.0
  name: release-name-kubernetes-dashboard-settings
```

Before dealing with the proper population of the `data` entries as JSON content a brief detour for covering the built-in HULL metadata features is useful.

### Adding Metdata to objects globally, on object type and object level

Say you want to add more metadata to your objects. For instance you may want to add further labels or annotations to all your objects (the `commonLabels` and `commonAnnotations` from the original chart) or only some of them. Maybe all ConfigMaps should have a particular metadata label or annotation. Or there is a set of custom metadata labels you want to associate on objects of different types which are to be grouped in some sense? All of this can be done conveniently with HULL!

To start testing out configurational variations to the current `values.yaml` or enabling/disabling features it is time to bring in another (familiar) layer of configuration that enables performing our tests of the functionality and simulate system specific settings in addition to the `values.yaml` defaults. An 'external' or 'system specific' `values.yaml`' is required to vary the configuration on top of the defaults and see the effect on rendering outputs. Before any HULL related processing takes place, the contents of this external file are simply overlayed or merged entry by entry with the `values.yaml` structure when [applied with `-f <file-path>`](https://helm.sh/docs/chart_best_practices/values/#consider-how-users-will-use-your-values) to any `helm` command. Note that any data that is added via the `-f` switch has precedence over the `values.yaml` defaults (which is what you'd expect anyways).

Start by creating a new `../configs` folder now to hold such 'external configurations' to be applied on top of our default `values.yaml`:

```sh
mkdir ../configs
```
Add a first test configuration like this by adding sample labels to our ConfigMap and `default` object instances. First you write the overlay data for our test and then overlay it when `helm template`ing it:

```sh
echo 'hull:
  config:
    general:
      metadata:
        labels:
          custom:
            label1: hello
            label2: global
            label3: labels!'> ../configs/global-custom-metadata.yaml\
&& helm template -f ../configs/global-custom-metadata.yaml .
```

and all objects have the shared labels attached as you can see from the output:

```yml
---
# Source: kubernetes-dashboard/templates/hull.yaml
apiVersion: v1
kind: ServiceAccount
metadata:
  annotations: {}
  labels:
    app.kubernetes.io/component: default
    app.kubernetes.io/instance: release-name
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/name: kubernetes-dashboard
    app.kubernetes.io/part-of: undefined
    app.kubernetes.io/version: 2.5.0
    helm.sh/chart: kubernetes-dashboard-5.2.0
    label1: hello
    label2: global
    label3: labels!
  name: release-name-kubernetes-dashboard-default
---
# Source: kubernetes-dashboard/templates/hull.yaml
apiVersion: v1
data:
  _global: ""
  _pinnedCRD: ""
kind: ConfigMap
metadata:
  annotations: {}
  labels:
    app.kubernetes.io/component: settings
    app.kubernetes.io/instance: release-name
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/name: kubernetes-dashboard
    app.kubernetes.io/part-of: undefined
    app.kubernetes.io/version: 2.5.0
    helm.sh/chart: kubernetes-dashboard-5.2.0
    label1: hello
    label2: global
    label3: labels!
  name: release-name-kubernetes-dashboard-settings
---
# Source: kubernetes-dashboard/templates/hull.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  annotations: {}
  labels:
    app.kubernetes.io/component: default
    app.kubernetes.io/instance: release-name
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/name: kubernetes-dashboard
    app.kubernetes.io/part-of: undefined
    app.kubernetes.io/version: 2.5.0
    helm.sh/chart: kubernetes-dashboard-5.2.0
    label1: hello
    label2: global
    label3: labels!
  name: release-name-kubernetes-dashboard-default
rules: []
---
# Source: kubernetes-dashboard/templates/hull.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  annotations: {}
  labels:
    app.kubernetes.io/component: default
    app.kubernetes.io/instance: release-name
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/name: kubernetes-dashboard
    app.kubernetes.io/part-of: undefined
    app.kubernetes.io/version: 2.5.0
    helm.sh/chart: kubernetes-dashboard-5.2.0
    label1: hello
    label2: global
    label3: labels!
  name: release-name-kubernetes-dashboard-default
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: release-name-kubernetes-dashboard-default
subjects:
- kind: ServiceAccount
  name: release-name-kubernetes-dashboard-default
```

If you'd want the labels only being attached to ConfigMap objects you can attach them to the HULL internal default ConfigMap object. All object types you can use in HULL support having a special default object (specified with key `_HULL_OBJECT_TYPE_DEFAULT_`) where you can attach default fields and values that are being inherited by all instances of that type. Individual instances may in turn overwrite them again specifically. Instances of `_HULL_OBJECT_TYPE_DEFAULT_` objects are of course never rendered, they only provide templates with default values for instances of the parent objects type.

In practice create another test configuration to test objecttype level metadata population:

```sh
echo 'hull:
  objects:
    configmap:
      _HULL_OBJECT_TYPE_DEFAULT_:
        labels:
          label1: hello
          label2: objecttype
          label3: labels!
      another:
        data: {}
      and_another:
        data: {}'> ../configs/objecttype-custom-metadata.yaml \
&& helm template -f ../configs/objecttype-custom-metadata.yaml .
```

you can see that only the three specified ConfigMaps have the labels assigned now:

```yml
---
# Source: kubernetes-dashboard/templates/hull.yaml
apiVersion: v1
kind: ServiceAccount
metadata:
  annotations: {}
  labels:
    app.kubernetes.io/component: default
    app.kubernetes.io/instance: release-name
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/name: kubernetes-dashboard
    app.kubernetes.io/part-of: undefined
    app.kubernetes.io/version: 2.5.0
    helm.sh/chart: kubernetes-dashboard-5.2.0
  name: release-name-kubernetes-dashboard-default
---
# Source: kubernetes-dashboard/templates/hull.yaml
apiVersion: v1
kind: ConfigMap
metadata:
  annotations: {}
  labels:
    app.kubernetes.io/component: and_another
    app.kubernetes.io/instance: release-name
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/name: kubernetes-dashboard
    app.kubernetes.io/part-of: undefined
    app.kubernetes.io/version: 2.5.0
    helm.sh/chart: kubernetes-dashboard-5.2.0
    label1: hello
    label2: objecttype
    label3: labels!
  name: release-name-kubernetes-dashboard-and_another
---
# Source: kubernetes-dashboard/templates/hull.yaml
apiVersion: v1
kind: ConfigMap
metadata:
  annotations: {}
  labels:
    app.kubernetes.io/component: another
    app.kubernetes.io/instance: release-name
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/name: kubernetes-dashboard
    app.kubernetes.io/part-of: undefined
    app.kubernetes.io/version: 2.5.0
    helm.sh/chart: kubernetes-dashboard-5.2.0
    label1: hello
    label2: objecttype
    label3: labels!
  name: release-name-kubernetes-dashboard-another
---
# Source: kubernetes-dashboard/templates/hull.yaml
apiVersion: v1
data:
  _global: ""
  _pinnedCRD: ""
kind: ConfigMap
metadata:
  annotations: {}
  labels:
    app.kubernetes.io/component: settings
    app.kubernetes.io/instance: release-name
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/name: kubernetes-dashboard
    app.kubernetes.io/part-of: undefined
    app.kubernetes.io/version: 2.5.0
    helm.sh/chart: kubernetes-dashboard-5.2.0
    label1: hello
    label2: objecttype
    label3: labels!
  name: release-name-kubernetes-dashboard-settings
---
# Source: kubernetes-dashboard/templates/hull.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  annotations: {}
  labels:
    app.kubernetes.io/component: default
    app.kubernetes.io/instance: release-name
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/name: kubernetes-dashboard
    app.kubernetes.io/part-of: undefined
    app.kubernetes.io/version: 2.5.0
    helm.sh/chart: kubernetes-dashboard-5.2.0
  name: release-name-kubernetes-dashboard-default
rules: []
---
# Source: kubernetes-dashboard/templates/hull.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  annotations: {}
  labels:
    app.kubernetes.io/component: default
    app.kubernetes.io/instance: release-name
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/name: kubernetes-dashboard
    app.kubernetes.io/part-of: undefined
    app.kubernetes.io/version: 2.5.0
    helm.sh/chart: kubernetes-dashboard-5.2.0
  name: release-name-kubernetes-dashboard-default
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: release-name-kubernetes-dashboard-default
subjects:
- kind: ServiceAccount
  name: release-name-kubernetes-dashboard-default
```

It is important to mention that object type defaults do not only apply to metadata, you can basically default any property value on the object definition with a value, for example to predefine a `data` entry that all ConfigMaps should have you could do the following:

```sh
echo 'hull:
  objects:
    configmap:
      _HULL_OBJECT_TYPE_DEFAULT_:
        labels:
          label1: hello
          label2: objecttype
          label3: labels!
        data:
          see:
            inline: "I am in every ConfigMap!"
      another:
        data: {}
      and_another:
        data: {}'> ../configs/objecttype-data-default.yaml \
&& helm template -f ../configs/objecttype-data-default.yaml .
```

and see it actually is in every ConfigMap:

```yml
---
# Source: kubernetes-dashboard/templates/hull.yaml
apiVersion: v1
kind: ServiceAccount
metadata:
  annotations: {}
  labels:
    app.kubernetes.io/component: default
    app.kubernetes.io/instance: release-name
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/name: kubernetes-dashboard
    app.kubernetes.io/part-of: undefined
    app.kubernetes.io/version: 2.5.0
    helm.sh/chart: kubernetes-dashboard-5.2.0
  name: release-name-kubernetes-dashboard-default
---
# Source: kubernetes-dashboard/templates/hull.yaml
apiVersion: v1
data:
  see: I am in every ConfigMap!
kind: ConfigMap
metadata:
  annotations: {}
  labels:
    app.kubernetes.io/component: and_another
    app.kubernetes.io/instance: release-name
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/name: kubernetes-dashboard
    app.kubernetes.io/part-of: undefined
    app.kubernetes.io/version: 2.5.0
    helm.sh/chart: kubernetes-dashboard-5.2.0
    label1: hello
    label2: objecttype
    label3: labels!
  name: release-name-kubernetes-dashboard-and_another
---
# Source: kubernetes-dashboard/templates/hull.yaml
apiVersion: v1
data:
  see: I am in every ConfigMap!
kind: ConfigMap
metadata:
  annotations: {}
  labels:
    app.kubernetes.io/component: another
    app.kubernetes.io/instance: release-name
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/name: kubernetes-dashboard
    app.kubernetes.io/part-of: undefined
    app.kubernetes.io/version: 2.5.0
    helm.sh/chart: kubernetes-dashboard-5.2.0
    label1: hello
    label2: objecttype
    label3: labels!
  name: release-name-kubernetes-dashboard-another
---
# Source: kubernetes-dashboard/templates/hull.yaml
apiVersion: v1
data:
  _global: ""
  _pinnedCRD: ""
  see: I am in every ConfigMap!
kind: ConfigMap
metadata:
  annotations: {}
  labels:
    app.kubernetes.io/component: settings
    app.kubernetes.io/instance: release-name
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/name: kubernetes-dashboard
    app.kubernetes.io/part-of: undefined
    app.kubernetes.io/version: 2.5.0
    helm.sh/chart: kubernetes-dashboard-5.2.0
    label1: hello
    label2: objecttype
    label3: labels!
  name: release-name-kubernetes-dashboard-settings
---
# Source: kubernetes-dashboard/templates/hull.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  annotations: {}
  labels:
    app.kubernetes.io/component: default
    app.kubernetes.io/instance: release-name
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/name: kubernetes-dashboard
    app.kubernetes.io/part-of: undefined
    app.kubernetes.io/version: 2.5.0
    helm.sh/chart: kubernetes-dashboard-5.2.0
  name: release-name-kubernetes-dashboard-default
rules: []
---
# Source: kubernetes-dashboard/templates/hull.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  annotations: {}
  labels:
    app.kubernetes.io/component: default
    app.kubernetes.io/instance: release-name
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/name: kubernetes-dashboard
    app.kubernetes.io/part-of: undefined
    app.kubernetes.io/version: 2.5.0
    helm.sh/chart: kubernetes-dashboard-5.2.0
  name: release-name-kubernetes-dashboard-default
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: release-name-kubernetes-dashboard-default
subjects:
- kind: ServiceAccount
  name: release-name-kubernetes-dashboard-default
```

Adding a dedicated label or annotation just to your `settings` ConfigMap is easy with this overlay configuration for example:


```sh
echo 'hull:
  objects:
    configmap:
      _HULL_OBJECT_TYPE_DEFAULT_:
        labels:
          label1: hello
          label2: objecttype
          label3: labels!
      settings:
        labels:
          label4: I am a settings label!
        annotations:
          annotation: I am a settings annotation!
      another:
        data: {}
      and_another:
        data: {}'> ../configs/object-custom-metadata.yaml \
&& helm template -f ../configs/object-custom-metadata.yaml .
```

yielding added annotation on the `settings` ConfigMap:

```yml
---
# Source: kubernetes-dashboard/templates/hull.yaml
apiVersion: v1
kind: ServiceAccount
metadata:
  annotations: {}
  labels:
    app.kubernetes.io/component: default
    app.kubernetes.io/instance: release-name
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/name: kubernetes-dashboard
    app.kubernetes.io/part-of: undefined
    app.kubernetes.io/version: 2.5.0
    helm.sh/chart: kubernetes-dashboard-5.2.0
  name: release-name-kubernetes-dashboard-default
---
# Source: kubernetes-dashboard/templates/hull.yaml
apiVersion: v1
kind: ConfigMap
metadata:
  annotations: {}
  labels:
    app.kubernetes.io/component: and_another
    app.kubernetes.io/instance: release-name
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/name: kubernetes-dashboard
    app.kubernetes.io/part-of: undefined
    app.kubernetes.io/version: 2.5.0
    helm.sh/chart: kubernetes-dashboard-5.2.0
    label1: hello
    label2: objecttype
    label3: labels!
  name: release-name-kubernetes-dashboard-and_another
---
# Source: kubernetes-dashboard/templates/hull.yaml
apiVersion: v1
kind: ConfigMap
metadata:
  annotations: {}
  labels:
    app.kubernetes.io/component: another
    app.kubernetes.io/instance: release-name
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/name: kubernetes-dashboard
    app.kubernetes.io/part-of: undefined
    app.kubernetes.io/version: 2.5.0
    helm.sh/chart: kubernetes-dashboard-5.2.0
    label1: hello
    label2: objecttype
    label3: labels!
  name: release-name-kubernetes-dashboard-another
---
# Source: kubernetes-dashboard/templates/hull.yaml
apiVersion: v1
data:
  _global: ""
  _pinnedCRD: ""
kind: ConfigMap
metadata:
  annotations:
    annotation: I am a settings annotation!
  labels:
    app.kubernetes.io/component: settings
    app.kubernetes.io/instance: release-name
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/name: kubernetes-dashboard
    app.kubernetes.io/part-of: undefined
    app.kubernetes.io/version: 2.5.0
    helm.sh/chart: kubernetes-dashboard-5.2.0
    label1: hello
    label2: objecttype
    label3: labels!
    label4: I am a settings label!
  name: release-name-kubernetes-dashboard-settings
---
# Source: kubernetes-dashboard/templates/hull.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  annotations: {}
  labels:
    app.kubernetes.io/component: default
    app.kubernetes.io/instance: release-name
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/name: kubernetes-dashboard
    app.kubernetes.io/part-of: undefined
    app.kubernetes.io/version: 2.5.0
    helm.sh/chart: kubernetes-dashboard-5.2.0
  name: release-name-kubernetes-dashboard-default
rules: []
---
# Source: kubernetes-dashboard/templates/hull.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  annotations: {}
  labels:
    app.kubernetes.io/component: default
    app.kubernetes.io/instance: release-name
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/name: kubernetes-dashboard
    app.kubernetes.io/part-of: undefined
    app.kubernetes.io/version: 2.5.0
    helm.sh/chart: kubernetes-dashboard-5.2.0
  name: release-name-kubernetes-dashboard-default
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: release-name-kubernetes-dashboard-default
subjects:
- kind: ServiceAccount
  name: release-name-kubernetes-dashboard-default
```

As you already saw, HULL automatically creates a RBAC trinity of `default` ServiceAccount, Role and RoleBinding which can be put to use straight away and you saw how setting metadata on different hierarchical levels affected these object instances. However since there is gonna be a lot more of `helm template .` output to examine in the next tutorial parts it is recommended to turn off rendering the default RBAC instances and the ServiceAccount when it is not of concern. Using an overlay `values.yaml` like this one:

```sh
echo 'hull:
  objects:
    serviceaccount:
      default:
        enabled: false
    role:
      default:
        enabled: false
    rolebinding:
      default:
        enabled: false'> ../configs/disable-default-rbac.yaml
```

is a good approach to disable the RBAC objects whenever they are not of interest. Whenever it is relevant for the topic of the tutorial however you can just switch RBAC objects on again and simply omit the `-f ../configs/disable-default-rbac.yaml` overlay.

In the next part the focus is on populating the `data` section. For this we need to introduce a way to dynamically change the provided YAML contents to JSON while rendering. When explicitly not using any templates the question is how can you process the `values.yaml` contents dynamically before rendering?

With [HULL transformations](https://github.com/vidispine/hull/blob/main/hull/doc/transformations.md) you can. They provide the answer to this problem and they will be used wherever the need arises to model dynamic behavior for improving user experience.

## Using HULL transformations

In the regular Helm workflow any templating expressions or other dynamic behavior is not tolerated within the `values.yaml` itself since it must at all times conform to the YAML specification. Templating expressions are only allowed and evaluated in the `templates` folder.

The focus and main goal of creating HULL was to get rid of the need to model each simple 1:1 projection from `values.yaml` properties to Kubernetes object properties by providing direct access to all Kubernetes objects properties instead to only having to specify exactly what you want. This is solely achieved by configuration of the `values.yaml` and without any chart-specific custom templates in the `template` folder. Transformations in HULL put back the possibility to use template expressions for more complex projections which are driven from the `values.yaml` alone. But how is this done when Helm prohibits any templating expression in the `values.yaml`?

Since within the HULL processing pipeline the whole `values.yaml` tree is traversed internally it enables the possibility to inject templating again into property value calculation by processing keywords that signal the use of a dynamic function (=__transformation__) and its parameters. The return value of the transformation then replaces the property value where the transformation was called. A caveat is that the specification and use of those functions must not invalidate the YAML (and JSON schema) of the `values.yaml`, otherwise Helm would not be happy about processing the `values.yaml`. For this reason the transformations themselves are wrapped as strings, dictionaries or arrays. 

For HULL to know that a property value requires transforming, special transformation triggers/keyword prefixes  (`_HULL_TRANSFORMATION_` or any existing short forms of specific transformations such as `_HT?`, `_HT!`, `_HT*`, `_HT^` and `_HT&`) signal the use of a transformation to determine the actual value to be rendered. More information about transformations and how to specify and use them can be found in the [HULL docs on transformations](https://github.com/vidispine/hull/blob/main/hull/doc/transformations.md) but they are explored and utilized in this tutorial to a large degree to learn how to use them.

### The `_HT?` transformation

The `_HT?` (or in long form `hull.util.transformation.bool` function) is a first simple transformation example albeit a useful one. It needs a templating expression as input which needs to resolve to a boolean value when computed. It can therefore be used to set any boolean property but it is mainly useful to determine whether a particular object should be rendered or not at deploy time depending on automated conditions. Rendering for all HULL objects is controlled via the boolean `enabled` property on the object instance, you can set it for each main object defined through HULL. By binding the `enabled` property's value to an arbitrary `hull.util.transformation.bool` function returning a boolean value you can dynamically enable and disable rendering of the object depending on other configurational input to your chart.

To highlight the usage of transformations such as the `_HT?`, say for demo purposes you only want to render the `settings` ConfigMap in case its `_global` or `_pinnedCRD` entry has any content (meaning the `inline` property is not equal to an empty string). A built-in `hull.util.transformation.bool` transformation to be bound to the ConfigMaps `enabled` switch could look like this:

```yml
enabled: _HULL_TRANSFORMATION_<<<NAME=hull.util.transformation.bool>>><<<CONDITION=or (ne (index . "$").Values.hull.objects.configmap.settings.data._global.inline "") (ne (index . "$").Values.hull.objects.configmap.settings.data._pinnedCRD.inline "")>>>
```

wherein you check that any of the `_global.inline` or the `_pinnedCRD` data section is not empty and only then render the ConfigMap. You can see that it is possible to use the well-known Go templating functionalities here, the only thing to keep in mind is that you cannot use the global `.` context (as in `.Values.someProperty`) but must use the expression `(index . "$")` instead to access the global context (as in `(index . "$").Values.someProperty`).

To really save some typing in the future you should use the short form to built-in transformations. They can be viewed as a kind of macro that points to a specifc transformation and omit the argument naming because the transformation only has one argument whose value is supplied directly after the shortcut.

The same transformation as above in short form looks like this:

```yml
enabled: _HT?or (ne (index . "$").Values.hull.objects.configmap.settings.data._global.inline "") (ne (index . "$").Values.hull.objects.configmap.settings.data._pinnedCRD.inline "")
```

which minifies the call to the transformation and is way handier to type. Again please checkout the [HULL docs on transformations](https://github.com/vidispine/hull/blob/main/hull/doc/transformations.md) for more information.

Now test this out with a test config file:

```sh
echo 'hull:
  objects:
    configmap:
      settings:
        enabled: _HT?or (ne (index . "$").Values.hull.objects.configmap.settings.data._global.inline "") (ne (index . "$").Values.hull.objects.configmap.settings.data._pinnedCRD.inline "")'> ../configs/bool-transformation-1.yaml \
&& helm template -f ../configs/disable-default-rbac.yaml -f ../configs/bool-transformation-1.yaml .
```

and see that the settings ConfigMap is not rendered:

```yml

```

Since the `default` RBAC objects are also omitted via `-f ../configs/disable-default-rbac.yaml` the output is just empty.

For the counter example, add some content to the `_global` entry:

```sh
echo 'hull:
  objects:
    configmap:
      settings:
        enabled: _HT?or (ne (index . "$").Values.hull.objects.configmap.settings.data._global.inline "") (ne (index . "$").Values.hull.objects.configmap.settings.data._pinnedCRD.inline "")
        data:
          _global:
            inline: "test"'> ../configs/bool-transformation-2.yaml \
&& helm template -f ../configs/disable-default-rbac.yaml -f ../configs/bool-transformation-2.yaml .
```

observe the ConfigMap back again in the output:

```yml
---
# Source: kubernetes-dashboard/templates/hull.yaml
apiVersion: v1
data:
  _global: test
  _pinnedCRD: ""
kind: ConfigMap
metadata:
  annotations: {}
  labels:
    app.kubernetes.io/component: settings
    app.kubernetes.io/instance: release-name
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/name: kubernetes-dashboard
    app.kubernetes.io/part-of: undefined
    app.kubernetes.io/version: 2.5.0
    helm.sh/chart: kubernetes-dashboard-5.2.0
  name: release-name-kubernetes-dashboard-settings
```


After this detour introducing the `hull.util.transformation.bool`/`_HT?` transformation in practice you can focus again on the transformation to JSON part. 

Recap the `data` section of the ConfigMap you want to convert:

```yml
data:
{{- with .Values.settings }}
  _global: {{ toJson . | quote }}
{{- end }}
{{- with .Values.pinnedCRDs }}
  _pinnedCRD: {{ toJson . | quote }}
{{- end }}
```

You can see there is a redirection to the source of the entries values (`.Values.settings` and `.Values.pinnedCRDs`) where the data is defined and processed here in the template. To mimic this you need to add a similar redirection within `values.yaml` from which data is being read when processing `_global` and `_pinnedCRD` and then a function to convert it on rendering.

A similar use case is when individual properties are logically referenced in multiple places (1:n relations) within `values.yaml` it is best to define the actual data in a 'single source of truth' fashion, like it is done in the original Helm chart concept with the `.Values.settings` and `.Values.pinnedCRDs` fields being referencable basically everywhere in the templates.

With HULL you can do this too and there is a clearly defined place in the HULL concept for such properties that need to be referenced multiple distinct times for object creation or transformation in the `hull.objects` section: the `hull.config.specific` dictionary. This `hull.config.specific` section is specifically intended for this purpose of adding central configuration data that is either singled out for convenient documentation, highlighting and configuration or being used in multiple places within the chart. You can think of this section resembling one of the functions of the `values.yaml` itself within the classic Helm chart approach, namely to centralize access to configuration values. Technically the `hull.config` section is processed before the `hull.objects` section and hence properties defined here and their values (even if derived by a HULL transformation!) are always available for referencing to when defining the objects in the `hull.objects` section.

To cover the dynamic nature of the JSON transformation you can utilize the `hull.util.transformation.tpl` transformation.

### Enter the `_HT!` transformation

There are some convenience transformations like the `hull.util.transformation.bool` transformation within HULL but by far the most powerful and flexible is the HULL `hull.util.transformation.tpl` or `_HT!` transformation. It executes the Helm `tpl` function on a given string providing full range of Go templating capabilities as would be the case with Go templating expressions within template files.

Using the `hull.util.transformation.tpl` transformation you can return different datatypes that are expected for the property's values you apply it on:

- string
- int
- bool
- array/list
- dictionary/map

An important aspect to keep in mind when producing arrays and dictionaries with the HULL `hull.util.transformation.tpl` transformation is to use [`flow` style YAML](https://cloudslang-docs.readthedocs.io/en/latest/overview/yaml_overview.html#indentation-scoping) to define the transformation result. This style reduces syntax problems (mostly due to indendation) appearing with the `block` style YAML that is more typically used when dealing with Helm and Kubernetes. Check the next example on how flow style YAML looks like (it is more similar to JSON and does not rely on correct indentation).

To create the `_global` and `_pinnedCRD` sections in the ConfigMap first insert into the existing `values.yaml` the `config.specific` block which holds the `settings` and `pinnedCRDs` data:

```sh
sed '/^hull/r'<(
    echo "  config:"
    echo "    specific:"
    echo "      settings: {}"
    echo "      pinnedCRDs: {}"
  ) -i -- values.yaml
```

Time to add the `tpl` transformations to create the JSON content from the now created and referenced sections. The `hull.util.transformation.tpl`'s short form `_HT!` is being used and in the `tpl` argument you do a `toJson | quote` on the source data as in the original source ConfigMap. The result of this operation is returned as the `data` properties values.

### Adding ConfigMap and Secret contents via `inline` specification in `values.yaml`

You likely noticed that within the ConfigMap files `data` section the actual values for `data` keys were directly embedded in the `values.yaml` via the `inline` property. This is a handy way to be able to specify shorter contents right in place where the ConfigMap is defined.

Add the first JSON transformation via an `inline` specification:

```sh
sed '0,/inline: ""/{s/inline: ""/inline: _HT!{{ toJson (index . "$").Values.hull.config.specific.settings | quote }}/}' -i -- values.yaml
```

updating your `values.yaml` to:

```yml
metrics-server:
  enabled: false
hull:
  config:
    specific:
      settings: {}
      pinnedCRDs: {}
  objects:
    configmap:
      settings:
        data:
          _global:
            inline: _HT!{{ toJson (index . "$").Values.hull.config.specific.settings | quote }}
          _pinnedCRD:
            inline: ""
```

Test it out with some YAML content to convert to JSON:

```sh
echo 'hull:
  config:
    specific:
      settings:
        settings_one:
          a_string: Some config setting
          a_number: 333
        settings_two:
          a_boolean: true'> ../configs/inline-transformation.yaml \
&& helm template -f ../configs/disable-default-rbac.yaml -f ../configs/inline-transformation.yaml .
```

and check out the JSON content in the ConfigMap:

```yml
---
# Source: kubernetes-dashboard/templates/hull.yaml
apiVersion: v1
data:
  _global: '{"settings_one":{"a_number":333,"a_string":"Some config setting"},"settings_two":{"a_boolean":true}}'
  _pinnedCRD: ""
kind: ConfigMap
metadata:
  annotations: {}
  labels:
    app.kubernetes.io/component: settings
    app.kubernetes.io/instance: release-name
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/name: kubernetes-dashboard
    app.kubernetes.io/part-of: undefined
    app.kubernetes.io/version: 2.5.0
    helm.sh/chart: kubernetes-dashboard-5.2.0
  name: release-name-kubernetes-dashboard-settings
```

### Adding ConfigMap and Secret contents via `path` specification from external files 

Alternatively to the `inline` method you can specify the path to an external file to import content from via usage of the `path` property and omitting the `inline` property. It is really simple, just provide a relative path to a file and it will be rendered as a ConfigMap (or Secret) `data:` entry. This is useful if you have larger contents like from configuration files which you don't want to clutter your `values.yaml` with. Dynamically computed content of ConfigMap or Secret entries due to usage of template expressions is well suited for supplying it from external files. Especially if you'd have large configuration file like content with many templating expressions it is cleaner to provide the file as external content to not clutter the `values.yaml` unnecessarily.

Using an external file here for the `_pinnedCRD` key allows to directly highlight the difference to the `inline` handling of the `settings` key via HULL because functionally the same is happening. As a Bonus you can compare the HULL transformation syntax to the regular templating expression syntax as you can use it within an externally imported file, the HULL transformations do not apply to externally imported files but only within the `values.yaml` tree!

Now you can import the second JSON transformation content via specifying it in an external file to highlight this feature. First add the external file like this:

```sh
mkdir files
```

and echo same templating instruction as above into the file:

```sh
echo '{{ toJson .Values.hull.config.specific.pinnedCRDs }}' > files/pinnedCRDs
```

Lastly add the external file as a source replacing the former `inline` directive:

```sh
sed '0,/inline: ""/{s/inline: ""/path: files\/pinnedCRDs/}' -i -- values.yaml
 ```

Then see if this way of handling data works too:

```sh
echo 'hull:
  config:
    specific:
      pinnedCRDs:
        another:
          dicionary: test' > ../configs/path-transformation.yaml \
&& helm template -f ../configs/disable-default-rbac.yaml -f ../configs/path-transformation.yaml .
```

you can again see it does actually produces the JSON content:

```yml
---
# Source: kubernetes-dashboard/templates/hull.yaml
apiVersion: v1
data:
  _global: '{}'
  _pinnedCRD: '{"another":{"dicionary":"test"}}'
kind: ConfigMap
metadata:
  annotations: {}
  labels:
    app.kubernetes.io/component: settings
    app.kubernetes.io/instance: release-name
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/name: kubernetes-dashboard
    app.kubernetes.io/part-of: undefined
    app.kubernetes.io/version: 2.5.0
    helm.sh/chart: kubernetes-dashboard-5.2.0
  name: release-name-kubernetes-dashboard-settings
```

That's it. You have learned how you can start to leverage the power of transformations - particularly the `hull.util.transformation.tpl` transformation - to improve chart creation and configuration! Done with the ConfigMap and onto the Secret!

## Specifying Secrets with HULL

Conveniently creating secrets with HULL is exactly the same process as creating ConfigMaps, all required Base64 encoding is done automatically under the hood even when using the `inline` functionality.

Do take a look at the existing Secret file in the `kubernetes-dashboard` chart by typing:

```sh
cat ../kubernetes-dashboard/templates/secret.yaml
```

which gives:

```yml
# Copyright 2017 The Kubernetes Authors.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
#     http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

# kubernetes-dashboard-certs
apiVersion: v1
kind: Secret
metadata:
  labels:
    {{- include "kubernetes-dashboard.labels" . | nindent 4 }}
    {{- if .Values.commonLabels }}
    {{- include "common.tplvalues.render" ( dict "value" .Values.commonLabels "context" $ ) | nindent 4 }}
    {{- end }}
  annotations:
    {{- if .Values.commonAnnotations }}
    {{- include "common.tplvalues.render" ( dict "value" .Values.commonAnnotations "context" $ ) | nindent 4 }}
    {{- end }}
  name: {{ template "kubernetes-dashboard.fullname" . }}-certs
type: Opaque
---
# kubernetes-dashboard-csrf
apiVersion: v1
kind: Secret
metadata:
  labels:
{{ include "kubernetes-dashboard.labels" . | nindent 4 }}
  name: kubernetes-dashboard-csrf
type: Opaque
---
# kubernetes-dashboard-key-holder
apiVersion: v1
kind: Secret
metadata:
  labels:
{{ include "kubernetes-dashboard.labels" . | nindent 4 }}
  name: kubernetes-dashboard-key-holder
type: Opaque
```

Interestingly and uncommonly you'll find three secrets without any `data` entries pre-specified or even the possibility to specify them in the chart. You'll also notice that the `csrf` and `key-holder` secrets have static names set while the name of the `certs` secret is dynamically calulated:

```yml
...
name: {{ template "kubernetes-dashboard.fullname" . }}-certs
...
name: kubernetes-dashboard-csrf
...
name: kubernetes-dashboard-key-holder
...
```

This is likely because the Kubernetes Dashboard application is bound tightly to the Kubernetes world and the application itself accesses the secrets by their static names internally as indicated [here](https://github.com/kubernetes/dashboard/blob/fe4b5fc1f4851136e868f52dbd78299b0529da79/src/app/backend/client/api/types.go#L33). In order to be able to address them by their static names you need to keep their names static too. The default naming behavior for objects in HULL is to create a dynamic name for objects. But you can always opt to create objects with a fixed/static name instead. You only need to add the `staticName: true` property to the object definition and the object name matches exactly the key name.

Check this out by adding the Secrets with empty content now to our `values.yaml` configuration with:

```sh
echo '    secret:
      certs:
        data: {}
      kubernetes-dashboard-csrf:
        data: {}
        staticName: true
      kubernetes-dashboard-key-holder:
        data: {}
        staticName: true' >> values.yaml \
&& helm template -f ../configs/disable-default-rbac.yaml .
```

and the Secrets have been created with the differing naming schemes:

```yml
---
---
# Source: kubernetes-dashboard/templates/hull.yaml
apiVersion: v1
kind: Secret
metadata:
  annotations: {}
  labels:
    app.kubernetes.io/component: certs
    app.kubernetes.io/instance: release-name
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/name: kubernetes-dashboard
    app.kubernetes.io/part-of: undefined
    app.kubernetes.io/version: 2.5.0
    helm.sh/chart: kubernetes-dashboard-5.2.0
  name: release-name-kubernetes-dashboard-certs
---
# Source: kubernetes-dashboard/templates/hull.yaml
apiVersion: v1
kind: Secret
metadata:
  annotations: {}
  labels:
    app.kubernetes.io/component: kubernetes-dashboard-csrf
    app.kubernetes.io/instance: release-name
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/name: kubernetes-dashboard
    app.kubernetes.io/part-of: undefined
    app.kubernetes.io/version: 2.5.0
    helm.sh/chart: kubernetes-dashboard-5.2.0
  name: kubernetes-dashboard-csrf
---
# Source: kubernetes-dashboard/templates/hull.yaml
apiVersion: v1
kind: Secret
metadata:
  annotations: {}
  labels:
    app.kubernetes.io/component: kubernetes-dashboard-key-holder
    app.kubernetes.io/instance: release-name
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/name: kubernetes-dashboard
    app.kubernetes.io/part-of: undefined
    app.kubernetes.io/version: 2.5.0
    helm.sh/chart: kubernetes-dashboard-5.2.0
  name: kubernetes-dashboard-key-holder
---
# Source: kubernetes-dashboard/templates/hull.yaml
apiVersion: v1
data:
  _global: '{}'
  _pinnedCRD: '{}'
kind: ConfigMap
metadata:
  annotations: {}
  labels:
    app.kubernetes.io/component: settings
    app.kubernetes.io/instance: release-name
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/name: kubernetes-dashboard
    app.kubernetes.io/part-of: undefined
    app.kubernetes.io/version: 2.5.0
    helm.sh/chart: kubernetes-dashboard-5.2.0
  name: release-name-kubernetes-dashboard-settings
```

In the same vein it is a good time to discuss a transformation that allows to create dynamic names from a static string in other places of charts where objects are being refered to by name and not where they themselves are being named.

### Utilizing the `hull.util.transformation.makefullname`/`_HT^` transformation to produce chart specific dynamic names

As touched upon, a common scenario when writing Helm charts is to produce dynamic names for all the Helm charts objects. Regularly the dynamic name consists of a static object name part and a dynamic prefix where the chart and/or release name is prefixed, e.g. assume the static instance name is `settings`, the chart name `my-application` and the release name `my-release` and you may get `my-application-my-release-settings` as a name for a ConfigMap maybe.

The reasoning behind these dynamic names is that they are supposed to be unique: even when you deploy one chart twice in the same namespace you would have to give the releases individual names, otherwise Helm would fail since it does not allow two releases with the same name in a single namespace. Even though it would be easy to produce counter examples this suffices in the practical world to have no naming clashes of objects managed via Helm charts.

Another configuration option commonly offered in Helm charts is to allow the user to specify a so-called `fullnameOverride` which then replaces the `my-application-my-release` prefix with a static prefix of your choosing. While this raises the chance of object instance name collisions obviously it can help with object names becoming unnecessarily long (there is a length cap of at most 63 chars for object names in Kubernetes).

By default all objects that HULL creates and manages have a unique dynamic name and there clearly defined places in Kubernetes object specifications where you address another object of a known and fixed type and there it suffices to supply just the object key name and HULL automatically converts it to the full dynamic name on rendering.

The places where dynamic names are created from key names automatically are:

- `horizontalpodautoscaler`:
  - the `scaleTargetRef.name`
- `ingress`:
  - the `tls.secretName`
  - the `backend.service.name`
- `container`:
  - the `env.valueFrom.configMapKeyRef`
  - the `env.valueFrom.secretKeyRef`
  - the `envFrom.configMapRef`
  - the `envFrom.secretRef`
- `volume`:
  - the `configMap.name`
  - the `secret.secretName`
  - the `persistentVolumeClaim.claimName`

In all these places you can simply use the key of the instance you address and HULL will by default create a dynamic name. However, if it is not a dynamic name you want to reference to you can always add `staticName: true` next to the property holding the instance name and the value will not be modified on rendering. Examples of this in use will follow.

However in some other situations you may also need to reference a Kubernetes object (created within the same chart) by its name and it is not as clear in advance that the value is a reference to a Kubernetes object by name. Think the configuration of RoleBindings, [here is the Kubernetes documentation example of this](https://kubernetes.io/docs/reference/access-authn-authz/rbac/#rolebinding-example):

```yml
apiVersion: rbac.authorization.k8s.io/v1
# This role binding allows "jane" to read pods in the "default" namespace.
# You need to already have a Role named "pod-reader" in that namespace.
kind: RoleBinding
metadata:
  name: read-pods
  namespace: default
subjects:
# You can specify more than one "subject"
- kind: User
  name: jane # "name" is case sensitive
  apiGroup: rbac.authorization.k8s.io
roleRef:
  # "roleRef" specifies the binding to a Role / ClusterRole
  kind: Role #this must be Role or ClusterRole
  name: pod-reader # this must match the name of the Role or ClusterRole you wish to bind to
  apiGroup: rbac.authorization.k8s.io
```

If `pod-reader` would be created within the Helm chart the RoleBinding is defined in you need the capability to enforce a dynamic name in the `roleRef`. This is where the `_HT^` transformation steps in to help you out by providing the same dynamic naming mechanism explained above. You only need to feed it the key name and it will produce a dynamic fullname based on the key name and the charts configuration. If you want to use the `fullnameOverride` functionality you can also specify the `fullnameOverride` under the `hull.config.general.fullnameOverride` configuration property and it will be respected in the creation of all dynamic names. This will be put to good use in the tutorial part on RBAC configuration (where there is a lot of referencing of object instances by name) so the example can be skipped here and left for later.

Besides the `_HT!`, `_HT?` and `_HT^` transformations there is one more helpful transformation that concludes the out-of-the-box transformations that HULL provides: the `hull.util.transformation.get`/`_HT*` get transformation.

### Getting values with the `hull.util.transformation.get`/`_HT*` transformation

Similar to the `_HT?` transformation you can utilize the `_HT*` transformation to directly get a value via the dot notation path to the source of the reference. When using `_HT*` you can specify the path to the value to get relative to the `.Values` dictionary that means you explicitly don't specify any `.Values` or `(index . "$").Values` prefix. This works for referenced values of string, integer and boolean types and even for arrays or dictionaries.

Remember the metadata setting usecase where you may want to apply a given set of labels to all objects of a particular 'group'? With `_HT*` this now becomes possible as in the next example.

By now you have defined a `settings` ConfigMap and the three Secrets `certs`, `kubernetes-dashboard-csrf` and `kubernetes-dashboard-key-holder`. Say the `settings` ConfigMap and the `certs` Secret should be grouped by a common set of labels indicating they are relevant for cluster-level configuration. Additionally the `kubernetes-dashboard-csrf` and `kubernetes-dashboard-key-holder` Secrets are to be marked as relevant on application-level. This is not required for the chart conversion but may serve as a fictional grouping example. 

Now in this demo sketch the shared labels under `hull.config.specific`:

```sh
echo 'hull:
  config:
    specific:
      labels:
        configuration:
          cluster:
            configuration: "true"
            relevancy: cluster-level
          application:
            configuration: "true"
            relevancy: application-level
  objects:
    configmap:
      settings:
        labels: _HT*hull.config.specific.labels.configuration.cluster
    secret:
      certs:
        labels: _HT*hull.config.specific.labels.configuration.cluster
      kubernetes-dashboard-csrf:
        labels: _HT*hull.config.specific.labels.configuration.application
      kubernetes-dashboard-key-holder:
        labels: _HT*hull.config.specific.labels.configuration.application'> ../configs/get-transformation.yaml \
&& helm template -f ../configs/disable-default-rbac.yaml -f ../configs/get-transformation.yaml .
```

we have successfully grouped objects by a shared set of labels:

```yml
---
# Source: kubernetes-dashboard/templates/hull.yaml
apiVersion: v1
kind: Secret
metadata:
  annotations: {}
  labels:
    app.kubernetes.io/component: certs
    app.kubernetes.io/instance: release-name
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/name: kubernetes-dashboard
    app.kubernetes.io/part-of: undefined
    app.kubernetes.io/version: 2.5.0
    configuration: true
    helm.sh/chart: kubernetes-dashboard-5.2.0
    relevancy: cluster-level
  name: release-name-kubernetes-dashboard-certs
---
# Source: kubernetes-dashboard/templates/hull.yaml
apiVersion: v1
kind: Secret
metadata:
  annotations: {}
  labels:
    app.kubernetes.io/component: kubernetes-dashboard-csrf
    app.kubernetes.io/instance: release-name
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/name: kubernetes-dashboard
    app.kubernetes.io/part-of: undefined
    app.kubernetes.io/version: 2.5.0
    configuration: true
    helm.sh/chart: kubernetes-dashboard-5.2.0
    relevancy: application-level
  name: kubernetes-dashboard-csrf
---
# Source: kubernetes-dashboard/templates/hull.yaml
apiVersion: v1
kind: Secret
metadata:
  annotations: {}
  labels:
    app.kubernetes.io/component: kubernetes-dashboard-key-holder
    app.kubernetes.io/instance: release-name
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/name: kubernetes-dashboard
    app.kubernetes.io/part-of: undefined
    app.kubernetes.io/version: 2.5.0
    configuration: true
    helm.sh/chart: kubernetes-dashboard-5.2.0
    relevancy: application-level
  name: kubernetes-dashboard-key-holder
---
# Source: kubernetes-dashboard/templates/hull.yaml
apiVersion: v1
data:
  _global: '{}'
  _pinnedCRD: '{}'
kind: ConfigMap
metadata:
  annotations: {}
  labels:
    app.kubernetes.io/component: settings
    app.kubernetes.io/instance: release-name
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/name: kubernetes-dashboard
    app.kubernetes.io/part-of: undefined
    app.kubernetes.io/version: 2.5.0
    configuration: true
    helm.sh/chart: kubernetes-dashboard-5.2.0
    relevancy: cluster-level
  name: release-name-kubernetes-dashboard-settings
```

## Wrap up

This concludes the subject of handling ConfigMaps and Secrets within HULL. Along the way you have learned about HULL transformations which put powerful templating into the `values.yaml` processing. At this point feel free to play around with the already created example files in the `config` folder and check if you get the expected output rendered. Of course you can also create new ones for testing the functionality!

Your `values.yaml` should at this stage look like this:

```sh
cat values.yaml
```
```yml
metrics-server:
  enabled: false
hull:
  config:
    specific:
      settings: {}
      pinnedCRDs: {}
  objects:
    configmap:
      settings:
        data:
          _global:
            inline: _HT!{{ toJson (index . "$").Values.hull.config.specific.settings | quote }}
          _pinnedCRD:
            path: files/pinnedCRDs
    secret:
      certs:
        data: {}
      kubernetes-dashboard-csrf:
        data: {}
        staticName: true
      kubernetes-dashboard-key-holder:
        data: {}
        staticName: true
```

Pretty compact. The 80 lines of configuration in the original `kubernetes-dashboard` chart that were spent to define the ConfigMap and the Secrets were reduced to 20 and that is even excluding the relevant lines from the original `values.yaml`. That means a reduction of more than half of the lines of code that require maintenance. Yet at the same time you are gaining the full possibility to overwrite any objects property. When working on the workload objects (Deployments, DaemonSet and Jobs) this reduction of the code to be maintained is even more visible and you will see that you need roughly half the code lines then to achieve even more than what is possible with the original chart.

Now is the right time to store your current `values.yaml` as `values.tutorial-part.yaml` in the charts folder:

```sh
cp values.yaml values.tutorial-part.yaml
```

This file is supposed to hold the individual outcome of each tutorial part from now on, concentrating on the aspects that are in focus of the respective tutorial part. 

Additionally a `values.full.yaml` will from now on be maintained and will be built upon in each tutorial part and become the final complete `values.yaml` for the chart. Basically it represents the outcome of the whole tutorial series. Within this constantly growing file all intermediate tutorial results are being added. For this first tutorial where first objects were created it is the same as the `values.intermediate.yaml` obviously but starting from the next tutorial part it will grow further. So at this time just copy the `values.yaml` too:

```sh
cp values.yaml values.full.yaml
```

By backing up your progress this way the 'working' `values.yaml` can be reset or modified for each tutorial part to limit the output to look at (since we do not require all objects that were created so far each time) but only those in focus of a tutorial part.

Otherwise that is the end of this tutorial part, the outcome of it is found [here](https://github.com/vidispine/hull-tutorials/tree/test/dev-to/hull/03_configmaps_and_secrets). [The next tutorial looks at configuration of ServiceAccounts and RBAC matters](https://dev.to/gre9ory/hull-tutorial-04-configuring-serviceaccounts-and-rbac-153a).

__Thanks for reading and hopefully you found it useful and/or interesting!__