Pod uses dynamic environment variable
Tags: kubernetes, csi, dell, isilon
Published: 2020-07-06 09:06:51

---
title: Pod uses dynamic environment variable
published: true
date: 2020-06-19 07:00:00 UTC
tags: #kubernetes #csi #dell #isilon
canonical_url: https://storage-chaos.io/dell-csi-isilon-custom-envvars.html
---

# TL; DR

This post is build-up on the [Merge ConfigMap and Secrets](/configmap-and-secret.html) post.It is another use of [initContainers](https://kubernetes.io/docs/concepts/workloads/pods/init-containers/), [templating](https://docs.ruby-lang.org/en/master/ERB.html) and [entrypoint](https://docs.docker.com/engine/reference/builder/#entrypoint) to customize a container startup.

# The premise

I worked on a Kubernetes architecture where the hosts of the cluster had several NIC Cards connected to different networks (one to expose the services, one for management, one for storage, etc.).

When to create and mount an NFS volume, the [CSI driver for PowerScale/Isilon](https://github.com/dell/csi-isilon/) passes a client IP that is used to create the export array-side.The driver picks the IP return by the fieldRef<sup id="fnref:1" role="doc-noteref"><a href="#fn:1">1</a></sup> `status.hostIP`, as you can see [here](https://github.com/dell/csi-isilon/blob/master/helm/csi-isilon/templates/node.yaml#L113)).

The problem is that IP is used to serve Kubernetes services (aka the Internal IP displayed by `kubectl get node -o wide`). So how to change that value to use the storage network-related IP ?

# The implementation

In my setup, I know which NIC card connects to which network (in this case `ens33`).The patch to the native [csi-isilon](https://github.com/dell/csi-isilon/) deployment aims to :

1. Have a simple way to get the IP address of a specific NIC card
2. Pass that information on the driver startup

The first piece of configuration is to create a custom [entrypoint](https://docs.docker.com/engine/reference/builder/#entrypoint) that will set the `X_NODE_IP` variable with the proper.

Here I use an [ERB](https://docs.ruby-lang.org/en/master/ERB.html) template in which I call the `ip addr` [command in a subshell](https://ruby-doc.org/core-2.7.1/Kernel.html#method-i-60) with `%x@ @` syntax, then I extract the IP with the [substring](https://ruby-doc.org/core-2.7.1/String.html#method-i-5B-5D) `[/inet\s+(\d+(\.\d+){3})/,1]`. If you use IPv6 or another NIC card you can easily adjust it at line 9 of the following snippet.

It is not displayed in the configuration above, but the `ip addr` command works because the Isilon Node Pod has access to the host network thanks to `hostNetwork: true` in its definition.

<noscript><pre>400: Invalid request</pre></noscript><script src="https://gist.github.com/d4260e4e17910bbdefbbecce4286074f.js?file=nodeip-configmap.yaml"> </script>

The second step is to add an [initContainers](https://kubernetes.io/docs/concepts/workloads/pods/init-containers/) to the [DaemonSet](https://github.com/dell/csi-isilon/blob/master/helm/csi-isilon/templates/node.yaml) to generate a new entrypoint, and then force the driver Pod to use the new entrypoint :

<noscript><pre>400: Invalid request</pre></noscript><script src="https://gist.github.com/d4260e4e17910bbdefbbecce4286074f.js?file=isilon-ds.yaml"> </script>

To apply the patch you can create the config map with :

```
kubectl create -f nodeip-configmap.yaml

```

And patch the Isilon daemon set with :

```
kubectl patch daemonset isilon-node -n isilon --patch "$(cat isilon-ds.patch)"

```

# Wrap-up

The same tools (ERB, ConfigMap, initContainer, Entrypoint), can be use to tune pretty much any Kubernetes Pod deployments to customize or add extra-capabilities to your Pod startup (integration with [Vault](https://www.vaultproject.io/), tweak program startup, etc.).

1. 

The list of fieldRef possible values is documented [here](https://kubernetes.io/docs/tasks/inject-data-application/downward-api-volume-expose-pod-information/#capabilities-of-the-downward-api), [↩︎](#fnref:1)