Repository: influxdata/telegraf-operator (go)

Issue:
Support setting global tags from labels, annotations, etc
I'd like to be able to set global tags based on annotations/labels/k8s namespace/etc. More specifically, I'd like to define a class like this:

```yaml
classes:
  data:
    foo: |+
      [global_tags]
        hostname = "$HOSTNAME"
        nodename = "$NODENAME"
        app = "$KUBERNETES_LABEL_APP"
```

To be clear, obviously I can _already_ define a class like that, but it won't work because there's no way to get the environment variable `$KUBERNETES_LABEL_APP` populated in the Telegraf container. If I was creating the container myself I'd be able to do something like,
```
env:
  - name: KUBERNETES_LABEL_APP
    valueFrom:
      fieldRef:
        fieldPath: metadata.labels['app']
```
But there's no way to set up anything like that when the container is injected automatically by the Operator. 

As an alternative, being able to just specify `telegraf.influxdata.com/include-labels: 'app,whatever'` or `telegraf.influxdata.com/tags: 'app=foo,whatever=bar'` as an annotation and have the Operator automatically construct...

```toml
[global_tags]
app = "foo"
whatever = "bar"
```
...would also be pretty good, but I get the feeling that could get awkward to implement since you can't define the same table multiple times in a TOML document, so if the class defines a `[global_tags]` table as well then they'd clash.


Pull request:
fix: Support setting global tags from labels, annotations, etc (#44)
Closes #44

I made the implementation as discussed on #44 

- [x] Signed the [InfluxData CLA](https://www.influxdata.com/legal/cla/)
  ([if necessary](https://github.com/influxdata/telegraf-operator/blob/master/CONTRIBUTING.md#sign-the-influxdata-cla))
- [x] Tests pass (no build errors)
- [x] Rebased/mergeable


Patch:
diff --git a/Makefile b/Makefile
index cbc6220..228bc9c 100644
--- a/Makefile
+++ b/Makefile
@@ -112,8 +112,7 @@ kind-test:
 	kubectl config use-context kind-telegraf-operator-test
 	kubectl apply -f examples/classes.yml
 	kubectl apply -f deploy/dev.yml
-	# wait 20 seconds to ensure the pod is already in running state
-	sleep 20
+	kubectl wait --timeout=5m -n telegraf-operator --for=condition=available deployment/telegraf-operator
 	kubectl apply -f examples/redis.yml
 	sleep 2
 	kubectl describe pod --namespace=test redis
diff --git a/README.md b/README.md
index 154c154..ffee30f 100644
--- a/README.md
+++ b/README.md
@@ -194,12 +194,57 @@ Additional pod annotations that can be used to configure the Telegraf sidecar:
 - `telegraf.influxdata.com/image` : is used to configure telegraf image to be used for the `telegraf` sidecar container
 - `telegraf.influxdata.com/class` : configures which kind of class to use (classes are configured on the operator)
 - `telegraf.influxdata.com/secret-env` : allows adding secrets to the telegraf sidecar in the form of environment variables
+- `telegraf.influxdata.com/env-configmapkeyref-<VARIABLE_NAME>` : allows adding configmap key references to the telegraf sidecar in the form of an environment variable
+- `telegraf.influxdata.com/env-fieldref-<VARIABLE_NAME>` : allows adding fieldref references to the telegraf sidecar in the form of an environment variable
+- `telegraf.influxdata.com/env-literal-<VARIABLE_NAME>` : allows adding a literal to the telegraf sidecar in the form of an environment variable
+- `telegraf.influxdata.com/env-secretkeyref-<VARIABLE_NAME>` : allows adding secret key references to the telegraf sidecar in the form of an environment variable
 - `telegraf.influxdata.com/requests-cpu` : allows specifying resource requests for CPU
 - `telegraf.influxdata.com/requests-memory` : allows specifying resource requests for memory
 - `telegraf.influxdata.com/limits-cpu` : allows specifying resource limits for CPU
 - `telegraf.influxdata.com/limits-memory` : allows specifying resource limits for memory
 
 
+##### Example of extra additional options
+
+```
+apiVersion: apps/v1
+kind: StatefulSet
+  # ...
+spec:
+  template:
+    metadata:
+      labels:
+        app: redis
+      annotations:
+        telegraf.influxdata.com/env-fieldref-NAMESPACE: metadata.namespace
+        telegraf.influxdata.com/env-fieldref-APP: metadata.labels['app']
+        telegraf.influxdata.com/env-configmapkeyref-REDIS_SERVER: configmap-name.redis.url
+        telegraf.influxdata.com/env-secretkeyref-PASSWORD: app-secret.redis.password
+        telegraf.influxdata.com/env-literal-VERSION: "1.0"
+        telegraf.influxdata.com/inputs: |+
+          [[inputs.redis]]
+            servers = ["$REDIS_SERVER"]
+            password = "$PASSWORD"
+      # ...
+    spec:
+      containers:
+      # ...
+```
+These annotations result in additional environment variables available for the telegraf container, which can be used in for example the tags.   
+And they can be used in the additional input configuration provided in the annotation as shown above.
+
+```
+stringData:
+  basic: |+
+    [global_tags]
+      hostname = "$HOSTNAME"
+      nodename = "$NODENAME"
+      namespace = "$NAMESPACE"
+      app = "$APP"
+      version = "$VERSION"
+```
+
+
 # Contributing to telegraf-operator
 
 Please read the [CONTRIBUTING](CONTRIBUTING.md) file for more details on how to get started with contributing to to `telegraf-operator`.
diff --git a/examples/classes.yml b/examples/classes.yml
index cee2683..2e6a369 100644
--- a/examples/classes.yml
+++ b/examples/classes.yml
@@ -19,6 +19,7 @@ stringData:
     [global_tags]
       hostname = "$HOSTNAME"
       nodename = "$NODENAME"
+      namespace = "$NAMESPACE"
       type = "app"
   basic: |+
     [[outputs.influxdb]]
diff --git a/examples/redis.yml b/examples/redis.yml
index f010e78..0e4a529 100644
--- a/examples/redis.yml
+++ b/examples/redis.yml
@@ -17,6 +17,7 @@ spec:
           [[inputs.redis]]
             servers = ["tcp://localhost:6379"]
         telegraf.influxdata.com/class: app
+        telegraf.influxdata.com/env-fieldref-NAMESPACE: metadata.namespace
         telegraf.influxdata.com/limits-cpu: '750m'
         # invalid memory limit, which will be ignored
         telegraf.influxdata.com/limits-memory: '800x'
diff --git a/sidecar.go b/sidecar.go
index 59da1ca..9d8cb53 100644
--- a/sidecar.go
+++ b/sidecar.go
@@ -45,6 +45,14 @@ const (
 	TelegrafClass = "telegraf.influxdata.com/class"
 	// TelegrafSecretEnv allows adding secrets to the telegraf sidecar in the form of environment variables
 	TelegrafSecretEnv = "telegraf.influxdata.com/secret-env"
+	// TelegrafEnvFieldRefPrefix allows adding fieldref references to the telegraf sidecar in the form of an environment variable
+	TelegrafEnvFieldRefPrefix = "telegraf.influxdata.com/env-fieldref-"
+	// TelegrafEnvConfigMapKeyRefPrefix allows adding configmap key references to the telegraf sidecar in the form of an environment variable
+	TelegrafEnvConfigMapKeyRefPrefix = "telegraf.influxdata.com/env-configmapkeyref-"
+	// TelegrafEnvSecretKeyRefPrefix allows adding secret key references to the telegraf sidecar in the form of an environment variable
+	TelegrafEnvSecretKeyRefPrefix = "telegraf.influxdata.com/env-secretkeyref-"
+	// TelegrafEnvLiteralPrefix allows adding a literal to the telegraf sidecar in the form of an environment variable
+	TelegrafEnvLiteralPrefix = "telegraf.influxdata.com/env-literal-"
 	// TelegrafImage allows specifying a custom telegraf image to be used in the sidecar container
 	TelegrafImage = "telegraf.influxdata.com/image"
 	// TelegrafRequestsCPU allows specifying custom CPU resource requests
@@ -417,9 +425,79 @@ func (h *sidecarHandler) newContainer(pod *corev1.Pod, containerName string) (co
 			},
 		}
 	}
+
+	envFieldRef := AnnotationsWithPrefix(pod.Annotations, TelegrafEnvFieldRefPrefix)
+	for name, fieldPath := range envFieldRef {
+		baseContainer.Env = append(baseContainer.Env, corev1.EnvVar{
+			Name: name,
+			ValueFrom: &corev1.EnvVarSource{
+				FieldRef: &corev1.ObjectFieldSelector{
+					FieldPath: fieldPath,
+				},
+			},
+		})
+	}
+
+	literals := AnnotationsWithPrefix(pod.Annotations, TelegrafEnvLiteralPrefix)
+	for name, value := range literals {
+		baseContainer.Env = append(baseContainer.Env, corev1.EnvVar{
+			Name:  name,
+			Value: value,
+		})
+	}
+
+	configMapKeyRefs := AnnotationsWithPrefix(pod.Annotations, TelegrafEnvConfigMapKeyRefPrefix)
+	for name, value := range configMapKeyRefs {
+		selector := strings.SplitN(value, ".", 2)
+		if len(selector) == 2 {
+			baseContainer.Env = append(baseContainer.Env, corev1.EnvVar{
+				Name: name,
+				ValueFrom: &corev1.EnvVarSource{
+					ConfigMapKeyRef: &corev1.ConfigMapKeySelector{
+						LocalObjectReference: corev1.LocalObjectReference{
+							Name: selector[0],
+						},
+						Key: selector[1],
+					},
+				},
+			})
+		} else {
+			h.Logger.Info("unable to parse configmapkeyref %s with value of \"%s\"", name, value)
+		}
+	}
+
+	secretKeyRefs := AnnotationsWithPrefix(pod.Annotations, TelegrafEnvSecretKeyRefPrefix)
+	for name, value := range secretKeyRefs {
+		selector := strings.SplitN(value, ".", 2)
+		if len(selector) == 2 {
+			baseContainer.Env = append(baseContainer.Env, corev1.EnvVar{
+				Name: name,
+				ValueFrom: &corev1.EnvVarSource{
+					SecretKeyRef: &corev1.SecretKeySelector{
+						LocalObjectReference: corev1.LocalObjectReference{
+							Name: selector[0],
+						},
+						Key: selector[1],
+					},
+				},
+			})
+		} else {
+			h.Logger.Info("unable to parse secretkeyref %s with value of \"%s\"", name, value)
+		}
+	}
 	return baseContainer, nil
 }
 
+func AnnotationsWithPrefix(annotations map[string]string, prefix string) map[string]string {
+	filtered := make(map[string]string)
+	for k, v := range annotations {
+		if strings.HasPrefix(k, prefix) {
+			filtered[strings.TrimPrefix(k, prefix)] = v
+		}
+	}
+	return filtered
+}
+
 func (h *sidecarHandler) newIstioContainer(pod *corev1.Pod, containerName string) (corev1.Container, error) {
 	var parsedRequestsCPU resource.Quantity
 	var parsedRequestsMemory resource.Quantity


Test patch:
diff --git a/sidecar_test.go b/sidecar_test.go
index 6d74fe7..d091a7a 100644
--- a/sidecar_test.go
+++ b/sidecar_test.go
@@ -907,6 +907,251 @@ status: {}
 `,
 			wantSecrets: []string{testEmptySecret},
 		},
+		{
+			name: "validate secret-env annotation usage creation",
+			pod: &corev1.Pod{
+				ObjectMeta: metav1.ObjectMeta{
+					Annotations: map[string]string{
+						TelegrafSecretEnv: "mysecret",
+					},
+				},
+			},
+			wantPod: `
+metadata:
+  annotations:
+    telegraf.influxdata.com/secret-env: mysecret
+  creationTimestamp: null
+spec:
+  containers:
+  - command:
+    - telegraf
+    - --config
+    - /etc/telegraf/telegraf.conf
+    env:
+    - name: NODENAME
+      valueFrom:
+        fieldRef:
+          fieldPath: spec.nodeName
+    envFrom:
+    - secretRef:
+        name: mysecret
+        optional: true
+    image: docker.io/library/telegraf:1.19
+    name: telegraf
+    resources:
+      limits:
+        cpu: 200m
+        memory: 200Mi
+      requests:
+        cpu: 10m
+        memory: 10Mi
+    volumeMounts:
+    - mountPath: /etc/telegraf
+      name: telegraf-config
+  volumes:
+  - name: telegraf-config
+    secret:
+      secretName: telegraf-config-myname
+status: {}
+      `,
+			wantSecrets: []string{testEmptySecret},
+		},
+		{
+			name: "validate env-fieldref- annotation usage creation",
+			pod: &corev1.Pod{
+				ObjectMeta: metav1.ObjectMeta{
+					Annotations: map[string]string{
+						TelegrafEnvFieldRefPrefix + "NAMESPACE": "metadata.namespace",
+					},
+				},
+			},
+			wantPod: `
+metadata:
+  annotations:
+    telegraf.influxdata.com/env-fieldref-NAMESPACE: metadata.namespace
+  creationTimestamp: null
+spec:
+  containers:
+  - command:
+    - telegraf
+    - --config
+    - /etc/telegraf/telegraf.conf
+    env:
+    - name: NODENAME
+      valueFrom:
+        fieldRef:
+          fieldPath: spec.nodeName
+    - name: NAMESPACE
+      valueFrom:
+        fieldRef:
+          fieldPath: metadata.namespace
+    image: docker.io/library/telegraf:1.19
+    name: telegraf
+    resources:
+      limits:
+        cpu: 200m
+        memory: 200Mi
+      requests:
+        cpu: 10m
+        memory: 10Mi
+    volumeMounts:
+    - mountPath: /etc/telegraf
+      name: telegraf-config
+  volumes:
+  - name: telegraf-config
+    secret:
+      secretName: telegraf-config-myname
+status: {}
+      `,
+			wantSecrets: []string{testEmptySecret},
+		},
+		{
+			name: "validate env-literal- annotation usage creation",
+			pod: &corev1.Pod{
+				ObjectMeta: metav1.ObjectMeta{
+					Annotations: map[string]string{
+						TelegrafEnvLiteralPrefix + "STACK_VERSION": "1.0",
+					},
+				},
+			},
+			wantPod: `
+metadata:
+  annotations:
+    telegraf.influxdata.com/env-literal-STACK_VERSION: "1.0"
+  creationTimestamp: null
+spec:
+  containers:
+  - command:
+    - telegraf
+    - --config
+    - /etc/telegraf/telegraf.conf
+    env:
+    - name: NODENAME
+      valueFrom:
+        fieldRef:
+          fieldPath: spec.nodeName
+    - name: STACK_VERSION
+      value: "1.0"
+    image: docker.io/library/telegraf:1.19
+    name: telegraf
+    resources:
+      limits:
+        cpu: 200m
+        memory: 200Mi
+      requests:
+        cpu: 10m
+        memory: 10Mi
+    volumeMounts:
+    - mountPath: /etc/telegraf
+      name: telegraf-config
+  volumes:
+  - name: telegraf-config
+    secret:
+      secretName: telegraf-config-myname
+status: {}
+      `,
+			wantSecrets: []string{testEmptySecret},
+		},
+		{
+			name: "validate env-configmapkeyref- annotation usage creation",
+			pod: &corev1.Pod{
+				ObjectMeta: metav1.ObjectMeta{
+					Annotations: map[string]string{
+						TelegrafEnvConfigMapKeyRefPrefix + "VERSION": "configmap-name.application.version",
+					},
+				},
+			},
+			wantPod: `
+metadata:
+  annotations:
+    telegraf.influxdata.com/env-configmapkeyref-VERSION: configmap-name.application.version
+  creationTimestamp: null
+spec:
+  containers:
+  - command:
+    - telegraf
+    - --config
+    - /etc/telegraf/telegraf.conf
+    env:
+    - name: NODENAME
+      valueFrom:
+        fieldRef:
+          fieldPath: spec.nodeName
+    - name: VERSION
+      valueFrom:
+        configMapKeyRef:
+          key: application.version
+          name: configmap-name
+    image: docker.io/library/telegraf:1.19
+    name: telegraf
+    resources:
+      limits:
+        cpu: 200m
+        memory: 200Mi
+      requests:
+        cpu: 10m
+        memory: 10Mi
+    volumeMounts:
+    - mountPath: /etc/telegraf
+      name: telegraf-config
+  volumes:
+  - name: telegraf-config
+    secret:
+      secretName: telegraf-config-myname
+status: {}
+      `,
+			wantSecrets: []string{testEmptySecret},
+		},
+		{
+			name: "validate env-secretref- annotation usage creation",
+			pod: &corev1.Pod{
+				ObjectMeta: metav1.ObjectMeta{
+					Annotations: map[string]string{
+						TelegrafEnvSecretKeyRefPrefix + "PASSWORD": "app-secret.user.password",
+					},
+				},
+			},
+			wantPod: `
+metadata:
+  annotations:
+    telegraf.influxdata.com/env-secretkeyref-PASSWORD: app-secret.user.password
+  creationTimestamp: null
+spec:
+  containers:
+  - command:
+    - telegraf
+    - --config
+    - /etc/telegraf/telegraf.conf
+    env:
+    - name: NODENAME
+      valueFrom:
+        fieldRef:
+          fieldPath: spec.nodeName
+    - name: PASSWORD
+      valueFrom:
+        secretKeyRef:
+          key: user.password
+          name: app-secret
+    image: docker.io/library/telegraf:1.19
+    name: telegraf
+    resources:
+      limits:
+        cpu: 200m
+        memory: 200Mi
+      requests:
+        cpu: 10m
+        memory: 10Mi
+    volumeMounts:
+    - mountPath: /etc/telegraf
+      name: telegraf-config
+  volumes:
+  - name: telegraf-config
+    secret:
+      secretName: telegraf-config-myname
+status: {}
+      `,
+			wantSecrets: []string{testEmptySecret},
+		},
 	}
 
 	for _, tt := range tests {