Repository: banzaicloud/logging-operator (go)

Issue:
fields column in Splunk Hec output plugin is not working . 
**Describe the bug**:
fields column in Splunk Hec output plugin is not converting as per fluentd spec in fluentd.conf in secret file 

**Expected behaviour**:
```
kind: ClusterOutput
metadata:
 name: splunk-output
spec:
 splunkHec:
    hec_host: http-inputs-
    hec_port: 443
    hec_token: 
        valueFrom:
           secretKeyRef:
              name:  splunk-token
              key: SplunkHecToken
    index: qa_main
    fields: 
      dummy: " "
    format:
      type: single_value 
      message_key: MESSAGE
```
expected config in fluentd.conf ( in secrets) 
```
 <fields>
     dummy
 </fileds> 
```
what i get is 
 fields {"dummy":" "}

so logging operator cannot separate dummy from original message to an indexed field.
 
**Steps to reproduce the bug**:
```
kind: ClusterOutput
metadata:
 name: splunk-output
spec:
 splunkHec:
    hec_host: http-inputs-
    hec_port: 443
    hec_token: 
        valueFrom:
           secretKeyRef:
              name:  splunk-token
              key: SplunkHecToken
    index: qa_main
    fields: 
      dummy: " "
    format:
      type: single_value 
      message_key: MESSAGE
 ```

**Additional context**:
the main idea is to separate dummy from the original message to an indexed field in Splunk. But when using above config there is no change in the original message. but when I edited secrets to change fluentd.conf  to  
```
 <fields>
     dummy
 </fileds> 
```
 instead of  fields {"dummy":" "}  then I could see the change in my Splunk output . So I doubt that Splunk output plugin in logging operator is not writing the correct format to underlying fluentd.conf 
**Environment details**:
- Kubernetes version (v1.15.11):
- Cloud-provider/provisioner ( EKS):
- logging operator version : 3.2.0-rc1 
- Install method (static manifests):
- Resource definition (possibly in YAML format) that caused the issue, without sensitive data:
```
apiVersion: logging.banzaicloud.io/v1beta1
kind: ClusterOutput
metadata:
 name: splunk-output
spec:
 splunkHec:
    hec_host: http-splunkcloud.com
    hec_port: 443
    hec_token:
        valueFrom:
           secretKeyRef:
              name:  splunk-token
              key: SplunkHecToken
    index: qa_main
    fields:
       dummy: " "
    format:
      type: single_value
      message_key: MESSAGE
      add_newline: true
```
/kind bug


Pull request:
fields column in Splunk Hec output plugin is not working . #505
| Q               | A
| --------------- | ---
| Bug fix?        | yes
| New feature?    | no
| API breaks?     | no
| Deprecations?   | no
| Related tickets | fixes #505 
| License         | Apache 2.0


### Checklist

- [x] Code meets the [Developer Guide](https://github.com/banzaicloud/developer-guide)
- [x] User guide and development docs updated (if needed)
- [x] Related Helm chart(s) updated (if needed)


Patch:
diff --git a/docs/plugins/outputs/splunk_hec.md b/docs/plugins/outputs/splunk_hec.md
index 5b621f0b..e2fcc837 100644
--- a/docs/plugins/outputs/splunk_hec.md
+++ b/docs/plugins/outputs/splunk_hec.md
@@ -51,6 +51,6 @@ More info at https://github.com/splunk/fluent-plugin-splunk-hec
 | ca_path | string | No | - | The path to a directory containing CA certificates in PEM format.<br> |
 | ssl_ciphers | string | No | - | List of SSL ciphers allowed.<br> |
 | insecure_ssl | *bool | No | false | Indicates if insecure SSL connection is allowed <br> |
-| fields | map[string]string | No | - | In this case, parameters inside <fields> are used as indexed fields and removed from the original input events<br> |
+| fields | Fields | No | - | In this case, parameters inside <fields> are used as indexed fields and removed from the original input events<br> |
 | format | *Format | No | - | [Format](../format/)<br> |
 | buffer | *Buffer | No | - | [Buffer](../buffer/)<br> |
diff --git a/pkg/sdk/model/output/splunk_hec.go b/pkg/sdk/model/output/splunk_hec.go
index 0214b2a8..0d6417bd 100644
--- a/pkg/sdk/model/output/splunk_hec.go
+++ b/pkg/sdk/model/output/splunk_hec.go
@@ -107,13 +107,26 @@ type SplunkHecOutput struct {
 	// Indicates if insecure SSL connection is allowed (default:false)
 	InsecureSSL *bool `json:"insecure_ssl,omitempty"`
 	// In this case, parameters inside <fields> are used as indexed fields and removed from the original input events
-	Fields map[string]string `json:"fields,omitempty"`
+	Fields Fields `json:"fields,omitempty"`
 	// +docLink:"Format,../format/"
 	Format *Format `json:"format,omitempty"`
 	// +docLink:"Buffer,../buffer/"
 	Buffer *Buffer `json:"buffer,omitempty"`
 }
 
+type Fields map[string]string
+
+func (r Fields) ToDirective(secretLoader secret.SecretLoader, id string) (types.Directive, error) {
+	recordSet := types.PluginMeta{
+		Directive: "fields",
+	}
+	directive := &types.GenericDirective{
+		PluginMeta: recordSet,
+		Params:     r,
+	}
+	return directive, nil
+}
+
 func (c *SplunkHecOutput) ToDirective(secretLoader secret.SecretLoader, id string) (types.Directive, error) {
 	pluginType := "splunk_hec"
 	splunkHec := &types.OutputPlugin{
@@ -150,6 +163,14 @@ func (c *SplunkHecOutput) ToDirective(secretLoader secret.SecretLoader, id strin
 		}
 	}
 
+	if c.Fields != nil {
+		if meta, err := c.Fields.ToDirective(secretLoader, ""); err != nil {
+			return nil, err
+		} else {
+			splunkHec.SubDirectives = append(splunkHec.SubDirectives, meta)
+		}
+	}
+
 	return splunkHec, nil
 }
 
diff --git a/pkg/sdk/model/output/zz_generated.deepcopy.go b/pkg/sdk/model/output/zz_generated.deepcopy.go
index 4a94acf8..6820b89c 100644
--- a/pkg/sdk/model/output/zz_generated.deepcopy.go
+++ b/pkg/sdk/model/output/zz_generated.deepcopy.go
@@ -980,7 +980,7 @@ func (in *SplunkHecOutput) DeepCopyInto(out *SplunkHecOutput) {
 	}
 	if in.Fields != nil {
 		in, out := &in.Fields, &out.Fields
-		*out = make(map[string]string, len(*in))
+		*out = make(Fields, len(*in))
 		for key, val := range *in {
 			(*out)[key] = val
 		}


Test patch:
diff --git a/pkg/sdk/model/output/splunk_hec_test.go b/pkg/sdk/model/output/splunk_hec_test.go
index 53f8f512..ea06df8a 100644
--- a/pkg/sdk/model/output/splunk_hec_test.go
+++ b/pkg/sdk/model/output/splunk_hec_test.go
@@ -28,6 +28,8 @@ hec_host: splunk.default.svc.cluster.local
 hec_port: 8088
 insecure_ssl: true
 index: foo
+fields:
+  dummy: " "
 host: foo
 protocol: http
 metrics_from_event: true
@@ -55,6 +57,9 @@ source: foo
 		protocol http
 		source foo
 		sourcetype foo
+		<fields>
+		  dummy  
+		</fields>
 	</match>
 `
 	es := &output.SplunkHecOutput{}