Repository: argoproj/argo (go)

Issue:
v2.10.0-rc4 nested maps in loop rendered as Golang map[]
Checklist:

 * [x] I've included the version.
 * [x] I've included reproduction steps.
 * [x] I've included the workflow YAML.
 * [x] I've included the logs.
 
**What happened**:
Using v2.10.0, when looping over a JSON list with nested object with `withParam`, references to nested JSON objects results in a Golang style map `%v` string rather than the nested JSON.

**What you expected to happen**:
Nested maps should remain the same as the input structure, escaped in the same way as referenced in #3409 (appears mostly fixed in 2.10.x).

**How to reproduce it (as minimally and precisely as possible)**:

Using following workflow:

```
kind: Workflow
metadata:
  generateName: nested-test-
spec:
  entrypoint: main

  templates:
  - name: main
    inputs:
      parameters:
      - name: things
        value: '[{"nested": {"B": "2 \"foo\""}}]'
    steps:
    - - name: echoitems
        template: echo
        arguments:
          parameters:
            - name: thing
              value: "{{item.nested}}"
        withParam: "{{inputs.parameters.things}}"
        
  - name: echo
    inputs:
      parameters:
        - name: thing
    container:
      image: busybox
      command: [echo]
      args: ["{{inputs.parameters.thing}}"]
```
Using `./tools/argo/argov2.10.0-rc4 submit nested-test.yaml --watch`

```
Name:                nested-test-v7v4d
Namespace:           default
ServiceAccount:      default
Status:              Succeeded
Conditions:          
 Completed           True
Created:             Fri Jul 31 19:48:27 +0100 (24 seconds ago)
Started:             Fri Jul 31 19:48:27 +0100 (24 seconds ago)
Finished:            Fri Jul 31 19:48:51 +0100 (now)
Duration:            24 seconds
ResourcesDuration:   4s*(1 cpu),4s*(100Mi memory)

STEP                                       TEMPLATE  PODNAME                       DURATION  MESSAGE
 ✔ nested-test-v7v4d                       main                                                
 └---✔ echoitems(0:nested:map[B:2 "foo"])  echo      nested-test-v7v4d-2387945936  22s
```
And, showing the workflow logs:

```
 ./tools/argo/argov2.10.0-rc4 logs nested-test-v7v4d
nested-test-v7v4d-2387945936: 2020-07-31T18:48:48.492777953Z map[B:2 "foo"]
```
In 2.9.4 I get the expected behaviour:

```
./tools/argo/argov2.9.4 logs nested-test-ptmh5
nested-test-ptmh5-4036727801: 2020-07-31T18:45:47.098717735Z {"B":"2 \"foo\""}
```

**Anything else we need to know?**:

**Environment**:

- Argo version:
```
/tools/argo/argov2.10.0-rc4 version
argo: v2.10.0-rc4+8d6dae6.dirty
  BuildDate: 2020-07-28T22:58:49Z
  GitCommit: 8d6dae6128074445d9bd0222c449643053568db8
  GitTreeState: dirty
  GitTag: v2.10.0-rc4
  GoVersion: go1.13.4
  Compiler: gc
  Platform: linux/amd64
```
- Kubernetes version :
```
clientVersion:
  buildDate: "2020-07-16T14:19:25Z"
  compiler: gc
  gitCommit: dff82dc0de47299ab66c83c626e08b245ab19037
  gitTreeState: clean
  gitVersion: v1.18.6
  goVersion: go1.13.13
  major: "1"
  minor: "18"
  platform: linux/amd64
serverVersion:
  buildDate: "2020-04-30T20:19:45Z"
  compiler: gc
  gitCommit: 52c56ce7a8272c798dbc29846288d7cd9fbae032
  gitTreeState: clean
  gitVersion: v1.18.2
  goVersion: go1.13.9
  major: "1"
  minor: "18"
  platform: linux/amd64
```

**Other debugging information (if applicable)**:

- workflow result:

```
DEBU[0000] CLI version                                   version="{v2.10.0-rc4+8d6dae6.dirty 2020-07-28T22:58:49Z 8d6dae6128074445d9bd0222c449643053568db8 v2.10.0-rc4 dirty go1.13.4 gc linux/amd64}"
DEBU[0000] Client options                                opts="{{ false false}  0x1708120 <nil> 0x1708170}"
Name:                nested-test-v7v4d
Namespace:           default
ServiceAccount:      default
Status:              Succeeded
Conditions:          
 Completed           True
Created:             Fri Jul 31 19:48:27 +0100 (15 minutes ago)
Started:             Fri Jul 31 19:48:27 +0100 (15 minutes ago)
Finished:            Fri Jul 31 19:48:51 +0100 (15 minutes ago)
Duration:            24 seconds
ResourcesDuration:   4s*(1 cpu),4s*(100Mi memory)

STEP                                       TEMPLATE  PODNAME                       DURATION  MESSAGE
 ✔ nested-test-v7v4d                       main                                                
 └---✔ echoitems(0:nested:map[B:2 "foo"])  echo      nested-test-v7v4d-2387945936  22s
```

- workflow-controller logs:

```
time="2020-07-31T18:48:27Z" level=info msg="Processing workflow" namespace=default workflow=nested-test-v7v4d
time="2020-07-31T18:48:27Z" level=info msg="Updated phase  -> Running" namespace=default workflow=nested-test-v7v4d
time="2020-07-31T18:48:27Z" level=info msg="Steps node nested-test-v7v4d initialized Running" namespace=default workflow=nested-test-v7v4d
time="2020-07-31T18:48:27Z" level=info msg="StepGroup node nested-test-v7v4d-1392141231 initialized Running" namespace=default workflow=nested-test-v7v4d
time="2020-07-31T18:48:27Z" level=info msg="Pod node nested-test-v7v4d-2387945936 initialized Pending" namespace=default workflow=nested-test-v7v4d
time="2020-07-31T18:48:27Z" level=info msg="Created pod: nested-test-v7v4d[0].echoitems(0:nested:map[B:2 \"foo\"]) (nested-test-v7v4d-2387945936)" namespace=default workflow=nested-test-v7v4d
time="2020-07-31T18:48:27Z" level=info msg="Workflow step group node nested-test-v7v4d-1392141231 not yet completed" namespace=default workflow=nested-test-v7v4d
time="2020-07-31T18:48:27Z" level=info msg="Workflow update successful" namespace=default phase=Running resourceVersion=2562 workflow=nested-test-v7v4d
time="2020-07-31T18:48:27Z" level=info msg="Processing workflow" namespace=default workflow=nested-test-v7v4d
time="2020-07-31T18:48:27Z" level=info msg="Updating node nested-test-v7v4d-2387945936 message: ContainerCreating"
time="2020-07-31T18:48:27Z" level=info msg="Skipped pod nested-test-v7v4d[0].echoitems(0:nested:map[B:2 \"foo\"]) (nested-test-v7v4d-2387945936) creation: already exists" namespace=default workflow=nested-test-v7v4d
time="2020-07-31T18:48:27Z" level=info msg="Workflow step group node nested-test-v7v4d-1392141231 not yet completed" namespace=default workflow=nested-test-v7v4d
time="2020-07-31T18:48:27Z" level=info msg="Workflow update successful" namespace=default phase=Running resourceVersion=2564 workflow=nested-test-v7v4d
time="2020-07-31T18:48:27Z" level=info msg="Processing workflow" namespace=default workflow=nested-test-v7v4d
time="2020-07-31T18:48:27Z" level=info msg="Skipped pod nested-test-v7v4d[0].echoitems(0:nested:map[B:2 \"foo\"]) (nested-test-v7v4d-2387945936) creation: already exists" namespace=default workflow=nested-test-v7v4d
time="2020-07-31T18:48:27Z" level=info msg="Workflow step group node nested-test-v7v4d-1392141231 not yet completed" namespace=default workflow=nested-test-v7v4d
time="2020-07-31T18:48:28Z" level=info msg="Processing workflow" namespace=default workflow=nested-test-v7v4d
time="2020-07-31T18:48:28Z" level=info msg="Skipped pod nested-test-v7v4d[0].echoitems(0:nested:map[B:2 \"foo\"]) (nested-test-v7v4d-2387945936) creation: already exists" namespace=default workflow=nested-test-v7v4d
time="2020-07-31T18:48:28Z" level=info msg="Workflow step group node nested-test-v7v4d-1392141231 not yet completed" namespace=default workflow=nested-test-v7v4d
time="2020-07-31T18:48:49Z" level=info msg="Processing workflow" namespace=default workflow=nested-test-v7v4d
time="2020-07-31T18:48:49Z" level=info msg="Updating node nested-test-v7v4d-2387945936 status Pending -> Running"
time="2020-07-31T18:48:49Z" level=info msg="Workflow step group node nested-test-v7v4d-1392141231 not yet completed" namespace=default workflow=nested-test-v7v4d
time="2020-07-31T18:48:49Z" level=info msg="Workflow update successful" namespace=default phase=Running resourceVersion=2639 workflow=nested-test-v7v4d
time="2020-07-31T18:48:49Z" level=info msg="Processing workflow" namespace=default workflow=nested-test-v7v4d
time="2020-07-31T18:48:49Z" level=info msg="insignificant pod change" key=default/nested-test-v7v4d-2387945936
time="2020-07-31T18:48:49Z" level=info msg="Setting node nested-test-v7v4d-2387945936 outputs"
time="2020-07-31T18:48:49Z" level=info msg="Workflow step group node nested-test-v7v4d-1392141231 not yet completed" namespace=default workflow=nested-test-v7v4d
time="2020-07-31T18:48:49Z" level=info msg="Workflow update successful" namespace=default phase=Running resourceVersion=2641 workflow=nested-test-v7v4d
time="2020-07-31T18:48:49Z" level=info msg="Processing workflow" namespace=default workflow=nested-test-v7v4d
time="2020-07-31T18:48:49Z" level=info msg="Workflow step group node nested-test-v7v4d-1392141231 not yet completed" namespace=default workflow=nested-test-v7v4d
time="2020-07-31T18:48:51Z" level=info msg="Processing workflow" namespace=default workflow=nested-test-v7v4d
time="2020-07-31T18:48:51Z" level=info msg="Updating node nested-test-v7v4d-2387945936 status Running -> Succeeded"
time="2020-07-31T18:48:51Z" level=info msg="Step group node &NodeStatus{ID:nested-test-v7v4d-1392141231,Name:nested-test-v7v4d[0],DisplayName:[0],Type:StepGroup,TemplateName:main,TemplateRef:nil,Phase:Running,BoundaryID:nested-test-v7v4d,Message:,StartedAt:2020-07-31 18:48:27 +0000 UTC,FinishedAt:0001-01-01 00:00:00 +0000 UTC,PodIP:,Daemoned:nil,Inputs:nil,Outputs:nil,Children:[nested-test-v7v4d-2387945936],OutboundNodes:[],StoredTemplateID:,WorkflowTemplateName:,TemplateScope:local/nested-test-v7v4d,ResourcesDuration:ResourcesDuration{},HostNodeName:,} successful" namespace=default workflow=nested-test-v7v4d
time="2020-07-31T18:48:51Z" level=info msg="node nested-test-v7v4d-1392141231 phase Running -> Succeeded" namespace=default workflow=nested-test-v7v4d
time="2020-07-31T18:48:51Z" level=info msg="node nested-test-v7v4d-1392141231 finished: 2020-07-31 18:48:51.830208255 +0000 UTC" namespace=default workflow=nested-test-v7v4d
time="2020-07-31T18:48:51Z" level=info msg="Outbound nodes of nested-test-v7v4d-2387945936 is [nested-test-v7v4d-2387945936]" namespace=default workflow=nested-test-v7v4d
time="2020-07-31T18:48:51Z" level=info msg="Outbound nodes of nested-test-v7v4d is [nested-test-v7v4d-2387945936]" namespace=default workflow=nested-test-v7v4d
time="2020-07-31T18:48:51Z" level=info msg="node nested-test-v7v4d phase Running -> Succeeded" namespace=default workflow=nested-test-v7v4d
time="2020-07-31T18:48:51Z" level=info msg="node nested-test-v7v4d finished: 2020-07-31 18:48:51.830410974 +0000 UTC" namespace=default workflow=nested-test-v7v4d
time="2020-07-31T18:48:51Z" level=info msg="Checking daemoned children of nested-test-v7v4d" namespace=default workflow=nested-test-v7v4d
time="2020-07-31T18:48:51Z" level=info msg="Updated phase Running -> Succeeded" namespace=default workflow=nested-test-v7v4d
time="2020-07-31T18:48:51Z" level=info msg="Marking workflow completed" namespace=default workflow=nested-test-v7v4d
time="2020-07-31T18:48:51Z" level=info msg="Checking daemoned children of " namespace=default workflow=nested-test-v7v4d
time="2020-07-31T18:48:51Z" level=info msg="Workflow update successful" namespace=default phase=Succeeded resourceVersion=2650 workflow=nested-test-v7v4d
time="2020-07-31T18:48:51Z" level=info msg="nested-test-v7v4d released all acquired locks"
time="2020-07-31T18:48:51Z" level=info msg="Labeled pod default/nested-test-v7v4d-2387945936 completed"
time="2020-07-31T18:48:51Z" level=info msg="Workflow update successful" namespace=default phase=Succeeded resourceVersion=2650 workflow=nested-test-v7v4d
time="2020-07-31T18:48:51Z" level=info msg="Labeled pod default/nested-test-v7v4d-2387945936 completed"
```

---
<!-- Issue Author: Don't delete this message to encourage other users to support your issue! -->
**Message from the maintainers**:

If you are impacted by this bug please add a 👍 reaction to this issue! We often sort issues this way to know what to prioritize.


Pull request:
fix(controller): Fix nested maps. Fixes #3653
Checklist:

* [x] Either (a) I've created an [enhancement proposal](https://github.com/argoproj/argo/issues/new/choose) and discussed it with the community, (b) this is a bug fix, or (c) this is a chore.
* [x] The title of the PR is (a) [conventional](https://www.conventionalcommits.org/en/v1.0.0/), (b) states what changed, and (c) suffixes the related issues number. E.g. `"fix(controller): Updates such and such. Fixes #1234"`.  
* [x] I've signed the CLA.
* [x] I have written unit and/or e2e tests for my change. PRs without these are unlikely to be merged.
* [x] My builds are green. Try syncing with master if they are not. 
* [x] My organization is added to [USERS.md](https://github.com/argoproj/argo/blob/master/USERS.md).

Fixes #3653

Patch:
diff --git a/pkg/apis/workflow/v1alpha1/item.go b/pkg/apis/workflow/v1alpha1/item.go
index 1a357abdb..9663e41c9 100644
--- a/pkg/apis/workflow/v1alpha1/item.go
+++ b/pkg/apis/workflow/v1alpha1/item.go
@@ -63,6 +63,7 @@ func (i *Item) String() string {
 	if err != nil {
 		panic(err)
 	}
+	// this convenience to remove quotes from strings will cause many problems
 	if jsonBytes[0] == '"' {
 		return string(jsonBytes[1 : len(jsonBytes)-1])
 	}
@@ -95,15 +96,15 @@ func (i Item) OpenAPISchemaType() []string {
 func (i Item) OpenAPISchemaFormat() string { return "" }
 
 // you MUST assert `GetType() == Map` before invocation as this does not return errors
-func (i *Item) GetMapVal() map[string]interface{} {
-	val := make(map[string]interface{})
+func (i *Item) GetMapVal() map[string]Item {
+	val := make(map[string]Item)
 	_ = json.Unmarshal(i.Value, &val)
 	return val
 }
 
 // you MUST assert `GetType() == List` before invocation as this does not return errors
-func (i *Item) GetListVal() []interface{} {
-	val := make([]interface{}, 0)
+func (i *Item) GetListVal() []Item {
+	val := make([]Item, 0)
 	_ = json.Unmarshal(i.Value, &val)
 	return val
 }


Test patch:
diff --git a/pkg/apis/workflow/v1alpha1/item_test.go b/pkg/apis/workflow/v1alpha1/item_test.go
index b94e0a37d..9e511a519 100644
--- a/pkg/apis/workflow/v1alpha1/item_test.go
+++ b/pkg/apis/workflow/v1alpha1/item_test.go
@@ -18,7 +18,7 @@ func TestItem(t *testing.T) {
 		"{\"val\":\"123\"}":               Map,
 		"[\"1\",\"2\",\"3\",\"4\",\"5\"]": List,
 	} {
-		t.Run(string(expectedType), func(t *testing.T) {
+		t.Run(fmt.Sprintf("%v", expectedType), func(t *testing.T) {
 			t.Run("Item", func(t *testing.T) {
 				runItemTest(t, data, expectedType)
 			})
@@ -47,7 +47,7 @@ func TestItem_GetMapVal(t *testing.T) {
 	err := json.Unmarshal([]byte(`{"foo":"bar"}`), &item)
 	assert.NoError(t, err)
 	val := item.GetMapVal()
-	assert.Len(t, val, 1)
+	assert.Equal(t, map[string]Item{"foo": {Value: []byte(`"bar"`)}}, val)
 }
 
 func TestItem_GetListVal(t *testing.T) {
@@ -55,7 +55,7 @@ func TestItem_GetListVal(t *testing.T) {
 	err := json.Unmarshal([]byte(`["foo"]`), &item)
 	assert.NoError(t, err)
 	val := item.GetListVal()
-	assert.Len(t, val, 1)
+	assert.Equal(t, []Item{{Value: []byte(`"foo"`)}}, val)
 }
 
 func TestItem_GetStrVal(t *testing.T) {
diff --git a/workflow/controller/operator_test.go b/workflow/controller/operator_test.go
index 6d166478a..2a0e85bdd 100644
--- a/workflow/controller/operator_test.go
+++ b/workflow/controller/operator_test.go
@@ -4582,7 +4582,7 @@ func TestFailSuspendedAndPendingNodesAfterShutdown(t *testing.T) {
 	})
 }
 
-func TestProcessItem(t *testing.T) {
+func Test_processItem(t *testing.T) {
 	task := wfv1.DAGTask{
 		WithParam: `[{"number": 2, "string": "foo", "list": [0, "1"], "json": {"number": 2, "string": "foo", "list": [0, "1"]}}]`,
 	}
@@ -4598,6 +4598,6 @@ func TestProcessItem(t *testing.T) {
 	var newTask wfv1.DAGTask
 	newTaskName, err := processItem(fstTmpl, "task-name", 0, items[0], &newTask)
 	if assert.NoError(t, err) {
-		assert.Equal(t, `task-name(0:json:map[list:[0 1] number:2 string:foo],list:[0 1],number:2,string:foo)`, newTaskName)
+		assert.Equal(t, `task-name(0:json:{"number":2,"string":"foo","list":[0,"1"]},list:[0,"1"],number:2,string:foo)`, newTaskName)
 	}
 }