Repository: ansible/ansible

Issue:
Template loop assignment failing and substituting value of tags property
##### ISSUE TYPE

Bug Report

##### COMPONENT NAME

Templating `with_items` / `with_subelements`

##### ANSIBLE VERSION

```
ansible 2.4.1.0
  config file = /etc/ansible/ansible.cfg
  configured module search path = [u'/home/dspruell/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/local/lib/python2.7/site-packages/ansible
  executable location = /usr/local/bin/ansible
  python version = 2.7.14 (default, Oct  2 2017, 09:52:26) [GCC 4.2.1 Compatible OpenBSD Clang 4.0.0 (tags/RELEASE_400/final)]
```

Jinja2 (2.10)

Tested also using latest Ansible devel branch (ansible-playbook 2.5.0)

##### CONFIGURATION

```
$ ansible-config dump --only-changed                                                          
DEFAULT_BECOME(/etc/ansible/ansible.cfg) = True                                                                      
DEFAULT_MANAGED_STR(/etc/ansible/ansible.cfg) = !!! Ansible managed - changes will be overridden
```

##### OS / ENVIRONMENT

- Managing host: OpenBSD amd64 6.2
- Managed host: OpenBSD amd64 6.2

##### SUMMARY

Attempting to template strings out using a nested data structure and the `with_items` or `with_subelements` options raises confusing error referencing value of `tags` option.

##### STEPS TO REPRODUCE

Invoke the following sample playbook using `ansible-playbook`.

```yaml
- hosts: localhost
  become: no
  vars:
    openvpn_client_configs:
    - cn: Charlie Root
      options:
      - ifconfig-push: 10.0.2.3
    - cn: Testy Mctestface
      options:
      - ifconfig-push: 10.0.2.4
      - test-val: foo
  tasks:
  - name: create openvpn client config directory files
    local_action:
      module: debug
      msg: '{% for k, v in item.1 %}{{ k }} {{ v }}{% endfor %}'
    with_subelements:
    - '{{ openvpn_client_configs }}'
    - options
    tags: [openvpn_test]
```

##### EXPECTED RESULTS

Expected that each dict under the options key would render out in space separated form by the debug module.

##### ACTUAL RESULTS

The task module raises an error that the "args" value has an invalid value and the reported value is the value of the tags field on the the task. The template doesn't take effect and then debug message is not rendered.

```
$ ansible-playbook -vvvv test.yml 2>&1 | tee test.txt
ansible-playbook 2.4.1.0
  config file = None
  configured module search path = [u'/home/dspruell/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/local/lib/python2.7/site-packages/ansible
  executable location = /usr/local/bin/ansible-playbook
  python version = 2.7.14 (default, Oct  2 2017, 09:52:26) [GCC 4.2.1 Compatible OpenBSD Clang 4.0.0 (tags/RELEASE_400/final)]
No config file found; using defaults
setting up inventory plugins
 [WARNING]: Unable to parse /etc/ansible/hosts as an inventory source
 [WARNING]: No inventory was parsed, only implicit localhost is available
 [WARNING]: Could not match supplied host pattern, ignoring: all
 [WARNING]: provided hosts list is empty, only localhost is available
Loading callback plugin default of type stdout, v2.0 from /usr/local/lib/python2.7/site-packages/ansible/plugins/callback/__init__.pyc

PLAYBOOK: test.yml **************************************************************************************************
1 plays in test.yml

PLAY [localhost] ****************************************************************************************************

TASK [Gathering Facts] **********************************************************************************************
Using module file /usr/local/lib/python2.7/site-packages/ansible/modules/system/setup.py
<127.0.0.1> ESTABLISH LOCAL CONNECTION FOR USER: dspruell
<127.0.0.1> EXEC /bin/sh -c 'echo ~ && sleep 0'
<127.0.0.1> EXEC /bin/sh -c '( umask 77 && mkdir -p "` echo /home/dspruell/.ansible/tmp/ansible-tmp-1511596979.27-70166965739965 `" && echo ansible-tmp-1511596979.27-70166965739965="` echo /home/dspruell/.ansible/tmp/ansible-tmp-1511596979.27-70166965739965 `" ) && sleep 0'
<127.0.0.1> PUT /tmp/tmpz3ta4U TO /home/dspruell/.ansible/tmp/ansible-tmp-1511596979.27-70166965739965/setup.py
<127.0.0.1> EXEC /bin/sh -c 'chmod u+x /home/dspruell/.ansible/tmp/ansible-tmp-1511596979.27-70166965739965/ /home/dspruell/.ansible/tmp/ansible-tmp-1511596979.27-70166965739965/setup.py && sleep 0'
<127.0.0.1> EXEC /bin/sh -c '/usr/local/bin/python2.7 /home/dspruell/.ansible/tmp/ansible-tmp-1511596979.27-70166965739965/setup.py; rm -rf "/home/dspruell/.ansible/tmp/ansible-tmp-1511596979.27-70166965739965/" > /dev/null 2>&1 && sleep 0'
ok: [localhost]
META: ran handlers

TASK [create openvpn client config directory files] *****************************************************************
task path: /home/dspruell/test.yml:13
fatal: [localhost]: FAILED! => {
    "msg": "the field 'args' has an invalid value ([u'openvpn_test']), and could not be converted to an dict.The error was: too many values to unpack\n\nThe error appears to have been in '/home/dspruell/test.yml': line 13, column 5, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n  tasks:\n  - name: create openvpn client config directory files\n    ^ here\n\nexception type: <type 'exceptions.ValueError'>\nexception: too many values to unpack"
}
        to retry, use: --limit @/home/dspruell/test.retry

PLAY RECAP **********************************************************************************************************
localhost                  : ok=1    changed=0    unreachable=0    failed=1
```

I also tried a variation of this using the same variable data element, and using the `with_items` parameter, and it behaved similarly (same error). 

If I remove the `tags` option from the task, the error message is then:

```
    "msg": "the field 'args' has an invalid value ([]), and could not be converted to an dict.The error was: too many values to unpack\n\nThe error appears to have been in '/home/dspruell/test.yml': line 13, column 5, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n  tasks:\n  - name: create openvpn client config directory files\n    ^ here\n"
```


Discussion:
Files identified in the description:
* [lib/ansible/playbook/loop_control.py](https://github.com/ansible/ansible/blob/devel/lib/ansible/playbook/loop_control.py)

If these files are inaccurate, please update the `component name` section of the description or use the `!component` bot command.

[click here for bot help](https://github.com/ansible/ansibullbot/blob/master/ISSUE_HELP.md)
<!--- boilerplate: components_banner --->
I believe the error is actually with your for loop:

```
for k, v in item.1 
```

The error indicates that you asked jinja/python to unpack more elements than are available. This is because `item.1` is a dict and by default looping a dict only loops the keys.

Instead you want something like:

```
for k, v in item.1|dictsort
```

Or

```
for k, v in item.1.items()
```
Confirmed that I botched the loop, testing with `item.1.items()` works as desired. Thanks!

Aside from the mistake on the loop - is there any way where the invalid value identified in the message (the value of the `tags` field) can be improved? Not clear why this value was in focus or where the referenced 'args' field came from.

    "msg": "the field 'args' has an invalid value ([u'openvpn_test']), and could not be converted to an dict.The error was: too many values to unpack\n\nThe error appears to have been in '/home/dspruell/test.yml': line 13, column 5, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n  tasks:\n  - name: create openvpn client config directory files\n    ^ here\n\nexception type: <type 'exceptions.ValueError'>\nexception: too many values to unpack"
`args` refers to any arguments passed to a module. I can't comment immediately on why the value of tags was referenced in the error.

Resolving patch:
diff --git a/lib/ansible/playbook/base.py b/lib/ansible/playbook/base.py
--- a/lib/ansible/playbook/base.py
+++ b/lib/ansible/playbook/base.py
@@ -427,6 +427,7 @@ def post_validate(self, templar):
                 # and assign the massaged value back to the attribute field
                 setattr(self, name, value)
             except (TypeError, ValueError) as e:
+                value = getattr(self, name)
                 raise AnsibleParserError("the field '%s' has an invalid value (%s), and could not be converted to an %s."
                                          "The error was: %s" % (name, value, attribute.isa, e), obj=self.get_ds(), orig_exc=e)
             except (AnsibleUndefinedVariable, UndefinedError) as e:


Test patch: