Repository: ansible/ansible

Issue:
Ansible Vault: Unable to use vaulted variables in loops (with_items, with_dict)
<!---
Verify first that your issue/request is not already reported on GitHub.
Also test if the latest release, and master branch are affected too.
-->

##### ISSUE TYPE
 - Bug Report

##### COMPONENT NAME
ansible-vault or jinja

##### ANSIBLE VERSION
```
ansible 2.3.0.0
  config file = /etc/ansible/ansible.cfg
  configured module search path = Default w/o overrides
  python version = 2.7.12 (default, Nov 19 2016, 06:48:10) [GCC 5.4.0 20160609]
```

##### CONFIGURATION
No special configuration

##### OS / ENVIRONMENT
Ubuntu 16.04

##### SUMMARY
When using the newly introduced [single encrypted variables ](http://docs.ansible.com/ansible/playbooks_vault.html#single-encrypted-variable) in lists or dictionaries these cannot be looped like expected with `with_items` or `with_dict`. The list or dictionary is somehow treated as one object and not as a "loopable" object.

##### STEPS TO REPRODUCE
<!---
For bugs, show exactly how to reproduce the problem, using a minimal test-case.
For new features, show how the feature would be used.
-->
Simple playbook containing a single encrypted variable `vaulted` in the list `test_with_vaulted_variable`. The vault password is "test" (without quotes) and the `vaulted` variable content is "vaulted variable".

<!--- Paste example playbooks or commands between quotes below -->
```yaml
---
- hosts: all
  vars:
      test_without_vaulted_variable:
          - not_vaulted: not vaulted variable
          - another_standard_variable: standard
      test_with_vaulted_variable:
          - not_vaulted: not vaulted variable
          - vaulted: !vault |
              $ANSIBLE_VAULT;1.1;AES256
              66376230363937306331353166333731633037326166626530393462636666346630366463313134
              6635313236366537346339313338633539643665313931390a373264326437663530616630623734
              31666136343232666235323865653838393830613432343561633465333837633531643564343064
              3237353766313835310a643963313163663632623064313034363531356330653131303833646138
              65366139376134396231353864383662623832376239336433623630383464303161
  tasks:
    - debug: var=test_without_vaulted_variable
    - debug: var=test_with_vaulted_variable
    - debug: 
      with_items: "{{ test_without_vaulted_variable }}"
    - debug: 
      with_items: "{{ test_with_vaulted_variable }}"
```
Start the play with (and enter the vault pass "test")
```
ansible-playbook -i 'lavego-test,' vault-with-items.yml --ask-vault-pass
```
<!--- You can also paste gist.github.com links for larger files -->

##### EXPECTED RESULTS
The list processing should be the same for the lists `test_without_vaulted_variable`and `test_with_vaulted_variable`: The loops should output each element of each list.

##### ACTUAL RESULTS
The list containing the vaulted variable is not looped like expected. In the last debug statement: Note that there is only one "Hello world!" message printed instead of two, one for each element, for the list with no vaulted variable.

Note that also the normal debug output of the two lists differs (first and second debug statements) - the list containing the vaulted variable "has no structure".

<!--- Paste verbatim command output between quotes below -->
```
$ ansible-playbook -i 'localhost,' vault-with-items.yml --ask-vault-pass
Vault password: 

PLAY [all] *********************************************************************************************************************************************************************************************************

TASK [Gathering Facts] *********************************************************************************************************************************************************************************************
ok: [localhost]

TASK [debug] *******************************************************************************************************************************************************************************************************
ok: [localhost] => {
    "changed": false, 
    "test_without_vaulted_variable": [
        {
            "not_vaulted": "not vaulted variable"
        }, 
        {
            "another_standard_variable": "standard"
        }
    ]
}

TASK [debug] *******************************************************************************************************************************************************************************************************
ok: [localhost] => {
    "changed": false, 
    "test_with_vaulted_variable": "[{u'not_vaulted': u'not vaulted variable'}, {u'vaulted': AnsibleVaultEncryptedUnicode($ANSIBLE_VAULT;1.1;AES256\n66376230363937306331353166333731633037326166626530393462636666346630366463313134\n6635313236366537346339313338633539643665313931390a373264326437663530616630623734\n31666136343232666235323865653838393830613432343561633465333837633531643564343064\n3237353766313835310a643963313163663632623064313034363531356330653131303833646138\n65366139376134396231353864383662623832376239336433623630383464303161\n)}]"
}

TASK [debug] *******************************************************************************************************************************************************************************************************
ok: [localhost] => (item={u'not_vaulted': u'not vaulted variable'}) => {
    "item": {
        "not_vaulted": "not vaulted variable"
    }, 
    "msg": "Hello world!"
}
ok: [localhost] => (item={u'another_standard_variable': u'standard'}) => {
    "item": {
        "another_standard_variable": "standard"
    }, 
    "msg": "Hello world!"
}

TASK [debug] *******************************************************************************************************************************************************************************************************
ok: [localhost => (item=[{u'not_vaulted': u'not vaulted variable'}, {u'vaulted': AnsibleVaultEncryptedUnicode($ANSIBLE_VAULT;1.1;AES256
66376230363937306331353166333731633037326166626530393462636666346630366463313134
6635313236366537346339313338633539643665313931390a373264326437663530616630623734
31666136343232666235323865653838393830613432343561633465333837633531643564343064
3237353766313835310a643963313163663632623064313034363531356330653131303833646138
65366139376134396231353864383662623832376239336433623630383464303161
)}]) => {
    "item": "[{u'not_vaulted': u'not vaulted variable'}, {u'vaulted': AnsibleVaultEncryptedUnicode($ANSIBLE_VAULT;1.1;AES256\n66376230363937306331353166333731633037326166626530393462636666346630366463313134\n6635313236366537346339313338633539643665313931390a373264326437663530616630623734\n31666136343232666235323865653838393830613432343561633465333837633531643564343064\n3237353766313835310a643963313163663632623064313034363531356330653131303833646138\n65366139376134396231353864383662623832376239336433623630383464303161\n)}]", 
    "msg": "Hello world!"
}

PLAY RECAP *********************************************************************************************************************************************************************************************************
localhost                : ok=5    changed=0    unreachable=0    failed=0   
```

Thank you for your help!


Discussion:
I can confirm this issue.
/cc @alikins 
The output for the 
``` yaml
- debug: 
      with_items: "{{ test_with_vaulted_variable }}"
``` 
task seems to be getting the python repr of the object instead of the str. 

```
    "changed": false, 
    "test_with_vaulted_variable": "[{u'not_vaulted': u'not vaulted variable'}, {u'vaulted': AnsibleVaultEncryptedUnicode($ANSIBLE_VAULT;1.1;AES256\n66376230363937306331353166333731633037326166626530393462636666346630366463313134\n6635313236366537346339313338633539643665313931390a373264326437663530616630623734\n31666136343232666235323865653838393830613432343561633465333837633531643564343064\n3237353766313835310a643963313163663632623064313034363531356330653131303833646138\n65366139376134396231353864383662623832376239336433623630383464303161\n)}]"
}
```

Not sure why yet, investigating...

Resolving patch:
diff --git a/lib/ansible/parsing/yaml/objects.py b/lib/ansible/parsing/yaml/objects.py
--- a/lib/ansible/parsing/yaml/objects.py
+++ b/lib/ansible/parsing/yaml/objects.py
@@ -111,7 +111,7 @@ def data(self, value):
         self._ciphertext = value
 
     def __repr__(self):
-        return 'AnsibleVaultEncryptedUnicode(%s)' % self._ciphertext
+        return repr(self.data)
 
     # Compare a regular str/text_type with the decrypted hypertext
     def __eq__(self, other):


Test patch: