Repository: ansible/ansible

Issue:
unreachable if ansible_password is encrypted by 'per variable' vault
##### ISSUE TYPE
 - Bug Report

##### COMPONENT NAME
'per variable' vault ( https://github.com/ansible/ansible/pull/16274 )

##### ANSIBLE VERSION
both 2.2.0 and devel

```
ansible 2.2.0.0
  config file = /etc/ansible/ansible.cfg
  configured module search path = Default w/o overrides
```

```
ansible 2.3.0 (devel 425ab5598e) last updated 2017/01/03 13:01:56 (GMT +900)
  config file =
  configured module search path = Default w/o overrides
```

##### CONFIGURATION
nothing special

##### OS / ENVIRONMENT
N/A

##### SUMMARY
The ssh connection becomes unreachable by authentication failure if the value of ansible_password is encrypted.

##### STEPS TO REPRODUCE

* create a vault password file with any password (filename: vault_pass)

* create an inventory file (filename: hosts)
```
$ cat hosts
[test]
test1 ansible_host=172.21.1.184 ansible_user=vagrant
```

* set a value encrypted by vault to ansible_password in host_vars/test1.yml
```
$ echo 'ansible_password: !vault-encrypted |' > host_vars/test1.yml; echo -n 'vagrant' | ansible-vault encrypt --vault-password-file vault_pass | xargs -I @ printf "%2s%s\n" '' @ >> host_vars/test1.yml
$ cat host_vars/test1.yml
ansible_password: !vault-encrypted |
  $ANSIBLE_VAULT;1.1;AES256
  64383761663636303762306234383966333935346165633464646535376563653064353161363164
  6232346236353135376635373236333134363065353031390a613733333133653663643863303038
  64303734346238306633336161303439366335323338383964303833323564336532666564323532
  6131663132626539620a326331666530666334393436383134326265653736663866343534333366
  3562
```

* launch `ansible test1 -i hosts --vault-password-file vault_pass -m ping -vvvv`

##### EXPECTED RESULTS
success

##### ACTUAL RESULTS

```
$ ansible test1 -i hosts --vault-password-file vault_pass -m ping -vvvv
No config file found; using defaults
Loading callback plugin minimal of type stdout, v2.0 from /home/vagrant/ansible/lib/ansible/plugins/callback/__init__.pyc
Using module file /home/vagrant/ansible/lib/ansible/modules/system/ping.py
<172.21.1.184> ESTABLISH SSH CONNECTION FOR USER: vagrant
<172.21.1.184> SSH: EXEC sshpass -d12 ssh -vvv -C -o ControlMaster=auto -o ControlPersist=60s -o User=vagrant -o ConnectTimeout=10 -o ControlPath=/home/vagrant/.ansible/cp/ansible-ssh-%h-%p-%r 172.21.1.184 '/bin/sh -c '"'"'( umask 77 && mkdir -p "` echo ~/.ansible/tmp/ansible-tmp-1483429987.59-257858382794001 `" && echo ansible-tmp-1483429987.59-257858382794001="` echo ~/.ansible/tmp/ansible-tmp-1483429987.59-257858382794001 `" ) && sleep 0'"'"''
test1 | UNREACHABLE! => {
    "changed": false,
    "msg": "Authentication failure.",
    "unreachable": true
}
```

Additional information:

* I can retrieve the value of ansible_password with the debug module.
```
$ ansible test1 -i hosts --vault-password-file vault_pass -m debug -a var=ansible_password
test1 | SUCCESS => {
    "ansible_password": "vagrant"
}
```

* This issue also affects ansible_become_pass.



Discussion:
I can reproduce this on devel/ branch (a601f2c6a341f7b4026fea5f60ddc67f3e6d02dc).
Looks like the ssh connection plugin does a to_bytes() on ansible_password which is an AnsibleEncryptedUnicode at that point, and the to_bytes just returns a obj.encode('utf-8', 'replace').
On py2, that is already a str, so AnsibleEncryptedUnicode never gets its __str__() method called
(and the __str__() is what actually returns the decrypted value). The connection plugin gets the full
yaml string '$ANSIBLE_VAULT...'. 

I suspect AnsibleEncryptedUnicode needs a encode() that decrypts as well.

Resolving patch:
diff --git a/lib/ansible/parsing/yaml/objects.py b/lib/ansible/parsing/yaml/objects.py
--- a/lib/ansible/parsing/yaml/objects.py
+++ b/lib/ansible/parsing/yaml/objects.py
@@ -132,3 +132,6 @@ def __str__(self):
 
     def __unicode__(self):
         return unicode(self.data)
+
+    def encode(self, encoding=None, errors=None):
+        return self.data.encode(encoding, errors)


Test patch:
diff --git a/test/units/parsing/yaml/test_objects.py b/test/units/parsing/yaml/test_objects.py
--- a/test/units/parsing/yaml/test_objects.py
+++ b/test/units/parsing/yaml/test_objects.py
@@ -121,6 +121,15 @@ def test_unicode_from_plaintext(self):
         avu = self._from_plaintext(seq)
         self.assert_values(avu,seq)
 
+    def test_unicode_from_plaintext_encode(self):
+        seq = u'some text here'
+        avu = self._from_plaintext(seq)
+        b_avu = avu.encode('utf-8', 'strict')
+        self.assertIsInstance(avu, objects.AnsibleVaultEncryptedUnicode)
+        self.assertEquals(b_avu, seq.encode('utf-8', 'strict'))
+        self.assertTrue(avu.vault is self.vault)
+        self.assertIsInstance(avu.vault, vault.VaultLib)
+
     # TODO/FIXME: make sure bad password fails differently than 'thats not encrypted'
     def test_empty_string_wrong_password(self):
         seq = ''