Repository: ansible/ansible

Issue:
'ansible-vault edit' on files that are non-ascii fails with UnicodeDecodeError
<!---
Please do not report issues/requests related to Ansible modules here !!

Report them to the appropriate modules-core or modules-extras project:
  - https://github.com/ansible/ansible-modules-core/issues
  - https://github.com/ansible/ansible-modules-extras/issues

Also verify first that your issue/request is not already reported in GitHub
-->


##### ISSUE TYPE
<!--- Pick one below and delete the rest: -->
 - Bug Report
 

##### COMPONENT NAME
<!--- Name of the plugin/task/feature -->
lib/ansible/parsing/vault

##### ANSIBLE VERSION
<!--- Paste verbatim output from “ansible --version” between quotes below -->
```
ansible 2.3.0 (devel e52ce12ba4) last updated 2016/11/08 12:41:48 (GMT -400)
  lib/ansible/modules/core: (detached HEAD 2584fca0ae) last updated 2016/11/08 11:39:04 (GMT -400)
  lib/ansible/modules/extras: (detached HEAD a1dcbf9ce5) last updated 2016/11/08 11:39:04 (GMT -400)
  config file = /home/adrian/.ansible.cfg
  configured module search path = Default w/o overrides

```

##### CONFIGURATION
<!---
Mention any settings you have changed/added/removed in ansible.cfg
(or using the ANSIBLE_* environment variables).
-->

##### OS / ENVIRONMENT
<!---
Mention the OS you are running Ansible from, and the OS you are
managing, or say “N/A” for anything that is not platform-specific.
-->
fedora 23, x86_64

##### SUMMARY
<!--- Explain the problem briefly -->
'ansible-vault edit' on files that are non-ascii fails with UnicodeDecodeError. 

This affects devel (2.3) and stable-2.2 (2.2). pr pending

##### STEPS TO REPRODUCE
<!---
For bugs, show exactly how to reproduce the problem.
For new features, show how the feature would be used.
-->

<!--- Paste example playbooks or commands between quotes below -->
```
[fedora-23:ansible (devel % u=)]$ cat example_fail.yml 

---
- name: 'A play with unicode: ¢ £ ¤ ¥'
  hosts: localhost
  connection: local
  vars:
    test_var: 'Ī ī Ĭ ĭ Į į İ ı Ĳ ĳ Ĵ ĵ Ķ ķ ĸ Ĺ ĺ Ļ ļ Ľ ľ Ŀ ŀ Ł ł Ń ń Ņ ņ Ň ň ŉ Ŋ ŋ Ō ō Ŏ ŏ Ő ő Œ'
    hostnames:
        - 'host-ϬϭϮϯϰ'
        - 'host-fóöbär'
        - 'host-ΙΚΛΜΝΞ'
        - 'host-στυφχψ'
        - 'host-ϬϭϮϯϰϱ'

[fedora-23:ansible (devel % u=)]$ ansible-vault encrypt example_fail.yml 
Vault password: 
Encryption successful
[fedora-23:ansible (devel % u=)]$ ansible-vault -vvv edit example_fail.yml 
Using /home/adrian/.ansible.cfg as config file
Vault password: 

#  NOTE: when in the interactive editor, edit the file in some way so that it is different so it will be reencrypted

ERROR! Unexpected Exception: 'ascii' codec can't decode byte 0xc2 in position 34: ordinal not in range(128)
the full traceback was:

Traceback (most recent call last):
  File "/home/adrian/src/ansible/bin/ansible-vault", line 103, in <module>
    exit_code = cli.run()
  File "/home/adrian/src/ansible/lib/ansible/cli/vault.py", line 123, in run
    self.execute()
  File "/home/adrian/src/ansible/lib/ansible/cli/__init__.py", line 147, in execute
    fn()
  File "/home/adrian/src/ansible/lib/ansible/cli/vault.py", line 159, in execute_edit
    self.editor.edit_file(f)
  File "/home/adrian/src/ansible/lib/ansible/parsing/vault/__init__.py", line 464, in edit_file
    self._edit_file_helper(filename, existing_data=plaintext, force_save=False)
  File "/home/adrian/src/ansible/lib/ansible/parsing/vault/__init__.py", line 411, in _edit_file_helper
    enc_data = self.vault.encrypt(tmpdata.decode())
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc2 in position 34: ordinal not in range(128)

```

<!--- You can also paste gist.github.com links for larger files -->

##### EXPECTED RESULTS
<!--- What did you expect to happen when running the steps above? -->
No errors. New file contents encrypted.

##### ACTUAL RESULTS
<!--- What actually happened? If possible run with extra verbosity (-vvvv) -->
UnicodeDecodeError is raised.



Discussion:

Resolving patch:
diff --git a/lib/ansible/parsing/vault/__init__.py b/lib/ansible/parsing/vault/__init__.py
--- a/lib/ansible/parsing/vault/__init__.py
+++ b/lib/ansible/parsing/vault/__init__.py
@@ -398,18 +398,18 @@ def _edit_file_helper(self, filename, existing_data=None, force_save=False):
             self._shred_file(tmp_path)
             raise
 
-        tmpdata = self.read_data(tmp_path)
+        b_tmpdata = self.read_data(tmp_path)
 
         # Do nothing if the content has not changed
-        if existing_data == tmpdata and not force_save:
+        if existing_data == b_tmpdata and not force_save:
             self._shred_file(tmp_path)
             return
 
         # encrypt new data and write out to tmp
         # An existing vaultfile will always be UTF-8,
         # so decode to unicode here
-        enc_data = self.vault.encrypt(tmpdata.decode())
-        self.write_data(enc_data, tmp_path)
+        b_ciphertext = self.vault.encrypt(b_tmpdata)
+        self.write_data(b_ciphertext, tmp_path)
 
         # shuffle tmp file into place
         self.shuffle_files(tmp_path, filename)
@@ -420,9 +420,9 @@ def encrypt_file(self, filename, output_file=None):
 
         # A file to be encrypted into a vaultfile could be any encoding
         # so treat the contents as a byte string.
-        plaintext = self.read_data(filename)
-        ciphertext = self.vault.encrypt(plaintext)
-        self.write_data(ciphertext, output_file or filename)
+        b_plaintext = self.read_data(filename)
+        b_ciphertext = self.vault.encrypt(b_plaintext)
+        self.write_data(b_ciphertext, output_file or filename)
 
     def decrypt_file(self, filename, output_file=None):
 


Test patch: