Repository: ansible/ansible

Issue:
Paramiko crashes if a non-ascii character is used in the root password when running the playbook with -K
##### ISSUE TYPE
- Bug Report
##### ANSIBLE VERSION

Ansible is installed from Github:

```
ansible 2.2.0 (devel 46a97e1f55) last updated 2016/06/30 21:16:46 (GMT +100)
  lib/ansible/modules/core: (detached HEAD 1d0d5db97a) last updated 2016/06/30 21:16:54 (GMT +100)
  lib/ansible/modules/extras: (detached HEAD 00b8b96906) last updated 2016/06/30 21:16:54 (GMT +100)
  config file = .../jonblog-build/app/ansible.cfg [removed unimportant path info]
  configured module search path = Default w/o overrides
```
##### CONFIGURATION

```
cat ansible.cfg 
[defaults]
transport=paramiko
```
##### OS / ENVIRONMENT

Ubuntu 14.04
Python 2.7.6

Paramiko from `apt-get`:

```
apt-cache show python-paramiko
Package: python-paramiko
Priority: optional
Section: python
Installed-Size: 541
Maintainer: Ubuntu Developers <ubuntu-devel-discuss@lists.ubuntu.com>
Original-Maintainer: Jeremy T. Bouse <jbouse@debian.org>
Architecture: all
Source: paramiko
Version: 1.10.1-1git1build1
Provides: python2.7-paramiko
Depends: python (>= 2.7), python (<< 2.8), python:any (>= 2.7.1-0ubuntu2), python-crypto (>= 2.1.0-2)
Filename: pool/main/p/paramiko/python-paramiko_1.10.1-1git1build1_all.deb
Size: 105776
MD5sum: e728ed22af7bead915a5e509140971bb
SHA1: 92f20ae140eeceb36d456bb696b4a5e5c7f36585
SHA256: b665fdc994b7d1944bc54aab781f9110d2e1e6e4d33257151bf6d2f8a2f56a7c
Description-en: Make ssh v2 connections with Python (Python 2)
 This is a library for making SSH2 connections (client or server).
 Emphasis is on using SSH2 as an alternative to SSL for making secure
 connections between Python scripts.  All major ciphers and hash methods
 are supported.  SFTP client and server mode are both supported too.
 .
 This is the Python 2 version of the package.
Description-md5: ef19708cf575fb15646b6a0f32043a47
Homepage: https://github.com/paramiko/paramiko/
Bugs: https://bugs.launchpad.net/ubuntu/+filebug
Origin: Ubuntu
Supported: 5y
Task: edubuntu-desktop-gnome
```
##### SUMMARY

Using Paramiko as an SSH transport layer, with a `-K` root password containing a `£` pound sign causes a Python crash in Paramiko.
##### STEPS TO REPRODUCE

I call my playbook with `ansible-playbook -K ansible/fetch-database.yml`. More details [in this post](https://stackoverflow.com/questions/38132270/do-i-need-to-explicitly-switch-paramiko-or-ansible-to-use-utf-8-to-handle-non-as).
##### EXPECTED RESULTS

I expected the playbook to execute without errors or crashes. The password is correct, and using the default SSH transport, it seems to work fine. (I had originally swapped to Paramiko as the default transport was hanging upon successful `su`, but a pull on the Ansible repo fixed that, so I have swapped back).
##### ACTUAL RESULTS

Upon encountering the `su` code, I get this crash inside Paramiko:

```
An exception occurred during task execution. The full traceback is:
Traceback (most recent call last):
  File "/home/jon/Development/Personal/server-build/ansible/lib/ansible/executor/task_executor.py", line 96, in run
    item_results = self._run_loop(items)
  File "/home/jon/Development/Personal/server-build/ansible/lib/ansible/executor/task_executor.py", line 252, in _run_loop
    res = self._execute(variables=task_vars)
  File "/home/jon/Development/Personal/server-build/ansible/lib/ansible/executor/task_executor.py", line 447, in _execute
    result = self._handler.run(task_vars=variables)
  File "/home/jon/Development/Personal/server-build/ansible/lib/ansible/plugins/action/normal.py", line 33, in run
    results = merge_hash(results, self._execute_module(tmp=tmp, task_vars=task_vars))
  File "/home/jon/Development/Personal/server-build/ansible/lib/ansible/plugins/action/__init__.py", line 647, in _execute_module
    res = self._low_level_execute_command(cmd, sudoable=sudoable, in_data=in_data)
  File "/home/jon/Development/Personal/server-build/ansible/lib/ansible/plugins/action/__init__.py", line 721, in _low_level_execute_command
    rc, stdout, stderr = self._connection.exec_command(cmd, in_data=in_data, sudoable=sudoable)
  File "/home/jon/Development/Personal/server-build/ansible/lib/ansible/plugins/connection/paramiko_ssh.py", line 311, in exec_command
    chan.sendall(self._play_context.become_pass + '\n')
  File "/usr/lib/python2.7/dist-packages/paramiko/channel.py", line 797, in sendall
    sent = self.send(s)
  File "/usr/lib/python2.7/dist-packages/paramiko/channel.py", line 729, in send
    m.add_string(s[:size])
  File "/usr/lib/python2.7/dist-packages/paramiko/message.py", line 259, in add_string
    self.packet.write(s)
UnicodeEncodeError: 'ascii' codec can't encode character u'\xa3' in position 1: ordinal not in range(128)

fatal: [server.example.com]: FAILED! => {"failed": true, "msg": "Unexpected failure during module execution.", "stdout": ""}
```

It is possible that this would be solved with a bleeding-edge install of Paramiko. If you'd like me to install that in order to try it, please let me know how to do that - I'm just using the version in Ubuntu at the moment.



Discussion:
Contributing...

I got the same issue by using non-ascii characters in a private key path.

### ANSIBLE VERSION

2.0.0.2 installed via apt-get

### CONFIGURATION

The default one.

### OS / ENVIRONMENT

Xubuntu 16.04.1 LTS 64 bits

### SUMMARY

As previously reported, using --private-key instead of -K.

### STEPS TO REPRODUCE

Run any playbook, using a path containing non-ascii characters as the --private-key option argument. For instance:

$ ansible-playbook -i inventory.txt all -m ping --private-key=/home/bruno/Segurança/Chaves/master.pem

The "ç" in "Segurança" is a non-ascii character.

### EXPECTED RESULTS

I expected the playbook to execute without errors or crashes.

### ACTUAL RESULTS

I got the following error:

$ ansible -i inventory.txt all -m ping --sudo --private-key=/home/bruno/Segurança/Chaves/professional/master.pem

An exception occurred during task execution. To see the full traceback, use -vvv. The error was: UnicodeEncodeError: 'ascii' codec can't encode character u'\xe7' in position 19: ordinal not in range(128)
back0031 | FAILED! => {
    "failed": true, 
    "stdout": ""
}

@halfer, this appears to be fixed in devel and stable-2.2 branches:

```
$ ansible -i '127.0.0.2,' 127.0.0.2 -m command -a 'whoami' -c paramiko --su -K
SU password: 
127.0.0.2 | SUCCESS | rc=0 >>
root
```

@brunitto That appears to be a different problem.  If is still occurring, please file a new ticket so we can track and fix it.  We'll need the traceback to figure out where the error is occurring.  Use the -vvv switch to run ansible in verbose mode to get that.  Thanks!

After fixing the local connection for the same error, I dug around in this a little more and found that the code looks like it should still fail in devel and stable-2.2 but I was unable to reproduce on my laptop.  I switched to an ubuntu14 virtual machine and the error still occurred on devel.  Seeing if  the same fix as I implemented for local connection will work for paramiko.


Resolving patch:
diff --git a/lib/ansible/playbook/play_context.py b/lib/ansible/playbook/play_context.py
--- a/lib/ansible/playbook/play_context.py
+++ b/lib/ansible/playbook/play_context.py
@@ -31,6 +31,7 @@
 from ansible.compat.six import iteritems, string_types
 from ansible import constants as C
 from ansible.errors import AnsibleError
+from ansible.module_utils._text import to_bytes
 from ansible.playbook.attribute import FieldAttribute
 from ansible.playbook.base import Base
 from ansible.utils.boolean import boolean
@@ -84,38 +85,38 @@
    module_compression = ('ansible_module_compression',),
 )
 
-SU_PROMPT_LOCALIZATIONS = [
-    'Password',
-    '암호',
-    'パスワード',
-    'Adgangskode',
-    'Contraseña',
-    'Contrasenya',
-    'Hasło',
-    'Heslo',
-    'Jelszó',
-    'Lösenord',
-    'Mật khẩu',
-    'Mot de passe',
-    'Parola',
-    'Parool',
-    'Pasahitza',
-    'Passord',
-    'Passwort',
-    'Salasana',
-    'Sandi',
-    'Senha',
-    'Wachtwoord',
-    'ססמה',
-    'Лозинка',
-    'Парола',
-    'Пароль',
-    'गुप्तशब्द',
-    'शब्दकूट',
-    'సంకేతపదము',
-    'හස්පදය',
-    '密码',
-    '密碼',
+b_SU_PROMPT_LOCALIZATIONS = [
+    to_bytes('Password'),
+    to_bytes('암호'),
+    to_bytes('パスワード'),
+    to_bytes('Adgangskode'),
+    to_bytes('Contraseña'),
+    to_bytes('Contrasenya'),
+    to_bytes('Hasło'),
+    to_bytes('Heslo'),
+    to_bytes('Jelszó'),
+    to_bytes('Lösenord'),
+    to_bytes('Mật khẩu'),
+    to_bytes('Mot de passe'),
+    to_bytes('Parola'),
+    to_bytes('Parool'),
+    to_bytes('Pasahitza'),
+    to_bytes('Passord'),
+    to_bytes('Passwort'),
+    to_bytes('Salasana'),
+    to_bytes('Sandi'),
+    to_bytes('Senha'),
+    to_bytes('Wachtwoord'),
+    to_bytes('ססמה'),
+    to_bytes('Лозинка'),
+    to_bytes('Парола'),
+    to_bytes('Пароль'),
+    to_bytes('गुप्तशब्द'),
+    to_bytes('शब्दकूट'),
+    to_bytes('సంకేతపదము'),
+    to_bytes('හස්පදය'),
+    to_bytes('密码'),
+    to_bytes('密碼'),
 ]
 
 TASK_ATTRIBUTE_OVERRIDES = (
@@ -508,9 +509,9 @@ def make_become_cmd(self, cmd, executable=None):
             elif self.become_method == 'su':
 
                 # passing code ref to examine prompt as simple string comparisson isn't good enough with su
-                def detect_su_prompt(data):
-                    SU_PROMPT_LOCALIZATIONS_RE = re.compile("|".join(['(\w+\'s )?' + x + ' ?: ?' for x in SU_PROMPT_LOCALIZATIONS]), flags=re.IGNORECASE)
-                    return bool(SU_PROMPT_LOCALIZATIONS_RE.match(data))
+                def detect_su_prompt(b_data):
+                    b_SU_PROMPT_LOCALIZATIONS_RE = re.compile(b"|".join([b'(\w+\'s )?' + x + b' ?: ?' for x in b_SU_PROMPT_LOCALIZATIONS]), flags=re.IGNORECASE)
+                    return bool(b_SU_PROMPT_LOCALIZATIONS_RE.match(b_data))
                 prompt = detect_su_prompt
 
                 becomecmd = '%s %s %s -c %s' % (exe, flags, self.become_user, pipes.quote(command))
@@ -521,8 +522,8 @@ def detect_su_prompt(data):
                 becomecmd = '%s -b %s -u %s %s' % (exe, flags, self.become_user, success_cmd)
 
             elif self.become_method == 'ksu':
-                def detect_ksu_prompt(data):
-                    return re.match("Kerberos password for .*@.*:", data)
+                def detect_ksu_prompt(b_data):
+                    return re.match(b"Kerberos password for .*@.*:", b_data)
 
                 prompt = detect_ksu_prompt
                 becomecmd = '%s %s %s -e %s' % (exe, self.become_user, flags, command)
diff --git a/lib/ansible/plugins/connection/local.py b/lib/ansible/plugins/connection/local.py
--- a/lib/ansible/plugins/connection/local.py
+++ b/lib/ansible/plugins/connection/local.py
@@ -90,7 +90,7 @@ def exec_command(self, cmd, in_data=None, sudoable=True):
         if self._play_context.prompt and sudoable:
             fcntl.fcntl(p.stdout, fcntl.F_SETFL, fcntl.fcntl(p.stdout, fcntl.F_GETFL) | os.O_NONBLOCK)
             fcntl.fcntl(p.stderr, fcntl.F_SETFL, fcntl.fcntl(p.stderr, fcntl.F_GETFL) | os.O_NONBLOCK)
-            become_output = ''
+            become_output = b''
             while not self.check_become_success(become_output) and not self.check_password_prompt(become_output):
 
                 rfd, wfd, efd = select.select([p.stdout, p.stderr], [], [p.stdout, p.stderr], self._play_context.timeout)
@@ -100,10 +100,10 @@ def exec_command(self, cmd, in_data=None, sudoable=True):
                     chunk = p.stderr.read()
                 else:
                     stdout, stderr = p.communicate()
-                    raise AnsibleError('timeout waiting for privilege escalation password prompt:\n' + become_output)
+                    raise AnsibleError('timeout waiting for privilege escalation password prompt:\n' + to_native(become_output))
                 if not chunk:
                     stdout, stderr = p.communicate()
-                    raise AnsibleError('privilege output closed while waiting for password prompt:\n' + become_output)
+                    raise AnsibleError('privilege output closed while waiting for password prompt:\n' + to_native(become_output))
                 become_output += chunk
             if not self.check_become_success(become_output):
                 p.stdin.write(to_bytes(self._play_context.become_pass, errors='surrogate_or_strict') + b'\n')
diff --git a/lib/ansible/plugins/connection/paramiko_ssh.py b/lib/ansible/plugins/connection/paramiko_ssh.py
--- a/lib/ansible/plugins/connection/paramiko_ssh.py
+++ b/lib/ansible/plugins/connection/paramiko_ssh.py
@@ -310,7 +310,7 @@ def exec_command(self, cmd, in_data=None, sudoable=True):
 
                 if passprompt:
                     if self._play_context.become and self._play_context.become_pass:
-                        chan.sendall(self._play_context.become_pass + '\n')
+                        chan.sendall(to_bytes(self._play_context.become_pass) + b'\n')
                     else:
                         raise AnsibleError("A password is reqired but none was supplied")
                 else:


Test patch: