Repository: ansible/ansible

Issue:
ansible_python_interpreter ignored in Ansible 2.4 for apt module when use async parameter
<!---
Verify first that your issue/request is not already reported on GitHub.
Also test if the latest release, and devel branch are affected too.
-->

##### ISSUE TYPE
<!--- Pick one below and delete the rest -->
 - Bug Report

##### COMPONENT NAME
apt module

##### ANSIBLE VERSION
<!--- Paste verbatim output from "ansible --version" between quotes below -->
```
ansible 2.4.3.0
  config file = /home/ozlevka/tmp/terraform/ansible.cfg
  configured module search path = ['/home/ozlevka/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /home/ozlevka/.virtualenv/ansible/lib/python3.6/site-packages/ansible
  executable location = /home/ozlevka/.virtualenv/ansible/bin/ansible
  python version = 3.6.3 (default, Nov 17 2017, 17:34:18) [GCC 5.4.0 20160609]

```

##### CONFIGURATION
<!---
If using Ansible 2.4 or above, paste the results of "ansible-config dump --only-changed"
Otherwise, mention any settings you have changed/added/removed in ansible.cfg
(or using the ANSIBLE_* environment variables).
-->

##### OS / ENVIRONMENT
<!---
Mention the OS you are running Ansible from, and the OS you are
managing, or say "N/A" for anything that is not platform-specific.
Also mention the specific version of what you are trying to control,
e.g. if this is a network bug the version of firmware on the network device.
-->
Linux ubuntu-server 4.4.0-87-generic #110-Ubuntu SMP Tue Jul 18 12:55:35 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
##### SUMMARY
<!--- Explain the problem briefly -->
The apt module ignore the ansible_python_interpreter variable when use async parameter
##### STEPS TO REPRODUCE
<!---
For bugs, show exactly how to reproduce the problem, using a minimal test-case.
For new features, show how the feature would be used.
-->
configure ansible_python_interpreter=/usr/bin/python3 in inventory
default python interpreter in Ubuntu 16.04
Run playbook

<!--- Paste example playbooks or commands between quotes below -->
``` yaml
 - name: Install docker dependencies
      apt:
        name: software-properties-common
        update_cache: true
        state: latest
      async: 3600
      poll: 0
```
<!--- You can also paste gist.github.com links for larger files -->

##### EXPECTED RESULTS
<!--- What did you expect to happen when running the steps above? -->
Playbook install desired package
##### ACTUAL RESULTS
<!--- What actually happened? If possible run with extra verbosity (-vvvv) -->
```
fatal: [first]: FAILED! => {"changed": true, "module_stderr": "Shared connection to 10.0.0.56 closed.\r\n", "module_stdout": "/bin/sh: 1: /usr/bin/python: not found\r\n", "msg": "MODULE FAILURE", "rc": 127}
```




Discussion:
Files identified in the description:
* [lib/ansible/modules/packaging/os/apt.py](https://github.com/ansible/ansible/blob/devel/lib/ansible/modules/packaging/os/apt.py)

If these files are inaccurate, please update the `component name` section of the description or use the `!component` bot command.

[click here for bot help](https://github.com/ansible/ansibullbot/blob/master/ISSUE_HELP.md)
<!--- boilerplate: components_banner --->
cc @mgwilliams
[click here for bot help](https://github.com/ansible/ansibullbot/blob/master/ISSUE_HELP.md)
<!--- boilerplate: notify --->

Resolving patch:
diff --git a/lib/ansible/executor/module_common.py b/lib/ansible/executor/module_common.py
--- a/lib/ansible/executor/module_common.py
+++ b/lib/ansible/executor/module_common.py
@@ -35,7 +35,7 @@
 from ansible.release import __version__, __author__
 from ansible import constants as C
 from ansible.errors import AnsibleError
-from ansible.module_utils._text import to_bytes, to_text
+from ansible.module_utils._text import to_bytes, to_text, to_native
 from ansible.plugins.loader import module_utils_loader, ps_module_utils_loader
 from ansible.plugins.shell.powershell import async_watchdog, async_wrapper, become_wrapper, leaf_exec, exec_wrapper
 # Must import strategy and use write_locks from there
@@ -60,6 +60,7 @@
 # We could end up writing out parameters with unicode characters so we need to
 # specify an encoding for the python source file
 ENCODING_STRING = u'# -*- coding: utf-8 -*-'
+b_ENCODING_STRING = b'# -*- coding: utf-8 -*-'
 
 # module_common is relative to module_utils, so fix the path
 _MODULE_UTILS_PATH = os.path.join(os.path.dirname(__file__), '..', 'module_utils')
@@ -911,25 +912,29 @@ def modify_module(module_name, module_path, module_args, task_vars=None, templar
     if module_style == 'binary':
         return (b_module_data, module_style, to_text(shebang, nonstring='passthru'))
     elif shebang is None:
-        lines = b_module_data.split(b"\n", 1)
-        if lines[0].startswith(b"#!"):
-            shebang = lines[0].strip()
-            args = shlex.split(str(shebang[2:]))
+        b_lines = b_module_data.split(b"\n", 1)
+        if b_lines[0].startswith(b"#!"):
+            b_shebang = b_lines[0].strip()
+            # shlex.split on python-2.6 needs bytes.  On python-3.x it needs text
+            args = shlex.split(to_native(b_shebang[2:], errors='surrogate_or_strict'))
+
+            # _get_shebang() takes text strings
+            args = [to_text(a, errors='surrogate_or_strict') for a in args]
             interpreter = args[0]
-            interpreter = to_bytes(interpreter)
+            b_new_shebang = to_bytes(_get_shebang(interpreter, task_vars, templar, args[1:])[0],
+                                     errors='surrogate_or_strict', nonstring='passthru')
 
-            new_shebang = to_bytes(_get_shebang(interpreter, task_vars, templar, args[1:])[0], errors='surrogate_or_strict', nonstring='passthru')
-            if new_shebang:
-                lines[0] = shebang = new_shebang
+            if b_new_shebang:
+                b_lines[0] = b_shebang = b_new_shebang
 
-            if os.path.basename(interpreter).startswith(b'python'):
-                lines.insert(1, to_bytes(ENCODING_STRING))
+            if os.path.basename(interpreter).startswith(u'python'):
+                b_lines.insert(1, b_ENCODING_STRING)
+
+            shebang = to_text(b_shebang, nonstring='passthru', errors='surrogate_or_strict')
         else:
             # No shebang, assume a binary module?
             pass
 
-        b_module_data = b"\n".join(lines)
-    else:
-        shebang = to_bytes(shebang, errors='surrogate_or_strict')
+        b_module_data = b"\n".join(b_lines)
 
-    return (b_module_data, module_style, to_text(shebang, nonstring='passthru'))
+    return (b_module_data, module_style, shebang)


Test patch: