Repository: ansible/ansible

Issue:
ansible_shell_executable is not used by default by module_utils/basic.py AnsibleModule run_command
##### ISSUE TYPE
 - Bug Report

##### COMPONENT NAME
core ?

##### ANSIBLE VERSION

```
ansible 2.3.0.0
  config file = /etc/ansible/ansible.cfg
  configured module search path = Default w/o overrides
  python version = 2.7.13 (default, Jan 12 2017, 17:59:37) [GCC 6.3.1 20161221 (Red Hat 6.3.1-1)]
```

##### CONFIGURATION

##### OS / ENVIRONMENT
N/A

##### SUMMARY
In order to manage "exotic" platforms (embedded Linux, Android, etc), one might need to set android_shell_executable in the inventory. This inventory variable isn't used as default by all modules relying on shell (setup, shell, and possibly others), so that they fail by default with an unclear 'file not found' error.
This is due to module_utils/basic.py AnsibleModule.run_command having the parameter `executable` empty by default, defaulting to `/bin/sh`, whereas it should IMHO default to the shell set by the user.
One can work around this issue where modules like `shell` offer the possibility to explicitly overwrite the shell used, but not with modules like `setup`, which don't offer this possibility.

##### STEPS TO REPRODUCE
1. move away `/bin/sh` from your managed host
2. set in your inventory ansible_shell_executable to the new shell location e.g. /opt/bin/sh
3. call `ansible -i my.inv -m setup managedhost` or `ansible -i my.inv -m shell -a id managedhost` and they fail.
4. Call `ansible -i my.inv -m shell -a 'executable=/opt/bin/sh id' managedhost` and it succeeds

##### EXPECTED RESULTS
The modules called use ansible_shell_executable and succeed.

##### ACTUAL RESULTS
The modules fail with a file not found error, which relates obscurely to /bin/sh not being found.

```
$ ansible -i ansidroid.inventory -m setup s4 -vvv
Using /etc/ansible/ansible.cfg as config file
META: ran handlers
Using module file /usr/lib/python2.7/site-packages/ansible/modules/system/setup.py                                                                              
<192.168.1.208> ESTABLISH SSH CONNECTION FOR USER: shell
<192.168.1.208> SSH: EXEC ssh -C -o ControlMaster=auto -o ControlPersist=60s -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o User=shell -o ConnectTimeout=10 -o ControlPath=/home/ericl/.ansible/cp/c950afe49b 192.168.1.208 '/system/xbin/bash -c '"'"'echo ~ && sleep 0'"'"''                                 
<192.168.1.208> (0, '/data/data/com.android.shell\n', '')
<192.168.1.208> ESTABLISH SSH CONNECTION FOR USER: shell
<192.168.1.208> SSH: EXEC ssh -C -o ControlMaster=auto -o ControlPersist=60s -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o User=shell -o ConnectTimeout=10 -o ControlPath=/home/ericl/.ansible/cp/c950afe49b 192.168.1.208 '/system/xbin/bash -c '"'"'( umask 77 && mkdir -p "` echo /data/data/com.android.shell/.ansible/tmp/ansible-tmp-1493649135.44-32469499696969 `" && echo ansible-tmp-1493649135.44-32469499696969="` echo /data/data/com.android.shell/.ansible/tmp/ansible-tmp-1493649135.44-32469499696969 `" ) && sleep 0'"'"''
<192.168.1.208> (0, 'ansible-tmp-1493649135.44-32469499696969=/data/data/com.android.shell/.ansible/tmp/ansible-tmp-1493649135.44-32469499696969\n', '')
<192.168.1.208> PUT /tmp/tmpAQncVb TO /data/data/com.android.shell/.ansible/tmp/ansible-tmp-1493649135.44-32469499696969/setup.py
<192.168.1.208> SSH: EXEC sftp -b - -C -o ControlMaster=auto -o ControlPersist=60s -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o User=shell -o ConnectTimeout=10 -o ControlPath=/home/ericl/.ansible/cp/c950afe49b '[192.168.1.208]'
<192.168.1.208> (0, 'sftp> put /tmp/tmpAQncVb /data/data/com.android.shell/.ansible/tmp/ansible-tmp-1493649135.44-32469499696969/setup.py\n', '')
<192.168.1.208> ESTABLISH SSH CONNECTION FOR USER: shell
<192.168.1.208> SSH: EXEC ssh -C -o ControlMaster=auto -o ControlPersist=60s -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o User=shell -o ConnectTimeout=10 -o ControlPath=/home/ericl/.ansible/cp/c950afe49b 192.168.1.208 '/system/xbin/bash -c '"'"'chmod u+x /data/data/com.android.shell/.ansible/tmp/ansible-tmp-1493649135.44-32469499696969/ /data/data/com.android.shell/.ansible/tmp/ansible-tmp-1493649135.44-32469499696969/setup.py && sleep 0'"'"''
<192.168.1.208> (0, '', '')
<192.168.1.208> ESTABLISH SSH CONNECTION FOR USER: shell
<192.168.1.208> SSH: EXEC ssh -C -o ControlMaster=auto -o ControlPersist=60s -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o User=shell -o ConnectTimeout=10 -o ControlPath=/home/ericl/.ansible/cp/c950afe49b -tt 192.168.1.208 '/system/xbin/bash -c '"'"'/data/data/com.android.shell/python/bin/python /data/data/com.android.shell/.ansible/tmp/ansible-tmp-1493649135.44-32469499696969/setup.py; rm -rf "/data/data/com.android.shell/.ansible/tmp/ansible-tmp-1493649135.44-32469499696969/" > /dev/null 2>&1 && sleep 0'"'"''

<192.168.1.208> (0, '\r\n{"msg": "[Errno 2] No such file or directory", "failed": true, "cmd": "ps -p 1 -o \'comm|tail\' -n 1", "rc": 2, "invocation": {"module_args": {"filter": "*", "gather_subset": ["all"], "fact_path": "/etc/ansible/facts.d", "gather_timeout": 10}}}\r\n', 'Shared connection to 192.168.1.208 closed.\r\n')
s4 | FAILED! => {
    "changed": false, 
    "cmd": "ps -p 1 -o 'comm|tail' -n 1", 
    "failed": true, 
    "invocation": {
        "module_args": {
            "fact_path": "/etc/ansible/facts.d", 
            "filter": "*", 
            "gather_subset": [
                "all"
            ], 
            "gather_timeout": 10
        }
    }, 
    "msg": "[Errno 2] No such file or directory", 
    "rc": 2
}
```



Discussion:
We don't pass the executable information to modules, not all of them execute commands.

For those that do, most use run_command w/o shell, the fact gathering module is one of the few exceptions that relies on shell semantics, but it does not explicitly set it, it relies on python's `Popen` and passes `shell=True` which leads to the execution of `/bin/sh` (which is expected to be present on all POSIX platforms).
Hi,

On 01/05/17 18:27, Brian Coca wrote:
> We don't pass the executable information to modules, not all of them
> execute commands.
Correct and understood, but the question is if it's the right decision.

> For those that do, most use run_command w/o shell, the fact gathering
> module is one of the few exceptions that relies on shell semantics, but
> it does not explicitly set it, it relies on python's |Popen| and passes
> |shell=True| which leads to the execution of |/bin/sh| (which is
> expected to be present on all POSIX platforms).

But there is a reason for the ansible_shell_executable parameter? And if
it's not passed everywhere, it's kind of a moot parameter, or what
remains as use case: either it's needed everywhere or it's needed nowhere?

Thanks, Eric

I was explaining the current state, I did not close this issue as we might want to revisit the current state of affairs.

The parameter is used mostly for 'controller side' construction of commands, we remotely create/copy/execute/remove files (which includes modules).

Looking at the Popen code it actually hard codes it `args = ["/bin/sh", "-c"] `,  

I'm thinking the simplest solution is for run_command to use `$SHELL` and override Popen with that when using `shell=True`, there would be no need to pass it as you are already 'using it' and executing the module under it.

Resolving patch:
diff --git a/lib/ansible/module_utils/basic.py b/lib/ansible/module_utils/basic.py
--- a/lib/ansible/module_utils/basic.py
+++ b/lib/ansible/module_utils/basic.py
@@ -2313,7 +2313,6 @@ def run_command(self, args, check_rc=False, close_fds=True, executable=None, dat
             strings on python3, use encoding=None to turn decoding to text off.
         '''
 
-        shell = False
         if isinstance(args, list):
             if use_unsafe_shell:
                 args = " ".join([shlex_quote(x) for x in args])
@@ -2321,17 +2320,27 @@ def run_command(self, args, check_rc=False, close_fds=True, executable=None, dat
         elif isinstance(args, (binary_type, text_type)) and use_unsafe_shell:
             shell = True
         elif isinstance(args, (binary_type, text_type)):
-            # On python2.6 and below, shlex has problems with text type
-            # On python3, shlex needs a text type.
-            if PY2:
-                args = to_bytes(args, errors='surrogate_or_strict')
-            elif PY3:
-                args = to_text(args, errors='surrogateescape')
-            args = shlex.split(args)
+            if not use_unsafe_shell:
+                # On python2.6 and below, shlex has problems with text type
+                # On python3, shlex needs a text type.
+                if PY2:
+                    args = to_bytes(args, errors='surrogate_or_strict')
+                elif PY3:
+                    args = to_text(args, errors='surrogateescape')
+                args = shlex.split(args)
         else:
             msg = "Argument 'args' to run_command must be list or string"
             self.fail_json(rc=257, cmd=args, msg=msg)
 
+        shell = False
+        if use_unsafe_shell:
+            if executable is None:
+                executable = os.environ.get('SHELL')
+            if executable:
+                args = [executable, '-c', args]
+            else:
+                shell = True
+
         prompt_re = None
         if prompt_regex:
             if isinstance(prompt_regex, text_type):


Test patch: