Repository: ansible/ansible

Issue:
v2 regression: fetch/su is broken
Using the latest unmodified devel (@e2e9319f), a task like this:

```
- fetch: src=/etc/motd dest=motd
  su: yes
```

produces this in the debug output:

```
<raven.toroid.org> EXEC ssh -C -tt -vvv -o ControlMaster=auto -o ControlPersist=60s -o ControlPath="/home/ams/.ansible/cp/ansible-ssh-%h-%p-%r" -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi
-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o ConnectTimeout=10 raven.toroid.org /bin/sh -c 'su  root -c "/bin/sh -c '"'"'echo BECOME-SUCCESS-nrkohdhsakbxamjlftxhmfgphtzxevls; rc=flag;
 [ -r /etc/motd ] || rc=2; [ -f /etc/motd ] || rc=1; [ -d /etc/motd ] && rc=3; python -V 2>/dev/null || rc=4; [ x"$rc" != "xflag" ] && echo "${rc}  "/etc/motd && exit 0; (python -c '"'"'"'"'"'"'"'"'import hashlib;
 BLOCKSIZE = 65536; hasher = hashlib.sha1();
afile = open("'"'"'"'"'"'"'"'"'/etc/motd'"'"'"'"'"'"'"'"'", "rb")
buf = afile.read(BLOCKSIZE)
while len(buf) > 0:
        hasher.update(buf)
        buf = afile.read(BLOCKSIZE)
afile.close()
print(hasher.hexdigest())'"'"'"'"'"'"'"'"' 2>/dev/null) || (python -c '"'"'"'"'"'"'"'"'import sha; BLOCKSIZE = 65536; hasher = sha.sha();
afile = open("'"'"'"'"'"'"'"'"'/etc/motd'"'"'"'"'"'"'"'"'", "rb")
buf = afile.read(BLOCKSIZE)
while len(buf) > 0:
        hasher.update(buf)
        buf = afile.read(BLOCKSIZE)
afile.close()
print(hasher.hexdigest())'"'"'"'"'"'"'"'"' 2>/dev/null) || (echo '"'"'"'"'"'"'"'"'0  '"'"'"'"'"'"'"'"'/etc/motd)'"'"'"'
 24251 1441789764.96724: Handling privilege escalation password prompt.
 24251 1441789764.96734: Waiting for Privilege Escalation input
 24251 1441789764.97364: Read next chunks
 24251 1441789764.97372: stderr chunk is: [removed, irrelevant ssh output]

 24251 1441789764.97394: Waiting for Privilege Escalation input
 24251 1441789764.97403: Read next chunks
 24251 1441789765.33084: stderr chunk is: debug1: mux_client_request_session: master session id: 2^M

 24251 1441789765.33136: Waiting for Privilege Escalation input
 24251 1441789765.33157: Read next chunks
 24251 1441789765.69062: stdout chunk is: /bin/sh: -c: line 0: syntax error near unexpected token `('^M
/bin/sh: -c: line 0: `su  root -c "/bin/sh -c 'echo BECOME-SUCCESS-nrkohdhsakbxamjlftxhmfgphtzxevls; rc=flag; [ -r /etc/motd ] || rc=2; [ -f /etc/motd ] || rc=1; [ -d /etc/motd ] && rc=3; python -V 2>/dev/null ||
rc=4; [ x"$rc" != "xflag" ] && echo "${rc}  "/etc/motd && exit 0; (python -c '"'"'import hashlib; BLOCKSIZE = 65536; hasher = hashlib.sha1();'^M
```

Notice the `/bin/sh: -c: line 0: syntax error near unexpected token `(`



Discussion:
I notice that we do this in `playbook/play_context.py`, in the `make_become_cmd` method:

```
                prompt = detect_su_prompt
                becomecmd = '%s %s %s -c "%s -c %s"' % (exe, flags, self.become_user, executable, success_cmd)
                ...
            return ('%s -c %s' % (executable, pipes.quote(becomecmd)))
```

And so we issue `sh -c 'su root -c 'sh -c 'command'''` or `sh -c 'sudo 'sh -c 'command''` (with extra quoting). I do not know why we don't just use `su -c 'command'` or `sudo sh -c 'command'`. Indeed, when I change the code to do this, I can no longer reproduce the shell error above.

@amenonsen the above patch fixes this, and also passes the following test playbook:

<pre>
- hosts: all
  gather_facts: no
  remote_user: testing
  tasks:
  - command: whoami
  - command: whoami
    become: yes
    become_method: sudo
  - command: whoami
    become: yes
    become_method: su
  - fetch: src=/etc/hosts dest=/tmp/
  - fetch: src=/etc/passwd dest=/tmp/
    become: yes
    become_method: sudo
  - fetch: src=/etc/shadow dest=/tmp/
    become: yes
    become_method: su
</pre>

When run:

<pre>
TASK [command] *****************************************************************
changed: [awxlocal] => {"changed": true, "cmd": ["whoami"], "delta": "0:00:00.001569", "end": "2015-09-20 08:12:53.779747", "rc": 0, "start": "2015-09-20 08:12:53.778178", "stderr": "", "stdout": "testing", "stdout_lines": ["testing"], "warnings": []}
TASK [command] *****************************************************************
changed: [awxlocal] => {"changed": true, "cmd": ["whoami"], "delta": "0:00:00.001440", "end": "2015-09-20 08:12:53.922317", "rc": 0, "start": "2015-09-20 08:12:53.920877", "stderr": "", "stdout": "root", "stdout_lines": ["root"], "warnings": []}
TASK [command] *****************************************************************
changed: [awxlocal] => {"changed": true, "cmd": ["whoami"], "delta": "0:00:00.001540", "end": "2015-09-20 08:12:54.086455", "rc": 0, "start": "2015-09-20 08:12:54.084915", "stderr": "", "stdout": "root", "stdout_lines": ["root"], "warnings": []}
TASK [fetch dest=/tmp/ src=/etc/hosts] *****************************************
changed: [awxlocal] => {"changed": true, "checksum": "5beaa461c1878cd5fc23a46a37f543b4b3628a0b", "dest": "/tmp/awxlocal/etc/hosts", "md5sum": "2eadc1e7ebf3572b44e120b46472dafe", "remote_checksum": "5beaa461c1878cd5fc23a46a37f543b4b3628a0b", "remote_md5sum": null}
TASK [fetch dest=/tmp/ src=/etc/passwd] ****************************************
changed: [awxlocal] => {"changed": true, "checksum": "8bc22051fa5e475f92fa9af8c889144fe4207972", "dest": "/tmp/awxlocal/etc/passwd", "md5sum": "9a0d7bc68563cf326dfb37faa867b0bd", "remote_checksum": "8bc22051fa5e475f92fa9af8c889144fe4207972", "remote_md5sum": null}
TASK [fetch dest=/tmp/ src=/etc/shadow] ****************************************
changed: [awxlocal] => {"changed": true, "checksum": "f58b07aae7c39eb14f50a6eeb2bb27e6caa074a2", "dest": "/tmp/awxlocal/etc/shadow", "md5sum": "4ce3c26026f755954ff0fafae3cadf30", "remote_checksum": "f58b07aae7c39eb14f50a6eeb2bb27e6caa074a2", "remote_md5sum": null}
PLAY RECAP *********************************************************************
awxlocal                   : ok=6    changed=6    unreachable=0    failed=0   
</pre>

When re-run:

<pre>
TASK [command] *****************************************************************
changed: [awxlocal] => {"changed": true, "cmd": ["whoami"], "delta": "0:00:00.001555", "end": "2015-09-20 08:13:10.183234", "rc": 0, "start": "2015-09-20 08:13:10.181679", "stderr": "", "stdout": "testing", "stdout_lines": ["testing"], "warnings": []}
TASK [command] *****************************************************************
changed: [awxlocal] => {"changed": true, "cmd": ["whoami"], "delta": "0:00:00.001539", "end": "2015-09-20 08:13:10.323168", "rc": 0, "start": "2015-09-20 08:13:10.321629", "stderr": "", "stdout": "root", "stdout_lines": ["root"], "warnings": []}
TASK [command] *****************************************************************
changed: [awxlocal] => {"changed": true, "cmd": ["whoami"], "delta": "0:00:00.001589", "end": "2015-09-20 08:13:10.483456", "rc": 0, "start": "2015-09-20 08:13:10.481867", "stderr": "", "stdout": "root", "stdout_lines": ["root"], "warnings": []}
TASK [fetch dest=/tmp/ src=/etc/hosts] *****************************************
ok: [awxlocal] => {"changed": false, "checksum": "5beaa461c1878cd5fc23a46a37f543b4b3628a0b", "dest": "/tmp/awxlocal/etc/hosts", "file": "/etc/hosts", "md5sum": "2eadc1e7ebf3572b44e120b46472dafe"}
TASK [fetch dest=/tmp/ src=/etc/passwd] ****************************************
ok: [awxlocal] => {"changed": false, "checksum": "8bc22051fa5e475f92fa9af8c889144fe4207972", "dest": "/tmp/awxlocal/etc/passwd", "file": "/etc/passwd", "md5sum": "9a0d7bc68563cf326dfb37faa867b0bd"}
TASK [fetch dest=/tmp/ src=/etc/shadow] ****************************************
ok: [awxlocal] => {"changed": false, "checksum": "f58b07aae7c39eb14f50a6eeb2bb27e6caa074a2", "dest": "/tmp/awxlocal/etc/shadow", "file": "/etc/shadow", "md5sum": "4ce3c26026f755954ff0fafae3cadf30"}
</pre>

cc @bcoca 

Due to the nature of this change, I would definitely like some increased testing, especially on BSD systems (which traditionally have caused us more command quoting issues).

so i cannot reproduce this anymore, @amenonsen  please check against current head, it seems other work has fixed this as a side effect.

Still happens for me.

```
fatal: [raven.toroid.org]: FAILED! => {"failed": true, "msg": "ERROR! Timeout (0s) waiting for privilege escalation prompt: /bin/sh: -c: line 0: syntax error near unexpected token `('\n/bin/sh: -c: line 0: `su  root -c \"/bin/sh -c 'echo BECOME-SUCCESS-zffxclzljewksgggykwkqwusobdtpzua; rc=flag; [ -r /etc/shadow ] || rc=2; [ -f /etc/shadow ] || rc=1; [ -d /etc/shadow ] && rc=3; python -V 2>/dev/null || rc=4; [ x\"$rc\" != \"xflag\" ] && echo \"${rc}  \"/etc/shadow && exit 0; (python -c '\"'\"'import hashlib; BLOCKSIZE = 65536; hasher = hashlib.sha1();'\n"}
```

Noting here info from IRC discussions:

This seems to happen in certain combinations of Debian/Openssh when using the ssh connection plugin, but not paramiko. 

@amenonsen seems to have fix related to become command construction in  the works. 

I've submitted a minimal fix for the quoting problem in PR #13176 (along with a fix to make `doas` support work properly on OpenBSD), and opened a new issue #13179 about a problem I discovered while testing the fix.

I did some more research on this. Here's what happens.
1. The `fetch` module calls `_remote_checksum`, which constructs a command, say `foo` (but we know that foo contains embedded double-quote characters).
2. `make_become_cmd` calls `pipes.quote()` on the above command (so we end up with `'echo XXX; foo'` where single quotes are escaped as `'"'"'` but double quotes aren't escaped at all), and then interpolates it into the `su` command string using `'… "%s -c %s"' % (…, success_cmd)`. This is simply **wrong**; the first unescaped internal double quote will terminate the argument string.
3. We send this command to the remote host, and get an error back from `/bin/sh` that is some variation of `/bin/sh: syntax error: `(' unexpected`.
4. `_remote_checksum` splits this up and returns `/bin/sh:` as its result checksum value.
5. Back in the fetch module, if we're using `become` (which we are, via su), we ignore the result of the `_remote_checksum()` call and fetch the data via `slurp` and recalculate the remote checksum.
6. Then the code goes on to transfer the file etc. and the `fetch` task succeeds.

https://gist.github.com/anonymous/2353b9cd0dcb5a8ff3c2 is the command produced in step (2) above and executed on the remote host. If you cut and paste this command into a shell, you'll get the same `unexpected (` error. (It's harmless: even if it succeeds, the worst it could do is to print a digest of `/etc/foo`.)

This is not in any way specific to my system or any particular version of the shell. I've seen it happen on three different versions of Debian now, on OpenBSD and FreeBSD, and Red Hat. Everyone I've asked to try the gist above has reported identical errors on their systems (including @willthames on Fedora). The same thing even happens with `-c paramiko`, it's just that it doesn't log the incoming data the same way, so I didn't notice the error. (I added some additional logging around `_remote_checksum` and verified that it also gets exactly the same `/bin/sh` error.)

TL;DR: we quote su commands incorrectly, and remote checksum commands fail, but when we're using su we ignore it and always recalculate the checksum, so the `fetch:` task succeeds.

@bcoca The only mystery is why you don't see the same error, but at this point I wonder if you're just missing it in among all the debug output (which is the only place where it's visible).

My fix is correct, and should be merged.

If anyone is curious, this can be reproduced easily using the following command. Substitute a suitable hostname for `some.ho.st`:

```
$ ANSIBLE_DEBUG=1 ansible -i 'some.ho.st,' -vvv --su --ask-su-pass all -m fetch -a "src=/etc/motd dest=/tmp/" &> experiment.12290
$ grep error experiment.12290
>>>/bin/sh: -c: line 0: syntax error near unexpected token `('
```

If you do **not** see the above error, then I would be very interested in seeing your `experiment.12290` file.

BTW, note that we should avoid issuing the checksum command altogether in copy/fetch if we're using become; it would save one completely unnecessary SSH connection. If this fix gets merged, I'll submit a separate PR for that one.

FWIW I can reproduce:

```
$ ansible --version
ansible 2.0.0 (devel 9b9fb51d9d) last updated 2015/11/18 11:45:05 (GMT +300)
  lib/ansible/modules/core: (detached HEAD b1c0249045) last updated 2015/11/18 11:45:09 (GMT +300)
  lib/ansible/modules/extras: (detached HEAD 7da1f8d4ca) last updated 2015/11/18 11:45:09 (GMT +300)
  config file = 
  configured module search path = Default w/o overrides

$ ANSIBLE_DEBUG=1 ansible -i 'fridge,' -vvv --su -R mg --ask-su-pass all -m fetch -a "src=/etc/motd dest=/tmp/" &> /tmp/experiment.12290

$ grep error /tmp/experiment.12290
>>>/bin/sh: 1: Syntax error: "(" unexpected
 15264 1447848952.78899: stdout=/bin/sh: 1: Syntax error: "(" unexpected

```

The command itself succeeds despite that error, and I find I have a /tmp/fridge/etc/motd file on my local machine.


Resolving patch:
diff --git a/lib/ansible/plugins/action/fetch.py b/lib/ansible/plugins/action/fetch.py
--- a/lib/ansible/plugins/action/fetch.py
+++ b/lib/ansible/plugins/action/fetch.py
@@ -60,20 +60,22 @@ def run(self, tmp=None, task_vars=None):
         source = self._connection._shell.join_path(source)
         source = self._remote_expand_user(source)
 
-        # calculate checksum for the remote file
-        remote_checksum = self._remote_checksum(source, all_vars=task_vars)
+        remote_checksum = None
+        if not self._play_context.become:
+            # calculate checksum for the remote file, don't bother if using become as slurp will be used
+            remote_checksum = self._remote_checksum(source, all_vars=task_vars)
 
-        # use slurp if sudo and permissions are lacking
+        # use slurp if permissions are lacking or privilege escalation is needed
         remote_data = None
-        if remote_checksum in ('1', '2') or self._play_context.become:
+        if remote_checksum in ('1', '2', None):
             slurpres = self._execute_module(module_name='slurp', module_args=dict(src=source), task_vars=task_vars, tmp=tmp)
             if slurpres.get('failed'):
                 if remote_checksum == '1' and not fail_on_missing:
                     result['msg'] = "the remote file does not exist, not transferring, ignored"
                     result['file'] = source
                     result['changed'] = False
-                    return result
-                result.update(slurpres)
+                else:
+                    result.update(slurpres)
                 return result
             else:
                 if slurpres['encoding'] == 'base64':
@@ -115,8 +117,8 @@ def run(self, tmp=None, task_vars=None):
         dest = dest.replace("//","/")
 
         if remote_checksum in ('0', '1', '2', '3', '4'):
-            # these don't fail because you may want to transfer a log file that possibly MAY exist
-            # but keep going to fetch other log files
+            # these don't fail because you may want to transfer a log file that
+            # possibly MAY exist but keep going to fetch other log files
             if remote_checksum == '0':
                 result['msg'] = "unable to calculate the checksum of the remote file"
                 result['file'] = source
@@ -162,8 +164,7 @@ def run(self, tmp=None, task_vars=None):
                 except (IOError, OSError) as e:
                     raise AnsibleError("Failed to fetch the file: %s" % e)
             new_checksum = secure_hash(dest)
-            # For backwards compatibility.  We'll return None on FIPS enabled
-            # systems
+            # For backwards compatibility. We'll return None on FIPS enabled systems
             try:
                 new_md5 = md5(dest)
             except ValueError:
@@ -171,16 +172,14 @@ def run(self, tmp=None, task_vars=None):
 
             if validate_checksum and new_checksum != remote_checksum:
                 result.update(dict(failed=True, md5sum=new_md5, msg="checksum mismatch", file=source, dest=dest, remote_md5sum=None, checksum=new_checksum, remote_checksum=remote_checksum))
-                return result
-            result.update(dict(changed=True, md5sum=new_md5, dest=dest, remote_md5sum=None, checksum=new_checksum, remote_checksum=remote_checksum))
-            return result
+            else:
+                result.update(dict(changed=True, md5sum=new_md5, dest=dest, remote_md5sum=None, checksum=new_checksum, remote_checksum=remote_checksum))
         else:
-            # For backwards compatibility.  We'll return None on FIPS enabled
-            # systems
+            # For backwards compatibility. We'll return None on FIPS enabled systems
             try:
                 local_md5 = md5(dest)
             except ValueError:
                 local_md5 = None
-
             result.update(dict(changed=False, md5sum=local_md5, file=source, dest=dest, checksum=local_checksum))
-            return result
+
+        return result


Test patch: