Repository: ansible/ansible

Issue:
Using synchronize, docker connection and become fails
##### ISSUE TYPE
 - Bug Report

##### COMPONENT NAME
synchronize

##### ANSIBLE VERSION
```
ansible 2.3.0 (devel 6ec0369c26) last updated 2017/01/11 14:27:03 (GMT +1100)
  config file = /etc/ansible/ansible.cfg
  configured module search path = Default w/o overrides
```

##### CONFIGURATION
N/A

##### OS / ENVIRONMENT
Fedora 24 to Ubuntu 16.04 - but not OS specific.

##### SUMMARY
<!--- Explain the problem briefly -->

##### STEPS TO REPRODUCE
```yaml
- hosts: all
  become: yes
  tasks:
    - synchronize:
        src: ~/test/
        dest: /test

- hosts: all
  become: yes
  become_user: testuser
  tasks:
    - synchronize:
        src: ~/test/
        dest: /home/testuser/test
```
```
docker run -d --name ansible-sync-test ubuntu:xenial sleep infinity
docker exec -i ansible-sync-test apt-get update
docker exec -i ansible-sync-test apt-get install -y sudo python rsync
docker exec -i ansible-sync-test useradd -m testuser
ansible-playbook -i "ansible-sync-test," reproduce.yml -c docker -vvvv
```

##### EXPECTED RESULTS
The first play should synchronize my ~/test directory to /test owned by root and the second should sync it to /home/testuser/test owned by testuser. Both fail today on devel.

##### ACTUAL RESULTS
```
Using /etc/ansible/ansible.cfg as config file
Loading callback plugin default of type stdout, v2.0 from /home/jamie/work/ansible/ansible/lib/ansible/plugins/callback/__init__.pyc

PLAYBOOK: reproduce.yml ********************************************************************************************************************************************************
2 plays in reproduce.yml

PLAY [all] *********************************************************************************************************************************************************************

TASK [Gathering Facts] *********************************************************************************************************************************************************
Using module file /home/jamie/work/ansible/ansible/lib/ansible/modules/system/setup.py
<ansible-sync-test> ESTABLISH DOCKER CONNECTION FOR USER: root
<ansible-sync-test> EXEC ['/usr/bin/docker', 'exec', '-i', u'ansible-sync-test', u'/bin/sh', '-c', u'/bin/sh -c \'( umask 77 && mkdir -p "` echo ~/.ansible/tmp/ansible-tmp-1484108840.86-130865061888096 `" && echo ansible-tmp-1484108840.86-130865061888096="` echo ~/.ansible/tmp/ansible-tmp-1484108840.86-130865061888096 `" ) && sleep 0\'']
<ansible-sync-test> PUT /tmp/tmpHX80Bb TO /root/.ansible/tmp/ansible-tmp-1484108840.86-130865061888096/setup.py
<ansible-sync-test> EXEC ['/usr/bin/docker', 'exec', '-i', u'ansible-sync-test', u'/bin/sh', '-c', u"/bin/sh -c 'chmod u+x /root/.ansible/tmp/ansible-tmp-1484108840.86-130865061888096/ /root/.ansible/tmp/ansible-tmp-1484108840.86-130865061888096/setup.py && sleep 0'"]
<ansible-sync-test> EXEC ['/usr/bin/docker', 'exec', '-i', u'ansible-sync-test', u'/bin/sh', '-c', u'/bin/sh -c \'sudo -H -S -n -u root /bin/sh -c \'"\'"\'echo BECOME-SUCCESS-cbsuucggbcggibzbnwrgxfwalqexznni; /usr/bin/python /root/.ansible/tmp/ansible-tmp-1484108840.86-130865061888096/setup.py; rm -rf "/root/.ansible/tmp/ansible-tmp-1484108840.86-130865061888096/" > /dev/null 2>&1\'"\'"\' && sleep 0\'']
ok: [ansible-sync-test]

TASK [synchronize] *************************************************************************************************************************************************************
task path: /home/jamie/test/reproduce.yml:4
Using module file /home/jamie/work/ansible/ansible/lib/ansible/modules/files/synchronize.py
<ansible-sync-test> ESTABLISH LOCAL CONNECTION FOR USER: jamie
<ansible-sync-test> EXEC /bin/sh -c '( umask 77 && mkdir -p "` echo ~/.ansible/tmp/ansible-tmp-1484108841.96-56189706790348 `" && echo ansible-tmp-1484108841.96-56189706790348="` echo ~/.ansible/tmp/ansible-tmp-1484108841.96-56189706790348 `" ) && sleep 0'
<ansible-sync-test> PUT /tmp/tmpyDZd9E TO /home/jamie/.ansible/tmp/ansible-tmp-1484108841.96-56189706790348/synchronize.py
<ansible-sync-test> EXEC /bin/sh -c 'chmod u+x /home/jamie/.ansible/tmp/ansible-tmp-1484108841.96-56189706790348/ /home/jamie/.ansible/tmp/ansible-tmp-1484108841.96-56189706790348/synchronize.py && sleep 0'
<ansible-sync-test> EXEC /bin/sh -c '/home/jamie/.virtualenvs/ansible/bin/python2 /home/jamie/.ansible/tmp/ansible-tmp-1484108841.96-56189706790348/synchronize.py; rm -rf "/home/jamie/.ansible/tmp/ansible-tmp-1484108841.96-56189706790348/" > /dev/null 2>&1 && sleep 0'
fatal: [ansible-sync-test]: FAILED! => {
    "changed": false, 
    "cmd": "/usr/bin/rsync --delay-updates -F --compress --archive --rsh 'ssh  -S none -o StrictHostKeyChecking=no' --rsync-path=\"sudo rsync\"  --blocking-io --rsh='/usr/bin/docker exec -i' --out-format='<<CHANGED>>%i %n%L' \"/home/jamie/test/\" \"ansible-sync-test:/test\"", 
    "failed": true, 
    "invocation": {
        "module_args": {
            "_local_rsync_path": "rsync", 
            "_substitute_controller": false, 
            "archive": true, 
            "checksum": false, 
            "compress": true, 
            "copy_links": null, 
            "delete": false, 
            "dest": "ansible-sync-test:/test", 
            "dest_port": null, 
            "dirs": false, 
            "existing_only": false, 
            "group": null, 
            "links": null, 
            "mode": "push", 
            "owner": null, 
            "partial": false, 
            "perms": null, 
            "private_key": null, 
            "recursive": null, 
            "rsync_opts": [
                "", 
                "--blocking-io", 
                "--rsh='/usr/bin/docker exec -i'"
            ], 
            "rsync_path": "\"sudo rsync\"", 
            "rsync_timeout": 0, 
            "set_remote_user": true, 
            "src": "/home/jamie/test/", 
            "ssh_args": null, 
            "times": null, 
            "verify_host": false
        }
    }, 
    "msg": "protocol version mismatch -- is your shell clean?\n(see the rsync man page for an explanation)\nrsync error: protocol incompatibility (code 2) at compat.c(178) [sender=3.1.2]\n", 
    "rc": 2
}
	to retry, use: --limit @/home/jamie/test/reproduce.retry

PLAY RECAP *********************************************************************************************************************************************************************
ansible-sync-test          : ok=1    changed=0    unreachable=0    failed=1   
```

If you take just the rsync command that was run there and execute it adding -vvvv you get: 
````
$ /usr/bin/rsync -vvvv --delay-updates -F --compress --archive --rsh 'ssh  -S none -o StrictHostKeyChecking=no' --rsync-path="sudo rsync"  --blocking-io --rsh='/usr/bin/docker exec -i' --out-format='<<CHANGED>>%i %n%L' /home/jamie/test/ ansible-sync-test:/test
cmd=/usr/bin/docker exec -i machine=ansible-sync-test user=<NULL> path=/test
cmd[0]=/usr/bin/docker cmd[1]=exec cmd[2]=-i cmd[3]=ansible-sync-test cmd[4]=sudo rsync cmd[5]=--server cmd[6]=-vvvvlogDtprze.iLsfxC cmd[7]=--log-format=%i cmd[8]=--delay-updates cmd[9]=. cmd[10]=/test 
opening connection using: /usr/bin/docker exec -i ansible-sync-test "sudo rsync" --server -vvvvlogDtprze.iLsfxC "--log-format=%i" --delay-updates . /test  (11 args)
msg checking charset: UTF-8
(Client) Protocol versions: remote=543387762, negotiated=31
protocol version mismatch -- is your shell clean?
(see the rsync man page for an explanation)
[sender] _exit_cleanup(code=2, file=compat.c, line=178): entered
rsync error: protocol incompatibility (code 2) at compat.c(178) [sender=3.1.2]
[sender] _exit_cleanup(code=2, file=compat.c, line=178): about to call exit(2)
````

Protocol version mismatch generally means that the command you executed printed something else and so the connection was unable to be established, so using the "opening connection using" string:

```
$ /usr/bin/docker exec -i ansible-sync-test "sudo rsync" --server -vvvvlogDtprze.iLsfxC "--log-format=%i" --delay-updates . /test 
rpc error: code = 13 desc = invalid header field value "oci runtime error: exec failed: container_linux.go:247: starting container process caused \"exec: \\\"sudo rsync\\\": executable file not found in $PATH\"\n"
```

So rsync appears to be interpretting everything in the rsync-path as a single arg and docker can't parse it within the container. This would seem to be confirmed by `cmd[4]` in the rsync output showing the whole string as 1 argument. If we remove the sudo it works because "rsync" is a valid command. Wait that's weird, how does this work for ssh? 

```
$ ssh 192.168.1.109 "sudo rsync" --version
rsync  version 3.0.9  protocol version 30
```
So you learn something new every day, ssh or my shell is trimming the quotes before executing the command, where docker doesn't. 


Whatever the rationale here, using `--rsync-path "sudo rsync"` doesn't work with docker and it seems to be rsync's fault and we're unlikely to get that behaviour changed so we should fix it on the ansible side. 



Discussion:

Resolving patch:
diff --git a/lib/ansible/plugins/action/synchronize.py b/lib/ansible/plugins/action/synchronize.py
--- a/lib/ansible/plugins/action/synchronize.py
+++ b/lib/ansible/plugins/action/synchronize.py
@@ -331,8 +331,13 @@ def run(self, tmp=None, task_vars=None):
         # Allow custom rsync path argument
         rsync_path = _tmp_args.get('rsync_path', None)
 
+        # backup original become as we are probably about to unset it
+        become = self._play_context.become
+
         if not dest_is_local:
-            if self._play_context.become and not rsync_path:
+            # don't escalate for docker. doing --rsync-path with docker exec fails
+            # and we can switch directly to the user via docker arguments
+            if self._play_context.become and not rsync_path and self._remote_transport != 'docker':
                 # If no rsync_path is set, become was originally set, and dest is
                 # remote then add privilege escalation here.
                 if self._play_context.become_method == 'sudo':
@@ -363,7 +368,9 @@ def run(self, tmp=None, task_vars=None):
                 _tmp_args['rsync_opts'] = self._task.args.get('rsync_opts', '').split(' ')
             if '--blocking-io' not in _tmp_args['rsync_opts']:
                 _tmp_args['rsync_opts'].append('--blocking-io')
-            if user is not None:
+            if become and self._play_context.become_user:
+                _tmp_args['rsync_opts'].append("--rsh='%s exec -u %s -i'" % (self._docker_cmd, self._play_context.become_user))
+            elif user is not None:
                 _tmp_args['rsync_opts'].append("--rsh='%s exec -u %s -i'" % (self._docker_cmd, user))
             else:
                 _tmp_args['rsync_opts'].append("--rsh='%s exec -i'" % self._docker_cmd)


Test patch: