Repository: ansible/ansible

Issue:
delegate_to with synchronize forgets ansible_user
##### ISSUE TYPE
Bug Report

##### COMPONENT NAME
synchronize module

##### ANSIBLE VERSION
```
ansible 2.1.0.0
```

##### CONFIGURATION
host_key_checking = False

##### OS / ENVIRONMENT
Fedora 23

##### SUMMARY
Doing a remote-to-remote push requires using delegate_to. When using synchronize on a host with ansible_user set, and using delegate_to to copy from a second host, Ansible fails to set the remote user for the destination. The remote user is set correctly when delegate_to is removed.

##### STEPS TO REPRODUCE
The task that triggers the bug:

```
      - synchronize:
          src: /home/gemserver/gems/gems
          dest: /var/geminabox/gems/gems
          delete: yes
        delegate_to: "{{ groups['gemserver_master'][0] }}"
```

##### EXPECTED RESULTS
When running with -vvv, the "dest" value should report as "user@host:/var/geminabox/gems/gems",

##### ACTUAL RESULTS
dest is set to just "host:/var/geminabox/gems/gems" (no user@) when delegate_to is present, resulting in being unable to sync.
```
fatal: [111.111.111.111 -> gemserver-master]: FAILED! => {"changed": false, "cmd": "/usr/bin/rsync --delay-updates -F --compress --dry-run --delete-after --archive --rsh 'ssh  -S none -o StrictHostKeyChecking=no' --out-format='<<CHANGED>>%i %n%L' \"/home/gemserver/gems/gems\" \"111.111.111.111:/var/geminabox/gems/gems\"", "failed": true, "invocation": {"module_args": {"_local_rsync_path": "rsync", "_substitute_controller": false, "archive": true, "checksum": false, "compress": true, "copy_links": null, "delete": true, "dest": "111.111.111.111:/var/geminabox/gems/gems", "dest_port": 22, "dirs": false, "existing_only": false, "group": null, "links": null, "mode": "push", "owner": null, "partial": false, "perms": null, "private_key": null, "recursive": null, "rsync_opts": null, "rsync_path": null, "rsync_timeout": 0, "set_remote_user": true, "src": "/home/gemserver/gems/gems", "ssh_args": null, "times": null, "verify_host": false}}, "msg": "Permission denied (publickey,gssapi-keyex,gssapi-with-mic).\r\nrsync: connection unexpectedly closed (0 bytes received so far) [sender]\nrsync error: unexplained error (code 255) at io.c(605) [sender=3.0.9]\n", "rc": 255}
```
(IP address replaced with 111.111.111.111)




Discussion:
Hello,
ansible_user is not taken into account for synchronize -> delegate_to (looks related to closed pull #15024 )
Ansible version 2.0.2.0 on Linux RedHat Enterprise Server 6.5 x86_64.

Snippet of task:
synchronize:
src: "{{ vertica_remote_package_location }}"
dest: "{{ vertica_package_location }}"
mode: pull
delegate_to: vertica-repo
when: not vertica_use_apt and ansible_os_family == 'RedHat' and vertica_remote_rpm

Hosts:
[vertica]
localhost ansible_connection=local

[vertica-repo]
remotehost.net ansible_user=userA ansible_ssh_pass=userAPW

Ansible fails with message: 
<vertica-repo> ESTABLISH CONNECTION FOR USER: **None** on PORT 22 TO vertica-repo
fatal: [localhost]: UNREACHABLE! => {"changed": false, "msg": "[Errno -2] Name or service not known", "unreachable": true}

User is "None" instead of the one specified in hosts file (ie: userA)

Thanks for your help, and don't hesitate to ask if you need more information.
Giovanni

Additionally, `become_user` is not taken into account.  This also affects 2.2-rc1.

I believe this issue results from the following section of [the synchronize action plugin](https://github.com/ansible/ansible/blob/8b2ca79507b1721056913be49810095f97977944/lib/ansible/plugins/action/synchronize.py#L301-L309):

```python
      user = None
            if boolean(_tmp_args.get('set_remote_user', 'yes')):
                if use_delegate:
                    user = task_vars.get('ansible_delegated_vars', dict()).get('ansible_ssh_user', None)
                    if not user:
                        user = C.DEFAULT_REMOTE_USER

                else:
                    user = task_vars.get('ansible_ssh_user') or self._play_context.remote_user
```
The `task_vars["ansible_delegated_vars"]` dictionary appears to contain facts about the _delegate host itself_ (the host specified with `delegate_to`) rather than the destination host, so:

1. If the delegate host doesn't have a value for `ansible_ssh_user`, the `user` variable will end up as `C.DEFAULT_REMOTE_USER`, which -- unless `remote_user` is specified in `ansible.cfg` -- is `None`.  When the value of `user` is none, [_process_remote](https://github.com/ansible/ansible/blob/8b2ca79507b1721056913be49810095f97977944/lib/ansible/plugins/action/synchronize.py#L79-L107)/[_format_rsync_rsh_target](https://github.com/ansible/ansible/blob/8b2ca79507b1721056913be49810095f97977944/lib/ansible/plugins/action/synchronize.py#L53-L68) omit the `<user@>` stanza from the constructed `rsync` target.  I believe this is what the OP saw.
2. If the delegate host has a different value for `ansible_ssh_user` than the destination host (in push mode) or the source host (in pull mode), the rsync invocation will use an inappropriate value for the `<user>@` stanza -- the user for the host running the `rsync` command rather than for the remote host.

I suspect that, for delegated `synchronize` tasks, the action plugin should instead use the value of the source host's `ansible_ssh_user` when in pull mode and the destination host's `ansible_ssh_user` when in push mode.
Seeing this error as well with Ansible 2.2.1 where using synchronize with delegate_to between two inventory hosts will not prefix ``username@`` to the destination.

I've worked around the issue using a pure rsync command with the shell module instead.

I have also encountered this issue. My workaround is to pass user with <code>ssh_args</code>. Eg:
```yaml
- name: Host-to-host synchronize example
  delegate_to: {{ src_hostname }}
  synchronize:
    ssh_args: "-l {{ dest_ssh_user }}"
    src: "..."
    dest: "..."
```
Facing the same issue, trying to synchronize directories from remote to remote using delegate_to
In the target machine, the directories are copied with root user

synchronize:
      src: "<some source>"
      dest: "<some target>"
      recursive: yes
      checksum: yes
      ssh_args: "-l {{ target_user }}"
      rsync_opts:
       - "--include=*"
       - "--include=.*"
  delegate_to: "{{ item }}"
  with_items:
    - "{{groups[ src_host ]}}"
  when: "'{{inventory_hostname}}' == '{{ hostvars[item]['inventory_hostname']}}'"

The files are copied like 

drwxr-xr-x 2 root root 4096 Jul 13 17:57 bin
drwxr-xr-x 3 root root 4096 Jul 13 17:57 conf
drwxrwxr-x 2 root root 4096 Jul 13 17:57 lib

When i use the ssh_args it gives the below error :

"sudo: no tty present and no askpass program specified\nrsync: connection unexpectedly closed (0 bytes received so far) [sender]\nrsync error: error in rsync protocol data stream (code 12) at io.c(605)

cc @tima
[click here for bot help](https://github.com/ansible/ansibullbot/blob/master/ISSUE_HELP.md)
<!--- boilerplate: notify --->
I'm encountering this issue as well. it's really annoying. 
In my below playbook (section) I would expect that on the delgated host the account 'fresh-dbs' should be used. but what happens is the user I'm running the playbook as is used and the whole thing falls down because there is no key copied from the delegated host to the target host. I made sure of that by actually creating a key and copying it, then everything works. (but that's not the goal). 

edit: this is ansible 2.3.1.0

I tried the ssh_args: "-l fresh-dbs" as someone suggested it above, but it doesn't work for me.


```
 - name: copy dump to remote
    become: true
    become_user: fresh-dbs
    synchronize:
      mode: push
      ssh_args: "-l fresh-dbs"
      src: "{{ hostvars['192.168.56.100'].dbdump.stat.path }}"
      dest: /tmp/restore.sql
    delegate_to: 192.168.56.100
    tags: rsync
```


Resolving patch:
diff --git a/lib/ansible/modules/files/synchronize.py b/lib/ansible/modules/files/synchronize.py
--- a/lib/ansible/modules/files/synchronize.py
+++ b/lib/ansible/modules/files/synchronize.py
@@ -467,7 +467,7 @@ def main():
     if is_rsh_needed(source, dest):
         ssh_cmd = [module.get_bin_path('ssh', required=True), '-S', 'none']
         if private_key is not None:
-            ssh_cmd.extend(['-i', private_key])
+            ssh_cmd.extend(['-i', os.path.expanduser(private_key) ])
         # If the user specified a port value
         # Note:  The action plugin takes care of setting this to a port from
         # inventory if the user didn't specify an explicit dest_port
diff --git a/lib/ansible/plugins/action/synchronize.py b/lib/ansible/plugins/action/synchronize.py
--- a/lib/ansible/plugins/action/synchronize.py
+++ b/lib/ansible/plugins/action/synchronize.py
@@ -316,6 +316,8 @@ def run(self, tmp=None, task_vars=None):
             if boolean(_tmp_args.get('set_remote_user', 'yes'), strict=False):
                 if use_delegate:
                     user = task_vars.get('ansible_delegated_vars', dict()).get('ansible_ssh_user', None)
+                    if not user:
+                        user = task_vars.get('ansible_ssh_user') or self._play_context.remote_user
                     if not user:
                         user = C.DEFAULT_REMOTE_USER
 
@@ -326,7 +328,6 @@ def run(self, tmp=None, task_vars=None):
             private_key = self._play_context.private_key_file
 
             if private_key is not None:
-                private_key = os.path.expanduser(private_key)
                 _tmp_args['private_key'] = private_key
 
             # use the mode to define src and dest's url


Test patch:
diff --git a/test/units/plugins/action/fixtures/synchronize/delegate_remote/meta.yaml b/test/units/plugins/action/fixtures/synchronize/delegate_remote/meta.yaml
--- a/test/units/plugins/action/fixtures/synchronize/delegate_remote/meta.yaml
+++ b/test/units/plugins/action/fixtures/synchronize/delegate_remote/meta.yaml
@@ -23,4 +23,4 @@ asserts:
     - "self.execute_called"
     - "self.final_module_args['_local_rsync_path'] == 'rsync'"
     - "self.final_module_args['src'] == '/tmp/deleteme'"
-    - "self.final_module_args['dest'] == 'el6host:/tmp/deleteme'"
+    - "self.final_module_args['dest'] == 'root@el6host:/tmp/deleteme'"
diff --git a/test/units/plugins/action/fixtures/synchronize/delegate_remote_su/meta.yaml b/test/units/plugins/action/fixtures/synchronize/delegate_remote_su/meta.yaml
--- a/test/units/plugins/action/fixtures/synchronize/delegate_remote_su/meta.yaml
+++ b/test/units/plugins/action/fixtures/synchronize/delegate_remote_su/meta.yaml
@@ -30,4 +30,4 @@ asserts:
     - "self.execute_called"
     - "self.final_module_args['_local_rsync_path'] == 'rsync'"
     - "self.final_module_args['src'] == '/tmp/deleteme'"
-    - "self.final_module_args['dest'] == 'el6host:/tmp/deleteme'"
+    - "self.final_module_args['dest'] == 'root@el6host:/tmp/deleteme'"