Repository: ansible/ansible

Issue:
synchronize module: add support for buildah connection
(I'm creating this issue so that I can coordinate on a proper solution. It's targeted for @chouseknecht @j00bar @gregdek; hence I won't be filing out all the fields)

I made more progress on the integration of buildah inside ansible-container. Unfortunately, it seems that not all ansible modules support different connection methods natively. This is the error I'm getting right now:
 
```
failed: [django] (item=requirements.yml) => {
    "changed": false,
    "item": "requirements.yml",
    "msg": "synchronize uses rsync to function. rsync needs to connect to the remote host via ssh, docker client or a direct filesystem copy. This remote host is being accessed via buildah instead so it cannot work."
}
```

Could you please point me to some code which I need to change in order to make it work?


##### ISSUE TYPE
 - Bug Report

##### COMPONENT NAME
synchronize module

##### ANSIBLE VERSION
```
$ ansible-playbook -vvvv  -i /tmp/tmpG1OY1t/hosts --connection=buildah  /tmp/tmpG1OY1t/playbook.yml
ansible-playbook 2.5.0
  config file = None
  configured module search path = [u'/root/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/lib/python2.7/site-packages/ansible
  executable location = /usr/bin/ansible-playbook
  python version = 2.7.14 (default, Nov  3 2017, 10:55:25) [GCC 7.2.1 20170915 (Red Hat 7.2.1-2)]
No config file found; using defaults
setting up inventory plugins
Parsed /tmp/tmpG1OY1t/hosts inventory source with ini plugin
Loading callback plugin default of type stdout, v2.0 from /usr/lib/python2.7/site-packages/ansible/plugins/callback/__init__.pyc

PLAYBOOK: playbook.yml *********************************************************
1 plays in /tmp/tmpG1OY1t/playbook.yml
```



Discussion:
Files identified in the description:
* [lib/ansible/modules/files/synchronize.py](https://github.com/ansible/ansible/blob/devel/lib/ansible/modules/files/synchronize.py)

If these files are inaccurate, please update the `component name` section of the description or use the `!component` bot command.

[click here for bot help](https://github.com/ansible/ansibullbot/blob/master/ISSUE_HELP.md)
<!--- boilerplate: components_banner --->
cc @tima
[click here for bot help](https://github.com/ansible/ansibullbot/blob/master/ISSUE_HELP.md)
<!--- boilerplate: notify --->
@chouseknecht was so nice and shared with me via IRC that I need to extend https://github.com/ansible/ansible/blob/devel/lib/ansible/plugins/action/synchronize.py in order to make this work; so expect a PR!

Resolving patch:
diff --git a/lib/ansible/plugins/action/synchronize.py b/lib/ansible/plugins/action/synchronize.py
--- a/lib/ansible/plugins/action/synchronize.py
+++ b/lib/ansible/plugins/action/synchronize.py
@@ -59,8 +59,8 @@ def _format_rsync_rsh_target(self, host, path, user):
         if path.startswith('rsync://'):
             return path
 
-        # If using docker, do not add user information
-        if self._remote_transport not in ['docker'] and user:
+        # If using docker or buildah, do not add user information
+        if self._remote_transport not in ['docker', 'buildah'] and user:
             user_prefix = '%s@' % (user, )
 
         if self._host_is_ipv6_address(host):
@@ -188,12 +188,16 @@ def run(self, tmp=None, task_vars=None):
         except (AttributeError, KeyError):
             delegate_to = None
 
-        # ssh paramiko docker and local are fully supported transports.  Anything
+        # ssh paramiko docker buildah and local are fully supported transports.  Anything
         # else only works with delegate_to
-        if delegate_to is None and self._connection.transport not in ('ssh', 'paramiko', 'local', 'docker'):
+        if delegate_to is None and self._connection.transport not in \
+                ('ssh', 'paramiko', 'local', 'docker', 'buildah'):
             result['failed'] = True
-            result['msg'] = ("synchronize uses rsync to function. rsync needs to connect to the remote host via ssh, docker client or a direct filesystem "
-                             "copy. This remote host is being accessed via %s instead so it cannot work." % self._connection.transport)
+            result['msg'] = (
+                "synchronize uses rsync to function. rsync needs to connect to the remote "
+                "host via ssh, docker client or a direct filesystem "
+                "copy. This remote host is being accessed via %s instead "
+                "so it cannot work." % self._connection.transport)
             return result
 
         use_ssh_args = _tmp_args.pop('use_ssh_args', None)
@@ -382,7 +386,7 @@ def run(self, tmp=None, task_vars=None):
 
         # If launching synchronize against docker container
         # use rsync_opts to support container to override rsh options
-        if self._remote_transport in ['docker']:
+        if self._remote_transport in ['docker', 'buildah']:
             # Replicate what we do in the module argumentspec handling for lists
             if not isinstance(_tmp_args.get('rsync_opts'), MutableSequence):
                 tmp_rsync_opts = _tmp_args.get('rsync_opts', [])
@@ -394,12 +398,16 @@ def run(self, tmp=None, task_vars=None):
 
             if '--blocking-io' not in _tmp_args['rsync_opts']:
                 _tmp_args['rsync_opts'].append('--blocking-io')
-            if become and self._play_context.become_user:
-                _tmp_args['rsync_opts'].append("--rsh=%s exec -u %s -i" % (self._docker_cmd, self._play_context.become_user))
-            elif user is not None:
-                _tmp_args['rsync_opts'].append("--rsh=%s exec -u %s -i" % (self._docker_cmd, user))
-            else:
-                _tmp_args['rsync_opts'].append("--rsh=%s exec -i" % self._docker_cmd)
+
+            if self._remote_transport in ['docker']:
+                if become and self._play_context.become_user:
+                    _tmp_args['rsync_opts'].append("--rsh=%s exec -u %s -i" % (self._docker_cmd, self._play_context.become_user))
+                elif user is not None:
+                    _tmp_args['rsync_opts'].append("--rsh=%s exec -u %s -i" % (self._docker_cmd, user))
+                else:
+                    _tmp_args['rsync_opts'].append("--rsh=%s exec -i" % self._docker_cmd)
+            elif self._remote_transport in ['buildah']:
+                _tmp_args['rsync_opts'].append("--rsh=buildah run --")
 
         # run the module and store the result
         result.update(self._execute_module('synchronize', module_args=_tmp_args, task_vars=task_vars))


Test patch:
diff --git a/test/integration/targets/synchronize-buildah/aliases b/test/integration/targets/synchronize-buildah/aliases
new file mode 100644
--- /dev/null
+++ b/test/integration/targets/synchronize-buildah/aliases
@@ -0,0 +1,2 @@
+non_local
+needs/root
diff --git a/test/integration/targets/synchronize-buildah/inventory b/test/integration/targets/synchronize-buildah/inventory
new file mode 100644
--- /dev/null
+++ b/test/integration/targets/synchronize-buildah/inventory
@@ -0,0 +1 @@
+buildah-container    ansible_host=buildah-container ansible_connection=buildah
diff --git a/test/integration/targets/synchronize-buildah/roles/test_buildah_synchronize/files/normal_file.txt b/test/integration/targets/synchronize-buildah/roles/test_buildah_synchronize/files/normal_file.txt
new file mode 100644
--- /dev/null
+++ b/test/integration/targets/synchronize-buildah/roles/test_buildah_synchronize/files/normal_file.txt
@@ -0,0 +1 @@
+abnormal content
diff --git a/test/integration/targets/synchronize-buildah/roles/test_buildah_synchronize/tasks/main.yml b/test/integration/targets/synchronize-buildah/roles/test_buildah_synchronize/tasks/main.yml
new file mode 100644
--- /dev/null
+++ b/test/integration/targets/synchronize-buildah/roles/test_buildah_synchronize/tasks/main.yml
@@ -0,0 +1,66 @@
+# test code for the synchronize module
+# (c) 2014, James Tanner <tanner.jc@gmail.com>
+
+# This file is part of Ansible
+#
+# Ansible is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# Ansible is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with Ansible.  If not, see <http://www.gnu.org/licenses/>.
+
+- name: cleanup old files
+  file:
+    path: '{{ output_dir }}'
+    state: absent
+
+- name: ensure the target directory exists
+  file:
+    path: '{{ output_dir }}'
+    state: directory
+
+- name: synchronize file to new filename
+  synchronize:
+    src: normal_file.txt
+    dest: '{{ output_dir }}/remote_file.txt'
+  register: sync_result
+
+- assert:
+    that:
+        - "'changed' in sync_result"
+        - "sync_result.changed == true"
+        - "'cmd' in sync_result"
+        - "'rsync' in sync_result.cmd"
+        - "'msg' in sync_result"
+        - "sync_result.msg.startswith('<f+')"
+        - "sync_result.msg.endswith('+ normal_file.txt\n')"
+
+- name: test that the file was really copied over
+  stat:
+    path: "{{ output_dir }}/remote_file.txt"
+  register: stat_result
+
+- assert:
+    that:
+      - "stat_result.stat.exists == True"
+      - "stat_result.stat.checksum == '4f11fb5cd9fe0171ea6fab02ae33f65138f3e44e'"
+
+- name: test that the file is not copied a second time
+  synchronize: src=normal_file.txt dest={{output_dir}}/remote_file.txt
+  register: sync_result
+
+- assert:
+    that:
+     - "sync_result.changed == False"
+
+- name: cleanup old files
+  file:
+    path: '{{ output_dir }}'
+    state: absent
diff --git a/test/integration/targets/synchronize-buildah/runme.sh b/test/integration/targets/synchronize-buildah/runme.sh
new file mode 100755
--- /dev/null
+++ b/test/integration/targets/synchronize-buildah/runme.sh
@@ -0,0 +1,15 @@
+#!/usr/bin/env bash
+
+set -ux
+
+CONTAINER_NAME=buildah-container
+
+buildah rm $CONTAINER_NAME >/dev/null 2>/dev/null
+
+set -e
+
+buildah from --name $CONTAINER_NAME docker.io/library/centos:7
+trap '{ buildah rm $CONTAINER_NAME; }' EXIT
+buildah run $CONTAINER_NAME -- yum install -y rsync
+
+ansible-playbook test_synchronize_buildah.yml -c buildah -i inventory -vv
diff --git a/test/integration/targets/synchronize-buildah/test_synchronize_buildah.yml b/test/integration/targets/synchronize-buildah/test_synchronize_buildah.yml
new file mode 100644
--- /dev/null
+++ b/test/integration/targets/synchronize-buildah/test_synchronize_buildah.yml
@@ -0,0 +1,8 @@
+---
+- hosts: buildah-container
+  connection: buildah
+  gather_facts: no
+  vars:
+    output_dir: /tmp/ansible_test_synchronize_buildah
+  roles:
+  - test_buildah_synchronize