Repository: ansible/ansible

Issue:
Copy module src path ends with "/" does not copy just the contents
<!---
Verify first that your issue/request is not already reported on GitHub.
Also test if the latest release, and master branch are affected too.
-->

##### ISSUE TYPE
<!--- Pick one below and delete the rest: -->
 - Bug Report


##### COMPONENT NAME
<!--- Name of the module/plugin/task/feature -->
Copy module

##### ANSIBLE VERSION
<!--- Paste verbatim output from “ansible --version” between quotes below -->
```
$ ansible --version
ansible 2.4.0 (devel 0fb64214a4) last updated 2017/07/27 12:32:13 (GMT -500)
  config file = None
  configured module search path = [u'/Users/tanner/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules']
  ansible python module location = /Users/tanner/projects/ansible.git/ansible/lib/ansible
  executable location = /Users/tanner/projects/ansible.git/ansible/bin/ansible
  python version = 2.7.10 (default, Feb  7 2017, 00:08:15) [GCC 4.2.1 Compatible Apple LLVM 8.0.0 (clang-800.0.34)]
```

##### CONFIGURATION
<!---
Mention any settings you have changed/added/removed in ansible.cfg
(or using the ANSIBLE_* environment variables).
-->

##### OS / ENVIRONMENT
<!---
Mention the OS you are running Ansible from, and the OS you are
managing, or say “N/A” for anything that is not platform-specific.
Also mention the specific version of what you are trying to control,
e.g. if this is a network bug the version of firmware on the network device.
-->
Management host macOS 10.12.5
Managed host Ubuntu 16.04

##### SUMMARY
<!--- Explain the problem briefly -->
http://docs.ansible.com/ansible/latest/copy_module.html says "...if [src] path ends with "/", only inside contents of that directory are copied to destination.." It's copying the directory and the contents.

##### STEPS TO REPRODUCE
<!---
For bugs, show exactly how to reproduce the problem, using a minimal test-case.
For new features, show how the feature would be used.
-->

<!--- Paste example playbooks or commands between quotes below -->
```yaml
- name: copy support users skeleton directory
  copy:
    src: "{{ private_data }}/users/tanner/" 
    dest: "/home/tanner"
    owner: "tanner"
    group: "tanner"
```

<!--- You can also paste gist.github.com links for larger files -->
Contents of  "{{ private_data }}/users/tanner/"
```
$ ls -al
total 32
drwxr-xr-x  7 tanner  staff   238 Jul 12 13:25 .
drwxr-xr-x  7 tanner  staff   238 Dec 19  2016 ..
-rw-r--r--  1 tanner  staff  3701 Apr  8  2016 .bashrc
-rw-r--r--  1 tanner  staff     7 Apr  8  2016 .digrc
drwxr-xr-x  4 tanner  staff   136 Apr  8  2016 .ssh
-rw-r--r--  1 tanner  staff   259 Dec 14  2016 .swatchrc
-rw-r--r--  1 tanner  staff   129 Jul 12 13:25 .vimrc
```
Before running Ansible
```
# pwd
/home/tanner
# ls -al
total 8
drwx------ 2 tanner tanner 4096 Jul 28 19:09 .
drwxr-xr-x 6 root   root   4096 Jul 28 19:00 ..
#
```
After running Ansible 
```
# pwd
/home/tanner
# ls -al
total 20
drwx------ 4 tanner tanner 4096 Jul 28 19:10 .
drwxr-xr-x 6 root   root   4096 Jul 28 19:00 ..
drwxr-xr-x 3 tanner tanner 4096 Jul 28 19:10 tanner
```

##### EXPECTED RESULTS
<!--- What did you expect to happen when running the steps above? -->
The **contents** of "{{ private_data }}/users/tanner/" would be copied into  "/home/tanner"

##### ACTUAL RESULTS
<!--- What actually happened? If possible run with extra verbosity (-vvvv) -->
The directory "{{ private_data }}/users/tanner/"  was copied into  "/home/tanner"

<!--- Paste verbatim command output between quotes below -->
```

```



Discussion:
git bisect f86ce0975d8496b61ed6e2206cab917d0779cec9 is the first bad commit
```
commit f86ce0975d8496b61ed6e2206cab917d0779cec9
Author: Toshio Kuratomi <a.badger@gmail.com>
Date:   Mon May 8 22:19:54 2017 -0700

    Add a directory walker to copy

    * We need a directory walker that can handle symlinks, empty directories,
      and some other odd needs.  This commit contains a directory walker that
      can do all that.  The walker returns information about the files in the
      directories that we can then use to implement different strategies for
      copying the files to the remote machines.
    * Add local_follow parameter to copy that follows local symlinks (follow
      is for remote symlinks)
    * Refactor the copying of files out of run into its own method
    * Add new integration tests for copy

    Fixes #24949
    Fixes #21513
```

cc/ @abadger 
Hmm... when I rewrote the copy module, I added an integration test to check that exact condition: https://github.com/ansible/ansible/blob/devel/test/integration/targets/copy/tasks/main.yml#L324

I'll have to take a look at your reproducer to figure out why the integration test isn't testing what it should.
Okay, I haven't figured out why yet, but the relevant difference is that I'm testing with a relative directory and you are using an absolute path.  I have a feeling that one of the functions I'm calling removes the trailing slash on absolute filenames but leaves it on for relative filenames.  Will figure that out and whether it's a bug in the function or if I'll have to detect my trailing slash before hand and restore it after calling that function.
You are correct "{{ private_data }}/users/tanner/" expands to an absolute path

Resolving patch:
diff --git a/lib/ansible/plugins/action/copy.py b/lib/ansible/plugins/action/copy.py
--- a/lib/ansible/plugins/action/copy.py
+++ b/lib/ansible/plugins/action/copy.py
@@ -442,8 +442,14 @@ def run(self, tmp=None, task_vars=None):
         elif remote_src:
             result.update(self._execute_module(task_vars=task_vars))
             return result
-        else:  # find in expected paths
+        else:
+            # find_needle returns a path that may not have a trailing slash on
+            # a directory so we need to determine that now (we use it just
+            # like rsync does to figure out whether to include the directory
+            # or only the files inside the directory
+            trailing_slash = source.endswith(os.path.sep)
             try:
+                # find in expected paths
                 source = self._find_needle('files', source)
             except AnsibleError as e:
                 result['failed'] = True
@@ -451,6 +457,12 @@ def run(self, tmp=None, task_vars=None):
                 result['exception'] = traceback.format_exc()
                 return result
 
+            if trailing_slash != source.endswith(os.path.sep):
+                if source[-1] == os.path.sep:
+                    source = source[:-1]
+                else:
+                    source = source + os.path.sep
+
         # A list of source file tuples (full_path, relative_path) which will try to copy to the destination
         source_files = {'files': [], 'directories': [], 'symlinks': []}
 


Test patch:
diff --git a/test/integration/targets/copy/tasks/main.yml b/test/integration/targets/copy/tasks/main.yml
--- a/test/integration/targets/copy/tasks/main.yml
+++ b/test/integration/targets/copy/tasks/main.yml
@@ -5,6 +5,9 @@
 # GNU General Public License v3 or later (see COPYING or https://www.gnu.org/licenses/gpl-3.0.txt )
 #
 
+- set_fact:
+    output_dir_expanded: '{{ output_dir | expanduser }}'
+
 - name: record the output directory
   set_fact: output_file={{output_dir}}/foo.txt
 
@@ -650,6 +653,83 @@
     that:
       - 'not copy_result.changed'
 
+#
+# Recursive copy with absolute paths (#27439)
+#
+- name: Test that output_dir is appropriate for this test (absolute path)
+  assert:
+    that:
+      - '{{ output_dir_expanded[0] == "/" }}'
+
+- name: create a directory to copy
+  file:
+    path: '{{ output_dir_expanded }}/source_recursive'
+    state: directory
+
+- name: create a file inside of the directory
+  copy:
+    content: "testing"
+    dest: '{{ output_dir_expanded }}/source_recursive/file'
+
+- name: Create a directory to place the test output in
+  file:
+    path: '{{ output_dir_expanded }}/destination'
+    state: directory
+
+- name: Copy the directory and files within (no trailing slash)
+  copy:
+    src: '{{ output_dir_expanded }}/source_recursive'
+    dest: '{{ output_dir_expanded }}/destination'
+
+- name: stat the recursively copied directory
+  stat: path={{output_dir}}/destination/{{item}}
+  register: copied_stat
+  with_items:
+    - "source_recursive"
+    - "source_recursive/file"
+    - "file"
+
+#- debug: var=copied_stat
+- name: assert with no trailing slash, directory and file is copied
+  assert:
+    that:
+      - "copied_stat.results[0].stat.exists"
+      - "copied_stat.results[1].stat.exists"
+      - "not copied_stat.results[2].stat.exists"
+
+- name: Cleanup
+  file:
+    path: '{{ output_dir_expanded }}/destination'
+    state: absent
+
+# Try again with no trailing slash
+
+- name: Create a directory to place the test output in
+  file:
+    path: '{{ output_dir_expanded }}/destination'
+    state: directory
+
+- name: Copy just the files inside of the directory
+  copy:
+    src: '{{ output_dir_expanded }}/source_recursive/'
+    dest: '{{ output_dir_expanded }}/destination'
+
+- name: stat the recursively copied directory
+  stat: path={{output_dir_expanded}}/destination/{{item}}
+  register: copied_stat
+  with_items:
+    - "source_recursive"
+    - "source_recursive/file"
+    - "file"
+
+#- debug: var=copied_stat
+- name: assert with trailing slash, only the file is copied
+  assert:
+    that:
+      - "not copied_stat.results[0].stat.exists"
+      - "not copied_stat.results[1].stat.exists"
+      - "copied_stat.results[2].stat.exists"
+
 #
 # issue 8394
 #
@@ -671,7 +751,7 @@
   assert:
     that:
       - "copy_result6.changed"
-      - "copy_result6.dest == '{{output_dir|expanduser}}/multiline.txt'"
+      - "copy_result6.dest == '{{output_dir_expanded}}/multiline.txt'"
       - "copy_result6.checksum == '9cd0697c6a9ff6689f0afb9136fa62e0b3fee903'"
 
 # test overwriting a file as an unprivileged user (pull request #8624)