Repository: ansible/ansible

Issue:
add new (broken) integration tests for copy and template modules
##### ISSUE TYPE
 - Test Pull Request

##### COMPONENT NAME
copy and template modules

##### ANSIBLE VERSION
```
ansible 2.3.0 (add_breaking_test 6fd0632c15) last updated 2017/03/01 22:14:06 (GMT -700)
  config file = 
  configured module search path = Default w/o overrides
```

##### SUMMARY
Adds tests that demonstrates issues that the copy and template modules have when a file being updated has been the source of hard link. Related to https://github.com/ansible/ansible/issues/10834. These changes are included in https://github.com/ansible/ansible/pull/22028 (where I'm trying to fix the bug).

Run with:
```
test/runner/ansible-test integration copy --docker
test/runner/ansible-test integration template --docker
```

Errors:
```
TASK [copy : copy test file to system 3] ***************************************
fatal: [testhost]: FAILED! => {"changed": false, "checksum": "c79a6506c1c948be0d456ab5104d5e753ab2f3e6", "failed": true, "gid": 0, "group": "root", "mode": "0644", "msg": "src file does not exist, use \"force=yes\" if you really want to create the link: /tmp/10834_test/foo.txt", "owner": "root", "path": "/tmp/10834_test/test_file", "size": 8, "src": "foo.txt", "state": "hard", "uid": 0}
        to retry, use: --limit @/root/ansible/test/integration/copy-bx6L1G.retry
```

```
TASK [template : template out test file to system 3] ***************************
fatal: [testhost]: FAILED! => {"changed": false, "failed": true, "msg": "src and dest are required for creating links"}
        to retry, use: --limit @/root/ansible/test/integration/template-DpjUvO.retry
```

Full Output: https://gist.github.com/aerickson/ed651bd6b18c438d5010962ae4ab0745

copy fails when file exists and has a hardlink
_From @rbu on February 22, 2016 16:30_

##### Issue Type: Bug Report
##### Component Name: 
copy module

##### Ansible Version:

```
2.2.3.0
```
##### Ansible Configuration:

```
[ssh_connection]
pipelining = True

[defaults]
vault_password_file = some_file
hash_behaviour = merge
ansible_managed = some_string
```
##### Environment:

From: OSX/Fedora 23. To: CentOS 7.
##### Summary:

When copying to a file which has hardlinks, ansible fails.
##### Steps To Reproduce:
1. Create a file (with Ansible)
2. Make a hardlink to the file
3. Create/update file (with Ansible)

```
$ ansible -m copy -a 'dest=/some_file content="" owner=root group=root mode="u=r,go="' host -i inventory
host | success >> {
    "changed": true, 
    "checksum": "da39a3ee5e6b4b0d3255bfef95601890afd80709", 
    "dest": "/some_file", 
    "gid": 0, 
    "group": "root", 
    "md5sum": "d41d8cd98f00b204e9800998ecf8427e", 
    "mode": "0400", 
    "owner": "root", 
    "size": 0, 
    "src": "/root/.ansible/tmp/ansible-tmp-1456158236.37-79978273840565/source", 
    "state": "file", 
    "uid": 0
}

$ ssh root@host
root@host / # stat some_file 
  File: ‘some_file’
  Size: 0           Blocks: 0          IO Block: 4096   regular empty file
Device: fd01h/64769d    Inode: 8650932     Links: 1
Access: (0400/-r--------)  Uid: (    0/    root)   Gid: (    0/    root)
Access: 2016-02-22 16:23:56.781407615 +0000
Modify: 2016-02-22 16:23:56.781407615 +0000
Change: 2016-02-22 16:23:57.123407733 +0000
 Birth: -
root@host / # cp -l some_file some_file.bak
root@host / # stat some_file
  File: ‘some_file’
  Size: 0           Blocks: 0          IO Block: 4096   regular empty file
Device: fd01h/64769d    Inode: 8650932     Links: 2
Access: (0400/-r--------)  Uid: (    0/    root)   Gid: (    0/    root)
Access: 2016-02-22 16:23:56.781407615 +0000
Modify: 2016-02-22 16:23:56.781407615 +0000
Change: 2016-02-22 16:24:42.079422952 +0000
 Birth: -
root@host / # exit

$ ansible -m copy -a 'dest=/some_file content="" owner=root group=root mode="u=r,go="' host -i inventory
host | FAILED >> {
    "checksum": "da39a3ee5e6b4b0d3255bfef95601890afd80709", 
    "failed": true, 
    "gid": 0, 
    "group": "root", 
    "mode": "0400", 
    "msg": "src file does not exist, use \"force=yes\" if you really want to create the link: /tmpmH7F1J", 
    "owner": "root", 
    "path": "/some_file", 
    "size": 0, 
    "src": "tmpmH7F1J", 
    "state": "hard", 
    "uid": 0
}
```
##### Expected Results:

File /some_file has content "" and mode set as provided. Link count stays the same OR is reset to 1 (unlinking before).
##### Actual Results:

Error, see above

This may be related to https://github.com/ansible/ansible-modules-core/issues/1568 and https://github.com/ansible/ansible/pull/9806


_Copied from original issue: ansible/ansible-modules-core#3094_
copy module: fix bug dealing with hard linked files
##### ISSUE TYPE
 - Bugfix Pull Request

##### COMPONENT NAME
files module

##### ANSIBLE VERSION
```
ansible 2.2.1.0
  config file = 
  configured module search path = Default w/o overrides
```

Also present on git devel branch.

##### SUMMARY
Fixes https://github.com/ansible/ansible/issues/10834. Pull request with just the tests (broken on devel) is at https://github.com/ansible/ansible/pull/22181.



Discussion:

Resolving patch:
diff --git a/lib/ansible/modules/files/file.py b/lib/ansible/modules/files/file.py
--- a/lib/ansible/modules/files/file.py
+++ b/lib/ansible/modules/files/file.py
@@ -250,6 +250,7 @@ def main():
         if basename:
             params['path'] = path = os.path.join(path, basename)
             b_path = to_bytes(path, errors='surrogate_or_strict')
+            prev_state = get_state(b_path)
 
     # make sure the target path is a directory when we're doing a recursive operation
     if recurse and state != 'directory':
diff --git a/lib/ansible/plugins/action/copy.py b/lib/ansible/plugins/action/copy.py
--- a/lib/ansible/plugins/action/copy.py
+++ b/lib/ansible/plugins/action/copy.py
@@ -331,7 +331,8 @@ def _copy_file(self, source_full, source_rel, content, content_tempfile,
                 dict(
                     src=source_rel,
                     dest=dest,
-                    original_basename=source_rel
+                    original_basename=source_rel,
+                    state='file',
                 )
             )
             if lmode:


Test patch:
diff --git a/test/integration/targets/copy/tasks/tests.yml b/test/integration/targets/copy/tasks/tests.yml
--- a/test/integration/targets/copy/tasks/tests.yml
+++ b/test/integration/targets/copy/tasks/tests.yml
@@ -156,6 +156,87 @@
        - "stat_results.stat.checksum == ('modified'|hash('sha1'))"
        - "stat_results.stat.mode == '0700'"
 
+- name: Create a hardlink to the file
+  file:
+    src: '{{ remote_file }}'
+    dest: '{{ output_dir }}/hard.lnk'
+    state: hard
+
+- name: copy the same contents into place
+  copy:
+    content: 'modified'
+    dest: '{{ remote_file }}'
+    mode: 0700
+  register: copy_results
+
+- name: Check the stat results of the file
+  stat:
+    path: "{{ remote_file }}"
+  register: stat_results
+
+- name: Check the stat results of the hard link
+  stat:
+    path: "{{ output_dir }}/hard.lnk"
+  register: hlink_results
+
+- name: Check that the file did not change
+  assert:
+    that:
+      - 'stat_results.stat.inode == hlink_results.stat.inode'
+      - 'copy_results.changed == False'
+      - "stat_results.stat.checksum == ('modified'|hash('sha1'))"
+
+- name: copy the same contents into place but change mode
+  copy:
+    content: 'modified'
+    dest: '{{ remote_file }}'
+    mode: 0404
+  register: copy_results
+
+- name: Check the stat results of the file
+  stat:
+    path: "{{ remote_file }}"
+  register: stat_results
+
+- name: Check the stat results of the hard link
+  stat:
+    path: "{{ output_dir }}/hard.lnk"
+  register: hlink_results
+
+- name: Check that the file changed permissions but is still the same
+  assert:
+    that:
+      - 'stat_results.stat.inode == hlink_results.stat.inode'
+      - 'copy_results.changed == True'
+      - 'stat_results.stat.mode == hlink_results.stat.mode'
+      - 'stat_results.stat.mode == "0404"'
+      - "stat_results.stat.checksum == ('modified'|hash('sha1'))"
+
+- name: copy the different contents into place
+  copy:
+    content: 'adjusted'
+    dest: '{{ remote_file }}'
+    mode: 0404
+  register: copy_results
+
+- name: Check the stat results of the file
+  stat:
+    path: "{{ remote_file }}"
+  register: stat_results
+
+- name: Check the stat results of the hard link
+  stat:
+    path: "{{ output_dir }}/hard.lnk"
+  register: hlink_results
+
+- name: Check that the file changed and hardlink was broken
+  assert:
+    that:
+      - 'stat_results.stat.inode != hlink_results.stat.inode'
+      - 'copy_results.changed == True'
+      - "stat_results.stat.checksum == ('adjusted'|hash('sha1'))"
+      - "hlink_results.stat.checksum == ('modified'|hash('sha1'))"
+
 - name: Try invalid copy input location fails
   copy:
     src: invalid_file_location_does_not_exist
@@ -1072,6 +1153,34 @@
        - "stat_results.stat.checksum == ('modified'|hash('sha1'))"
        - "not stat_results.stat.islnk"
 
+- name: setup directory for test
+  file: state=directory dest={{remote_dir }}/directory mode=0755
+
+- name: set file mode when the destination is a directory
+  copy: src=foo.txt dest={{remote_dir}}/directory/ mode=0705
+
+- name: set file mode when the destination is a directory
+  copy: src=foo.txt dest={{remote_dir}}/directory/ mode=0604
+  register: file_result
+
+- name: check that the file has the correct attributes
+  stat: path={{ remote_dir }}/directory/foo.txt
+  register: file_attrs
+
+- assert:
+    that:
+      - "file_attrs.stat.uid == 0"
+      - "file_attrs.stat.pw_name == 'root'"
+      - "file_attrs.stat.mode == '0604'"
+
+- name: check that the containing directory did not change attributes
+  stat: path={{ remote_dir }}/directory/
+  register: dir_attrs
+
+- assert:
+    that:
+      - "dir_attrs.stat.mode == '0755'"
+
 #
 # I believe the below section is now covered in the recursive copying section.
 # Hold on for now as an original test case but delete once confirmed that
diff --git a/test/integration/targets/file/tasks/main.yml b/test/integration/targets/file/tasks/main.yml
--- a/test/integration/targets/file/tasks/main.yml
+++ b/test/integration/targets/file/tasks/main.yml
@@ -135,6 +135,51 @@
     that:
       - "file6_touch_result.changed == true"
 
+- name: stat1
+  stat: path={{output_file}}
+  register: hlstat1
+
+- name: stat2
+  stat: path={{output_dir}}/hard.txt
+  register: hlstat2
+
+- name: verify that hard link is still the same after timestamp updated
+  assert:
+    that:
+      - "hlstat1.stat.inode == hlstat2.stat.inode"
+
+- name: create hard link to file 2
+  file: src={{output_file}} dest={{output_dir}}/hard.txt state=hard
+  register: hlink_result
+
+- name: verify that hard link creation is idempotent
+  assert:
+    that:
+      - "hlink_result.changed == False"
+
+- name: Change mode on a hard link
+  file: src={{output_file}} dest={{output_dir}}/hard.txt mode=0701
+  register: file6_mode_change
+
+- name: verify that the hard link was touched
+  assert:
+    that:
+      - "file6_touch_result.changed == true"
+
+- name: stat1
+  stat: path={{output_file}}
+  register: hlstat1
+
+- name: stat2
+  stat: path={{output_dir}}/hard.txt
+  register: hlstat2
+
+- name: verify that hard link is still the same after timestamp updated
+  assert:
+    that:
+      - "hlstat1.stat.inode == hlstat2.stat.inode"
+      - "hlstat1.stat.mode == '0701'"
+
 - name: create a directory
   file: path={{output_dir}}/foobar state=directory
   register: file7_result
diff --git a/test/integration/targets/template/tasks/main.yml b/test/integration/targets/template/tasks/main.yml
--- a/test/integration/targets/template/tasks/main.yml
+++ b/test/integration/targets/template/tasks/main.yml
@@ -306,6 +306,54 @@
     that:
     - "template_result|changed"
 
+#
+# template module can overwrite a file that's been hard linked
+# https://github.com/ansible/ansible/issues/10834
+#
+
+- name: ensure test dir is absent
+  file:
+    path: '{{ output_dir | expanduser }}/hlink_dir'
+    state: absent
+
+- name: create test dir
+  file:
+    path: '{{ output_dir | expanduser }}/hlink_dir'
+    state: directory
+
+- name: template out test file to system 1
+  template:
+    src: foo.j2
+    dest: '{{ output_dir | expanduser }}/hlink_dir/test_file'
+
+- name: make hard link
+  file:
+    src: '{{ output_dir | expanduser }}/hlink_dir/test_file'
+    dest: '{{ output_dir | expanduser }}/hlink_dir/test_file_hlink'
+    state: hard
+
+- name: template out test file to system 2
+  template:
+    src: foo.j2
+    dest: '{{ output_dir | expanduser }}/hlink_dir/test_file'
+  register: hlink_result
+
+- name: check that the files are still hardlinked
+  stat:
+    path: '{{ output_dir | expanduser }}/hlink_dir/test_file'
+  register: orig_file
+
+- name: check that the files are still hardlinked
+  stat:
+    path: '{{ output_dir | expanduser }}/hlink_dir/test_file_hlink'
+  register: hlink_file
+
+# We've done nothing at this point to update the content of the file so it should still be hardlinked
+- assert:
+    that:
+      - "hlink_result.changed == False"
+      - "orig_file.stat.inode == hlink_file.stat.inode"
+
 - name: change var for the template
   set_fact:
     templated_var: "templated_var_loaded"