Repository: ansible/ansible

Issue:
Recursive permissions change on a directory fails when directory contains a broken symlink
##### ISSUE TYPE
 - Bug Report

##### COMPONENT NAME
file

##### ANSIBLE VERSION

```
ansible 2.5.2
  config file = /etc/ansible/ansible.cfg
  configured module search path = [u'/users/zdw/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/lib/python2.7/dist-packages/ansible
  executable location = /usr/bin/ansible
  python version = 2.7.12 (default, Dec  4 2017, 14:50:18) [GCC 5.4.0 20160609]
```

##### CONFIGURATION
N/A

##### OS / ENVIRONMENT
N/A

##### SUMMARY

The file module in 2.5.2 fails when trying to change permissions recursively and encounters a symlink that points to something that that doesn't exist. 

This error also occurs in the devel 2.6.0 branch.

This failure does not occur in 2.5.1 and earlier.

##### STEPS TO REPRODUCE

Example playbook that shows the problem:

```yaml
---
- hosts: localhost
  connection: local

  tasks:
    - name: Create a directory for testing
      file:
        path: "{{ playbook_dir }}/test"
        state: directory

    - name: Create a symlink from a nonexistant file
      command: "ln -s /doesnt_exist {{ playbook_dir }}/test/doesnt_exist"

    - name: Attempt to change ownership on directory
      file:
        path: "{{ playbook_dir }}/test"
        state: directory
        recurse: yes
        owner: "{{ ansible_user_id }}"
        group: "{{ ansible_user_gid }}"
```

##### EXPECTED RESULTS

From Ansible 2.5.1:

```
$ ansible-playbook -i localhost, symlink_to_nowhere-playbook.yaml 

PLAY [localhost] *****************************************************************

TASK [Gathering Facts] ***********************************************************
ok: [localhost]

TASK [Create a directory for testing] ********************************************
changed: [localhost]

TASK [Create a symlink from a nonexistant file] **********************************
 [WARNING]: Consider using the file module with state=link rather than running
ln.  If you need to use command because file is insufficient you can add
warn=False to this command task or set command_warnings=False in ansible.cfg to
get rid of this message.

changed: [localhost]

TASK [Attempt to change ownership on directory] **********************************
ok: [localhost]

PLAY RECAP ***********************************************************************
localhost                  : ok=4    changed=2    unreachable=0    failed=0   
```
##### ACTUAL RESULTS

Error in 2.5.2 when trying to recursively set permissions:

```
$ ansible-playbook -i localhost, symlink_to_nowhere-playbook.yaml 

PLAY [localhost] *****************************************************************

TASK [Gathering Facts] ***********************************************************
ok: [localhost]

TASK [Create a directory for testing] ********************************************
changed: [localhost]

TASK [Create a symlink from a nonexistant file] **********************************
 [WARNING]: Consider using the file module with state=link rather than running
ln.  If you need to use command because file is insufficient you can add
warn=False to this command task or set command_warnings=False in ansible.cfg to
get rid of this message.

changed: [localhost]

TASK [Attempt to change ownership on directory] **********************************
An exception occurred during task execution. To see the full traceback, use -vvv. The error was: OSError: [Errno 2] No such file or directory: '/doesnt_exist'
fatal: [localhost]: FAILED! => {"changed": false, "module_stderr": "Traceback (most recent call last):\n  File \"/tmp/ansible_ZOFwo4/ansible_module_file.py\", line 473, in <module>\n    main()\n  File \"/tmp/ansible_ZOFwo4/ansible_module_file.py\", line 350, in main\n    changed |= recursive_set_attributes(module, to_bytes(file_args['path'], errors='surrogate_or_strict'), follow, file_args)\n  File \"/tmp/ansible_ZOFwo4/ansible_module_file.py\", line 172, in recursive_set_attributes\n    changed |= module.set_fs_attributes_if_different(tmp_file_args, changed, expand=False)\n  File \"/tmp/ansible_ZOFwo4/ansible_modlib.zip/ansible/module_utils/basic.py\", line 1507, in set_fs_attributes_if_different\n  File \"/tmp/ansible_ZOFwo4/ansible_modlib.zip/ansible/module_utils/basic.py\", line 1164, in set_owner_if_different\n  File \"/tmp/ansible_ZOFwo4/ansible_modlib.zip/ansible/module_utils/basic.py\", line 1059, in user_and_group\nOSError: [Errno 2] No such file or directory: '/doesnt_exist'\n", "module_stdout": "", "msg": "MODULE FAILURE", "rc": 1}
        to retry, use: --limit @/users/zdw/foo/symlink_to_nowhere-playbook.retry

PLAY RECAP ***********************************************************************
localhost                  : ok=3    changed=2    unreachable=0    failed=1   
```



Discussion:
Files identified in the description:
* [lib/ansible/modules/files/file.py](https://github.com/ansible/ansible/blob/devel/lib/ansible/modules/files/file.py)

If these files are inaccurate, please update the `component name` section of the description or use the `!component` bot command.

[click here for bot help](https://github.com/ansible/ansibullbot/blob/master/ISSUE_HELP.md)
<!--- boilerplate: components_banner --->
This change seems to be the only non-documentation to the `file` module between 2.5.1 and 2.5.2:

https://github.com/ansible/ansible/commit/cd7042857c5ed81223080c51fde1d597a2eeebca

cc @abadger 
Adding `follow: false` to the recursive owner/group change gives the previous behavior.

The interaction between `recurse` and `follow` should probably be made more clear - if you symlinked to a system file or directory, trying to chown/chgroup it as another user would at best fail like this, or at worst apply inappropriate permissions to a system file if running with superuser perms.
The default value of follow changed from False to True which is why this is appearing now.  I'm debating whether this is a bug in the interaction of recurse and follow and lean towards yes.  Reasoning is the following behaviour of chmod and chmod -R:
```
pts/1@peru /var/tmp]$ ln -s /nonexistent test                                                            (07:19:34)
[pts/1@peru /var/tmp]$ chmod a+rwx -R test                                                                   (07:19:41)
chmod: cannot operate on dangling symlink 'test'
[pts/1@peru /var/tmp]$ echo $?                                                                            (07:19:46)
1
[pts/1@peru /var/tmp]$ mkdir dir                                                                          (07:19:53)
[pts/1@peru /var/tmp]$ mv test dir                                                                        (07:19:59)
[pts/1@peru /var/tmp]$ chmod -R a+x dir                                                                   (07:20:03)
[pts/1@peru /var/tmp]$ echo $?                                                                            (07:20:10)
0
```

So if we mirror what chmod does, if path is a broken link, we should error.  If path is a directory and we encounter the broken link through recursion, then the command as a whole should succeed.  I'll run this by other core committers for their input.


Resolving patch:
diff --git a/lib/ansible/modules/files/file.py b/lib/ansible/modules/files/file.py
--- a/lib/ansible/modules/files/file.py
+++ b/lib/ansible/modules/files/file.py
@@ -161,16 +161,23 @@ def recursive_set_attributes(module, b_path, follow, file_args):
                 tmp_file_args['path'] = to_native(b_fsname, errors='surrogate_or_strict')
                 changed |= module.set_fs_attributes_if_different(tmp_file_args, changed, expand=False)
             else:
+                # Change perms on the link
                 tmp_file_args = file_args.copy()
                 tmp_file_args['path'] = to_native(b_fsname, errors='surrogate_or_strict')
                 changed |= module.set_fs_attributes_if_different(tmp_file_args, changed, expand=False)
+
                 if follow:
                     b_fsname = os.path.join(b_root, os.readlink(b_fsname))
-                    if os.path.isdir(b_fsname):
-                        changed |= recursive_set_attributes(module, b_fsname, follow, file_args)
-                    tmp_file_args = file_args.copy()
-                    tmp_file_args['path'] = to_native(b_fsname, errors='surrogate_or_strict')
-                    changed |= module.set_fs_attributes_if_different(tmp_file_args, changed, expand=False)
+                    # The link target could be nonexistent
+                    if os.path.exists(b_fsname):
+                        if os.path.isdir(b_fsname):
+                            # Link is a directory so change perms on the directory's contents
+                            changed |= recursive_set_attributes(module, b_fsname, follow, file_args)
+
+                        # Change perms on the file pointed to by the link
+                        tmp_file_args = file_args.copy()
+                        tmp_file_args['path'] = to_native(b_fsname, errors='surrogate_or_strict')
+                        changed |= module.set_fs_attributes_if_different(tmp_file_args, changed, expand=False)
     return changed
 
 


Test patch:
diff --git a/test/integration/targets/file/tasks/main.yml b/test/integration/targets/file/tasks/main.yml
--- a/test/integration/targets/file/tasks/main.yml
+++ b/test/integration/targets/file/tasks/main.yml
@@ -627,6 +627,9 @@
 - name: create a symlink to the directory
   file: path={{output_dir}}/test_follow_rec/test_link_dir state=link src="../test_follow_rec_target_dir"
 
+- name: create a symlink to a nonexistent file
+  file: path={{output_dir}}/test_follow_rec/nonexistent state=link src=does_not_exist force=True
+
 - name: try to change permissions without following symlinks
   file: path={{output_dir}}/test_follow_rec follow=False mode="a-x" recurse=True