Repository: ansible/ansible

Issue:
Ansible Fails to Write File when User has Permissions
##### Issue Type:
- Feature Report
##### Ansible Version:

```
ansible 1.9.4
  configured module search path = /usr/share/ansible/
```
##### Component:
modules/files/copy.py

module_utils/basic.py

##### Ansible Configuration:

The ANSIBLE_CONFIG environmental variable is set:

```
ANSIBLE_CONFIG=/opt/engadmin/.ansible.cfg
```
##### Environment:

Ansible is running on a CentOS 7 system, and is connecting to a RHEL 6 system
##### Summary:

We have a playbook setup to push a template to the /etc directory. The user that Ansible runs as (user1) specifically does _not_ have write permissions to this directory as a whole. It does however have the ability to write to a file within that directory (/etc/php.ini).

When executing a playbook to create this file, the playbook fails:

When stracing the sshd process on the destination host, we see the following:

```
[pid 24871] access("/etc", W_OK)        = -1 EACCES (Permission denied)
```

By my eye, it's checking to see if it can write to the directory, but it fails even when it has the ability to write to the file.
##### Steps To Reproduce:
1. Create playbook to deploy a template for a file in a directory the user Ansible runs as would not have permissions to write _new_ files to (but that can write to an existing file)
2. Execute playbook
##### Expected Results:

I would expect the template to deploy successfully, with a new /etc/php.ini.
##### Actual Results:

Ansible exited, indicating that permission was denied:

```
<testhost> ESTABLISH CONNECTION FOR USER: user1
<testhost> EXEC ssh -C -tt -v -o ControlMaster=auto -o ControlPersist=60s -o ControlPath="/opt/user1/.ansible/cp/ansible-ssh-%h-%p-%r" -o IdentityFile="/opt/engadmin/.ssh/engadmin.pem" -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o ConnectTimeout=10 testhost /bin/sh -c 'mkdir -p /tmp/.ansible-$USER/ansible-tmp-1457975941.18-48691083817511 && chmod a+rx /tmp/.ansible-$USER/ansible-tmp-1457975941.18-48691083817511 && echo /tmp/.ansible-$USER/ansible-tmp-1457975941.18-48691083817511' 
<testhost> EXEC ssh -C -tt -v -o ControlMaster=auto -o ControlPersist=60s -o ControlPath="/opt/user1/.ansible/cp/ansible-ssh-%h-%p-%r" -o IdentityFile="/opt/engadmin/.ssh/engadmin.pem" -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o ConnectTimeout=10 testhost /bin/sh -c 'rc=flag; [ -r /etc/php.ini ] || rc=2; [ -f /etc/php.ini ] || rc=1; [ -d /etc/php.ini ] && rc=3; python -V 2>/dev/null || rc=4; [ x"$rc" != "xflag" ] && echo "${rc} "/etc/php.ini && exit 0; (python -c '"'"'import hashlib; BLOCKSIZE = 65536; hasher = hashlib.sha1(); afile = open("'"'"'/etc/php.ini'"'"'", "rb") buf = afile.read(BLOCKSIZE) while len(buf) > 0: hasher.update(buf) buf = afile.read(BLOCKSIZE) afile.close() print(hasher.hexdigest())'"'"' 2>/dev/null) || (python -c '"'"'import sha; BLOCKSIZE = 65536; hasher = sha.sha(); afile = open("'"'"'/etc/php.ini'"'"'", "rb") buf = afile.read(BLOCKSIZE) while len(buf) > 0: hasher.update(buf) buf = afile.read(BLOCKSIZE) afile.close() print(hasher.hexdigest())'"'"' 2>/dev/null) || (echo '"'"'0 '"'"'/etc/php.ini)'
<testhost> PUT /tmp/tmpDWDjJ0 TO /tmp/.ansible-user1/ansible-tmp-1457975941.18-48691083817511/source
<testhost> PUT /tmp/tmpddzdbK TO /tmp/.ansible-user1/ansible-tmp-1457975941.18-48691083817511/copy
<testhost> EXEC ssh -C -tt -v -o ControlMaster=auto -o ControlPersist=60s -o ControlPath="/opt/user1/.ansible/cp/ansible-ssh-%h-%p-%r" -o IdentityFile="/opt/engadmin/.ssh/engadmin.pem" -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o ConnectTimeout=10 testhost /bin/sh -c 'LANG=C LC_CTYPE=C /usr/bin/python /tmp/.ansible-engadmin/ansible-tmp-1457975941.18-48691083817511/copy; rm -rf /tmp/.ansible-engadmin/ansible-tmp-1457975941.18-48691083817511/ >/dev/null 2>&1'
failed: [testhost] => {"failed": true}
msg: Destination /etc not writable
```



Discussion:
+1

Ansible replaces the file (atomically) to avoid data corruption and race conditions. With this permission setup it would need to write to the file directly as a stream (updating in place), which is prone to cause issues and is the reason we avoid it.

That makes sense, but wouldn't it be possible to simply move the new file (presumably in a temporary location) to the original location instead? Provided that the temporary and destination locations are on the same file system, that should also be an atomic operation, yes? If if they are not the same file system, perhaps Ansible could revert to the current behavior.

The quick-and-dirty alternative that I can think of to get around this is to symlink /etc/phi.ini to a location where our Ansible user could write to the directory of, but this is also a bit of a kludge.

@selble ansible already does that, it tries to move into place from tmpdir, if that does not work it copies file to same directory, which is where it is failing in this case.

@bcoca, thanks. Perhaps I'm being dense, but why is Ansible failing when I can succeed in copying the file from the command line?

```
nw-pweb-d01.qa.example.com(engadmin):/tmp>ll /etc/php.ini
-rw-rw-r-- 1 root phpadmin 68244 Mar 23 12:41 /etc/php.ini
nw-pweb-d01.qa.example.com(engadmin):/tmp>ll /tmp/php.ini.new
-rw-rw-r-- 1 engadmin engadmin 68244 Mar 15 14:38 /tmp/php.ini.new
nw-pweb-d01.qa.example.com(engadmin):/tmp>whoami
engadmin
nw-pweb-d01.qa.example.com(engadmin):/tmp>id engadmin
uid=65000(engadmin) gid=65000(engadmin) groups=65000(engadmin),65001(apacheadmin),65002(phpadmin)
nw-pweb-d01.qa.example.com(engadmin):/tmp>cp /tmp/php.ini.new /etc/php.ini
nw-pweb-d01.qa.example.com(engadmin):/tmp>ll /etc/php.ini
-rw-rw-r-- 1 root phpadmin 68244 Mar 23 12:42 /etc/php.ini
```

Is it also trying to preserve the permissions on the file (which would indeed fail)?

> nw-pweb-d01.qa.example.com(engadmin):/tmp>cp /tmp/php.ini.new /etc/php.ini

At this point you're copying but top make it atomic you have to try moving
the temporary file instead.  That's where the permission setup falls down.

@abadger, right, and it fails on the move, as it should, but I was questioning this part of Brian's reply:

> if that does not work it copies file to same directory, which is where it is failing in this case.

Basically, I was questioning if it couldn't be done atomically, would it revert to doing it in a non-atomic way, which it should be capable of, but doesn't seem to do.

because that can lead to data corruption of the file itself and/or processes that read it.

I get that. I took your prior comment to mean that if the move failed, it'd revert to a copy (non-atomic) operation. I get the concern about data loss, but why not at least offer it as an option, with the appropriate warnings? As it stands now, to give a non-root user the ability to write anything into /etc, we must loosen the permission on /etc as a whole, even if that user would otherwise have permissions to edit an existing file within /etc, and could do so manually. The corruption concern is a perfectly legitimate one, but so is security, and balancing the two here is a bit of a challenge.

it might be an option soon https://github.com/ansible/ansible-modules-core/pull/3015

@selble Greetings! Thanks for taking the time to open this issue. In order for the community to handle your issue effectively, we need a bit more information. 

Here are the items we could not find in your description:
- component name

Please set the description of this issue with this template:
https://raw.githubusercontent.com/ansible/ansible/devel/.github/ISSUE_TEMPLATE.md

[click here for bot help](https://github.com/ansible/ansibullbot/blob/master/ISSUE_HELP.md)
<!--- boilerplate: issue_missing_data --->

Resolving patch:
diff --git a/lib/ansible/module_utils/basic.py b/lib/ansible/module_utils/basic.py
--- a/lib/ansible/module_utils/basic.py
+++ b/lib/ansible/module_utils/basic.py
@@ -2163,58 +2163,67 @@ def atomic_move(self, src, dest, unsafe_writes=False):
                 native_dest_dir = b_dest_dir
                 native_suffix = os.path.basename(b_dest)
                 native_prefix = b('.ansible_tmp')
+                error_msg = None
+                tmp_dest_name = None
                 try:
                     tmp_dest_fd, tmp_dest_name = tempfile.mkstemp( prefix=native_prefix, dir=native_dest_dir, suffix=native_suffix)
                 except (OSError, IOError):
                     e = get_exception()
-                    self.fail_json(msg='The destination directory (%s) is not writable by the current user. Error was: %s' % (os.path.dirname(dest), e))
+                    error_msg = 'The destination directory (%s) is not writable by the current user. Error was: %s' % (os.path.dirname(dest), e)
                 except TypeError:
                     # We expect that this is happening because python3.4.x and
                     # below can't handle byte strings in mkstemp().  Traceback
                     # would end in something like:
                     #     file = _os.path.join(dir, pre + name + suf)
                     # TypeError: can't concat bytes to str
-                    self.fail_json(msg='Failed creating temp file for atomic move.  This usually happens when using Python3 less than Python3.5. '
-                                       'Please use Python2.x or Python3.5 or greater.', exception=traceback.format_exc())
+                    error_msg = ('Failed creating temp file for atomic move.  This usually happens when using Python3 less than Python3.5. '
+                                 'Please use Python2.x or Python3.5 or greater.')
+                finally:
+                    if error_msg:
+                        if unsafe_writes:
+                            self._unsafe_writes(b_src, b_dest)
+                        else:
+                            self.fail_json(msg=error_msg, exception=traceback.format_exc())
 
-                b_tmp_dest_name = to_bytes(tmp_dest_name, errors='surrogate_or_strict')
+                if tmp_dest_name:
+                    b_tmp_dest_name = to_bytes(tmp_dest_name, errors='surrogate_or_strict')
 
-                try:
                     try:
-                        # close tmp file handle before file operations to prevent text file busy errors on vboxfs synced folders (windows host)
-                        os.close(tmp_dest_fd)
-                        # leaves tmp file behind when sudo and not root
-                        try:
-                            shutil.move(b_src, b_tmp_dest_name)
-                        except OSError:
-                            # cleanup will happen by 'rm' of tempdir
-                            # copy2 will preserve some metadata
-                            shutil.copy2(b_src, b_tmp_dest_name)
-
-                        if self.selinux_enabled():
-                            self.set_context_if_different(
-                                b_tmp_dest_name, context, False)
                         try:
-                            tmp_stat = os.stat(b_tmp_dest_name)
-                            if dest_stat and (tmp_stat.st_uid != dest_stat.st_uid or tmp_stat.st_gid != dest_stat.st_gid):
-                                os.chown(b_tmp_dest_name, dest_stat.st_uid, dest_stat.st_gid)
-                        except OSError:
-                            e = get_exception()
-                            if e.errno != errno.EPERM:
-                                raise
-                        try:
-                            os.rename(b_tmp_dest_name, b_dest)
+                            # close tmp file handle before file operations to prevent text file busy errors on vboxfs synced folders (windows host)
+                            os.close(tmp_dest_fd)
+                            # leaves tmp file behind when sudo and not root
+                            try:
+                                shutil.move(b_src, b_tmp_dest_name)
+                            except OSError:
+                                # cleanup will happen by 'rm' of tempdir
+                                # copy2 will preserve some metadata
+                                shutil.copy2(b_src, b_tmp_dest_name)
+
+                            if self.selinux_enabled():
+                                self.set_context_if_different(
+                                    b_tmp_dest_name, context, False)
+                            try:
+                                tmp_stat = os.stat(b_tmp_dest_name)
+                                if dest_stat and (tmp_stat.st_uid != dest_stat.st_uid or tmp_stat.st_gid != dest_stat.st_gid):
+                                    os.chown(b_tmp_dest_name, dest_stat.st_uid, dest_stat.st_gid)
+                            except OSError:
+                                e = get_exception()
+                                if e.errno != errno.EPERM:
+                                    raise
+                            try:
+                                os.rename(b_tmp_dest_name, b_dest)
+                            except (shutil.Error, OSError, IOError):
+                                e = get_exception()
+                                if unsafe_writes and e.errno == errno.EBUSY:
+                                    self._unsafe_writes(b_tmp_dest_name, b_dest)
+                                else:
+                                    self.fail_json(msg='Unable to rename file: %s to %s: %s' % (src, dest, e), exception=traceback.format_exc())
                         except (shutil.Error, OSError, IOError):
                             e = get_exception()
-                            if unsafe_writes and e.errno == errno.EBUSY:
-                                self._unsafe_writes(b_tmp_dest_name, b_dest)
-                            else:
-                                self.fail_json(msg='Unable to rename file: %s to %s: %s' % (src, dest, e), exception=traceback.format_exc())
-                    except (shutil.Error, OSError, IOError):
-                        e = get_exception()
-                        self.fail_json(msg='Failed to replace file: %s to %s: %s' % (src, dest, e), exception=traceback.format_exc())
-                finally:
-                    self.cleanup(b_tmp_dest_name)
+                            self.fail_json(msg='Failed to replace file: %s to %s: %s' % (src, dest, e), exception=traceback.format_exc())
+                    finally:
+                        self.cleanup(b_tmp_dest_name)
 
         if creating:
             # make sure the file has the correct permissions
diff --git a/lib/ansible/modules/files/copy.py b/lib/ansible/modules/files/copy.py
--- a/lib/ansible/modules/files/copy.py
+++ b/lib/ansible/modules/files/copy.py
@@ -343,7 +343,8 @@ def main():
                 if "permission denied" in to_native(e).lower():
                     module.fail_json(msg="Destination directory %s is not accessible" % (os.path.dirname(dest)))
             module.fail_json(msg="Destination directory %s does not exist" % (os.path.dirname(dest)))
-    if not os.access(os.path.dirname(b_dest), os.W_OK):
+
+    if not os.access(os.path.dirname(b_dest), os.W_OK) and not module.params['unsafe_writes']:
         module.fail_json(msg="Destination %s not writable" % (os.path.dirname(dest)))
 
     backup_file = None


Test patch: