Repository: ansible/ansible

Issue:
atomic_move inside docker fails for /etc/hosts
When running ansible inside a docker container, ansible will fail if we try to change some systems file, such as /etc/hosts.

This issue is related to this docker issue : https://github.com/docker/docker/issues/9295

You can reproduce with the following code : https://github.com/pgrange/ansible_docker_resource_busy

I wrote an uggly, untested, patch just to validate an alternative that would consist of overwriting the content of the target file instead of renaming or copying. This patch is available here : https://github.com/pgrange/ansible/commit/844eb0bfea04473719e195fcad73a515516f1df6



Discussion:
Basically docker says to not do this, to use their command line options.

@pgrange I'm not liking the solution ... but it has nothing to do with your code, its the artificial limitation added at the source ...

I would consider the patch, but if we add an update_atomically=yes|no option to trigger it.

@bcoca sorry for the delay and thank you for your answer.

I forgot to explain my use case. First of all, I do not use ansible to configure a docker container. But I use a docker container to validate an ansible role before applying it on actual servers.

The idea here is to write ansible roles in a kind of TDD way. Running the role on the container, then asserting some states on this container to ensure role behaves as expected.

Is docker appropriate to do that is an open question but the fact is, exept for this issue, it is working quite well.

I make this precision so that you can understand that I do not want to tweak my ansible role so that it can run inside a docker container. Since the container is just a mean to test the role that will be executed on actual production servers.

I need some clarification on your suggestion. Do you suggest to introduce a global configuration option "update_atomically=yes|no" that one can set in a ansible.cfg file for instance and that will apply to every module that relies on update_atomically ? Or do your suggest to add such an option directly to the lineinfile module for instance ?

The first option could solve my issue. The second, not so much. Let me know which option you suggest.

Just so you know, I took inspiration in the patch that has been added to handle errno 26 (text file busy) which happens on vagrant synced folders:

https://github.com/ansible/ansible/commit/013c4631e3a65035471d85aabd9227c0fa701e10

And I think both issues compare quite well. Here, we introduce a patch to deal with errno.EBUSY for docker instead of errno 26 for vagrant. OK, the way we work around the issue is different (and the patch is crap ;)).

Thanks,
Pascal.

so did this 70ac47ae61eb3c0cb2e33a342128ef1a7653b17b based on @pgrange's code (py2.4/py3 compatibility sucks)

i'm thinking of adding a 'unsafe_writes=no|yes' with a very strongly worded description to the modules that operate on files to enable this .... still, this feels like working around with a bad hack across a purposefully broken implementation.


Resolving patch:
diff --git a/lib/ansible/module_utils/basic.py b/lib/ansible/module_utils/basic.py
--- a/lib/ansible/module_utils/basic.py
+++ b/lib/ansible/module_utils/basic.py
@@ -1941,7 +1941,7 @@ def atomic_move(self, src, dest, unsafe_writes=False):
             os.rename(b_src, b_dest)
         except (IOError, OSError):
             e = get_exception()
-            if e.errno not in [errno.EPERM, errno.EXDEV, errno.EACCES, errno.ETXTBSY]:
+            if e.errno not in [errno.EPERM, errno.EXDEV, errno.EACCES, errno.ETXTBSY, errno.EBUSY]:
                 # only try workarounds for errno 18 (cross device), 1 (not permitted),  13 (permission denied)
                 # and 26 (text file busy) which happens on vagrant synced folders and other 'exotic' non posix file systems
                 self.fail_json(msg='Could not replace file: %s to %s: %s' % (src, dest, e))
@@ -1983,29 +1983,15 @@ def atomic_move(self, src, dest, unsafe_writes=False):
                             e = get_exception()
                             if e.errno != errno.EPERM:
                                 raise
-                        os.rename(b_tmp_dest_name, b_dest)
+                        try:
+                            os.rename(b_tmp_dest_name, b_dest)
+                        except (shutil.Error, OSError, IOError):
+                            if unsafe_writes:
+                                self._unsafe_writes(b_tmp_dest_name, b_dest, get_exception())
+                            else:
+                                self.fail_json(msg='Could not replace file: %s to %s: %s' % (src, dest, exception))
                     except (shutil.Error, OSError, IOError):
-                        e = get_exception()
-                        # sadly there are some situations where we cannot ensure atomicity, but only if
-                        # the user insists and we get the appropriate error we update the file unsafely
-                        if unsafe_writes and e.errno == errno.EBUSY:
-                            #TODO: issue warning that this is an unsafe operation, but doing it cause user insists
-                            try:
-                                try:
-                                    out_dest = open(b_dest, 'wb')
-                                    in_src = open(b_src, 'rb')
-                                    shutil.copyfileobj(in_src, out_dest)
-                                finally: # assuring closed files in 2.4 compatible way
-                                    if out_dest:
-                                        out_dest.close()
-                                    if in_src:
-                                        in_src.close()
-                            except (shutil.Error, OSError, IOError):
-                                e = get_exception()
-                                self.fail_json(msg='Could not write data to file (%s) from (%s): %s' % (dest, src, e))
-
-                        else:
-                            self.fail_json(msg='Could not replace file: %s to %s: %s' % (src, dest, e))
+                        self.fail_json(msg='Could not replace file: %s to %s: %s' % (src, dest, exception))
                 finally:
                     self.cleanup(b_tmp_dest_name)
 
@@ -2022,6 +2008,28 @@ def atomic_move(self, src, dest, unsafe_writes=False):
             # rename might not preserve context
             self.set_context_if_different(dest, context, False)
 
+    def _unsafe_writes(self, src, dest, exception):
+      # sadly there are some situations where we cannot ensure atomicity, but only if
+      # the user insists and we get the appropriate error we update the file unsafely
+      if exception.errno == errno.EBUSY:
+          #TODO: issue warning that this is an unsafe operation, but doing it cause user insists
+          try:
+              try:
+                  out_dest = open(dest, 'wb')
+                  in_src = open(src, 'rb')
+                  shutil.copyfileobj(in_src, out_dest)
+              finally: # assuring closed files in 2.4 compatible way
+                  if out_dest:
+                      out_dest.close()
+                  if in_src:
+                      in_src.close()
+          except (shutil.Error, OSError, IOError):
+              e = get_exception()
+              self.fail_json(msg='Could not write data to file (%s) from (%s): %s' % (dest, src, e))
+      
+      else:
+          self.fail_json(msg='Could not replace file: %s to %s: %s' % (src, dest, exception))
+
     def run_command(self, args, check_rc=False, close_fds=True, executable=None, data=None, binary_data=False, path_prefix=None, cwd=None, use_unsafe_shell=False, prompt_regex=None, environ_update=None):
         '''
         Execute a command, returns rc, stdout, and stderr.


Test patch: