Repository: ansible/ansible

Issue:
Feature request: mutex file locking
From @RusDavies on 2016-01-06T15:07:42Z
##### ISSUE TYPE

Feature Idea
##### COMPONENT NAME

file module
##### ANSIBLE VERSION

N/A
##### SUMMARY

Ansible needs mutex file locking on various file-related modules, to prevent race conditions when delegating to a central resource.  

Mutex file locking was the subject of [this pull request](https://github.com/ansible/ansible-modules-core/pull/1902), which targeted only the lineinfile module, and was thus rejected as not being suitably generic.  

Can someone please add the feature_idea label.  

Copied from original issue: ansible/ansible-modules-core#2769



Discussion:
From @mxmo0rhuhn on 2016-01-06T15:07:42Z

+1 This would have saved me lots of work!

From @RusDavies on 2016-01-06T15:07:42Z

@mxmo0rhuhn, in which module(s) would mutex file locking be of greatest use to you?

From @mxmo0rhuhn on 2016-01-06T15:07:42Z

I ran into the problems in the lineinfile module. It is a situation where many hosts have to report into one file on a certain host. So having a "lock: yes" option on it - like you had in your former pull request - would have done the job.

A solution like "serial: 1 per task" (as diskussed [here](https://groups.google.com/forum/?utm_medium=email&utm_source=footer#!msg/ansible-project/rBcWzXjt-Xc/CDW2FdlqRh0J)) would also have been totally sufficient.

From @RusDavies on 2016-01-06T15:07:42Z

@mxmo0rhuhn, unless your deployment is small, then using <code>serial: 1</code> has performance implications. 

If you end up wanting to benefit from a <code>lock: yes</code> option, then you could download my modified version of [lineinfile](https://github.com/RusDavies/ansible-modules-core/blob/devel/files/lineinfile.py), and drop it into a folder called <code>library</code> within your playbook folder. 

From @mxmo0rhuhn on 2016-01-06T15:07:42Z

Yes of course: The `serial: 1` is not my favoured option at all. But at least it would work somehow and basically I had to do the same in the current version of my playbook.

I will discuss downloading the modified version. Thank you very much for your fast feedback.

From @ctx on 2016-01-06T15:07:42Z

Many modules can be affected and `serial: 1` doesn't help always:
Write to a file
- with_items 
- local_action that runs for more than one hosts

For example the known_hosts module shows the problem:
https://github.com/ansible/ansible-modules-extras/issues/2489

Are there other plans/solutions than this mutex feature request?
What about caching the writes instead of locking the file?

From @dsummersl on 2016-01-06T15:07:42Z

As a followup to this, I've found a couple more tickets that would be addressed by this feature:

https://github.com/ansible/ansible-modules-core/issues/467 (cron)
https://github.com/ansible/ansible-modules-core/issues/4542 (authorized_key)

I'm one of the maintainers for the cron module - not sure if I should just close that bug and reference this ticket, or try to address the issue.

I find this to be a big issue.  Ansible is only parallel when it comes to individual tasks or when using async with local runs.  runs, plays and tasks usually happen in order from top to bottom which result in very slow builds. I'm talking hours for something that puppet could handle in minutes. One way to work around this is to run multiple playbooks in parallel. This results in issues with file locking for various modules. 
related #31096

Resolving patch:
diff --git a/lib/ansible/module_utils/common/file.py b/lib/ansible/module_utils/common/file.py
new file mode 100644
--- /dev/null
+++ b/lib/ansible/module_utils/common/file.py
@@ -0,0 +1,114 @@
+# Copyright (c) 2018, Ansible Project
+# Simplified BSD License (see licenses/simplified_bsd.txt or https://opensource.org/licenses/BSD-2-Clause)
+
+import errno
+import os
+import stat
+import re
+import pwd
+import grp
+import time
+import shutil
+import tempfile
+import traceback
+import fcntl
+import sys
+
+from contextlib import contextmanager
+from ansible.module_utils._text import to_bytes, to_native, to_text
+from ansible.module_utils.six import b, binary_type
+
+try:
+    import selinux
+    HAVE_SELINUX = True
+except ImportError:
+    HAVE_SELINUX = False
+
+
+class LockTimeout(Exception):
+    pass
+
+
+class FileLock():
+    '''
+    Currently FileLock is implemented via fcntl.flock on a lock file, however this
+    behaviour may change in the future. Avoid mixing lock types fcntl.flock,
+    fcntl.lockf and module_utils.common.file.FileLock as it will certainly cause
+    unwanted and/or unexpected behaviour
+    '''
+    def __init__(self):
+        self.lockfd = None
+
+    @contextmanager
+    def lock_file(self, path, lock_timeout=None):
+        '''
+        Context for lock acquisition
+        '''
+        try:
+            self.set_lock(path, lock_timeout)
+            yield
+        finally:
+            self.unlock()
+
+    def set_lock(self, path, lock_timeout=None):
+        '''
+        Create a lock file based on path with flock to prevent other processes
+        using given path
+
+        :kw path: Path (file) to lock
+        :kw lock_timeout:
+            Wait n seconds for lock acquisition, fail if timeout is reached.
+            0 = Do not wait, fail if lock cannot be acquired immediately,
+            Default is None, wait indefinitely until lock is released.
+        :returns: True
+        '''
+        tmp_dir = tempfile.gettempdir()
+        lock_path = os.path.join(tmp_dir, 'ansible-{0}.lock'.format(os.path.basename(path)))
+        l_wait = 0.1
+        r_exception = IOError
+        if sys.version_info[0] == 3:
+            r_exception = BlockingIOError
+
+        self.lockfd = open(lock_path, 'w')
+
+        if lock_timeout <= 0:
+            fcntl.flock(self.lockfd, fcntl.LOCK_EX | fcntl.LOCK_NB)
+            os.chmod(lock_path, stat.S_IWRITE | stat.S_IREAD)
+            return True
+
+        if lock_timeout:
+            e_secs = 0
+            while e_secs < lock_timeout:
+                try:
+                    fcntl.flock(self.lockfd, fcntl.LOCK_EX | fcntl.LOCK_NB)
+                    os.chmod(lock_path, stat.S_IWRITE | stat.S_IREAD)
+                    return True
+                except r_exception:
+                    time.sleep(l_wait)
+                    e_secs += l_wait
+                    continue
+
+            self.lockfd.close()
+            raise LockTimeout('{0} sec'.format(lock_timeout))
+
+        fcntl.flock(self.lockfd, fcntl.LOCK_EX)
+        os.chmod(lock_path, stat.S_IWRITE | stat.S_IREAD)
+
+        return True
+
+    def unlock(self):
+        '''
+        Unlock the file descriptor locked by set_lock
+
+        :returns: True
+        '''
+        if not self.lockfd:
+            return True
+
+        try:
+            fcntl.flock(self.lockfd, fcntl.LOCK_UN)
+            self.lockfd.close()
+        except ValueError:  # file wasn't opened, let context manager fail gracefully
+            pass
+
+        return True


Test patch: