Repository: ansible/ansible

Issue:
nohup ansible-playbook leaks file descriptors
##### ISSUE TYPE
 - Bug Report

##### COMPONENT NAME
core

##### ANSIBLE VERSION
<!--- Paste verbatim output from “ansible --version” between quotes below -->
```
$ ansible --version
ansible 2.2.2.0
  config file = /home/rdu/sdodson/git/Openshift/openshift-ansible/ansible.cfg
  configured module search path = Default w/o overrides
```

##### CONFIGURATION
Default ansible.cfg

##### OS / ENVIRONMENT
RHEL 7.3


##### SUMMARY
When running ansible-playbook with nohup file descriptors for /dev/null are leaking

##### STEPS TO REPRODUCE
Prepare an inventory with several hosts, we observed this with 10 hosts
nohup ansible-playbook playbooks/byo/config.yml &

##### EXPECTED RESULTS
successful playbook execution immune to SIGHUP

##### ACTUAL RESULTS
files are leaked, once you hit the limits your playbook will abort with an error that may or may not point towards leaked files. Running inside of screen/tmux rather than nohup avoids this problem but we just had a customer do this and it was very difficult to debug.


Discussion:
@jctanner FYI
https://asciinema.org/a/50453 also demonstrates this
Tried to reproduce locally with no success, can you give a simple reproducer (using stock ansible, not sure what openstack-ansible is).
Yeah, we just figured out a simple reproducer, assuming your limit is 1024 this should blow up before hitting the last task

https://gist.github.com/sdodson/2a289411af0d3df18859fb2c67e14621

```
$ nohup ansible-playbook nohup.yml -i localhost -vvvvv

TASK [debug] *******************************************************************
task path: /home/rdu/sdodson/nohup.yml:1018
An exception occurred during task execution. The full traceback is:
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/ansible/executor/task_executor.py", line 126, in run
    res = self._execute()
  File "/usr/lib/python2.7/site-packages/ansible/executor/task_executor.py", line 462, in _execute
    self._connection = self._get_connection(variables=variables, templar=templar)
  File "/usr/lib/python2.7/site-packages/ansible/executor/task_executor.py", line 708, in _get_connection
    connection = self._shared_loader_obj.connection_loader.get(conn_type, self._play_context, self._new_stdin)
  File "/usr/lib/python2.7/site-packages/ansible/plugins/__init__.py", line 358, in get
    obj = obj(*args, **kwargs)
  File "/usr/lib/python2.7/site-packages/ansible/plugins/connection/__init__.py", line 94, in __init__
    for shell in shell_loader.all():
  File "/usr/lib/python2.7/site-packages/ansible/plugins/__init__.py", line 394, in all
    self._module_cache[path] = self._load_module_source(name, path)
  File "/usr/lib/python2.7/site-packages/ansible/plugins/__init__.py", line 324, in _load_module_source
    module = imp.load_source(name, path, module_file)
  File "/usr/lib/python2.7/site-packages/ansible/plugins/shell/fish.py", line 21, in <module>
    from ansible.plugins.shell.sh import ShellModule as ShModule
ImportError: No module named sh
fatal: [localhost]: FAILED! => {
    "failed": true, 
    "msg": "Unexpected failure during module execution.", 
    "stdout": ""
}
        to retry, use: --limit @/home/rdu/sdodson/nohup.retry

PLAY RECAP *********************************************************************
localhost                  : ok=1010 changed=0    unreachable=0    failed=1
```

To monitor open files
`watch -n .5 'lsof /dev/null | grep ansible- | wc -l'`
with ulimit 100
```
fatal: [localhost]: FAILED! => {
    "failed": true,
    "msg": "template error while templating string: unexpected char u'?' at 9. String: {{Success?}}"
}
```
Python version and OS? We've reproduced this on RHEL 7.3 w/ Python 2.7.5 and Fedora w/ 2.7.13
2.7.13 gentoo, i can get an import error but only after ulimit < 100 and adding more tasks.

the 'fun' part is that depending on which tasks get added, i get diff import error
I was able to reproduce on Fedora 25 and Ubuntu trusty and xenial.

Of note here, is that I decided to test on ansible 2.2.2, 2.1.5, and 2.0.2, and it appears there too.  It does not appear in 1.9.6, or at least the watch doesn't catch it in 1.9.6.
https://gist.github.com/bcoca/9426f1a5af589aaece956548cd46d56b <= try this patch
That does seem to limit it for me, from seeing 50-150 before using nohup, I saw a max of 3 with that patch.

It's still 2 higher than without nohup, which generally sits at 0 or 1.
expected, the issue was the stdin cloning in workers, nohup doesn't allow closing refs on it (which it redirected from /dev/null) this patch just lets the workers open their own /dev/null and shoudl also avoid the issue in a 'nicer' way.
https://gist.github.com/bcoca/b12023271aeb87e7cbaabb1034130f6f

Resolving patch:
diff --git a/lib/ansible/executor/process/worker.py b/lib/ansible/executor/process/worker.py
--- a/lib/ansible/executor/process/worker.py
+++ b/lib/ansible/executor/process/worker.py
@@ -68,21 +68,25 @@ def __init__(self, rslt_q, task_vars, host, task, play_context, loader, variable
         self._variable_manager  = variable_manager
         self._shared_loader_obj = shared_loader_obj
 
-        # dupe stdin, if we have one
-        self._new_stdin = sys.stdin
-        try:
-            fileno = sys.stdin.fileno()
-            if fileno is not None:
-                try:
-                    self._new_stdin = os.fdopen(os.dup(fileno))
-                except OSError:
-                    # couldn't dupe stdin, most likely because it's
-                    # not a valid file descriptor, so we just rely on
-                    # using the one that was passed in
-                    pass
-        except (AttributeError, ValueError):
-            # couldn't get stdin's fileno, so we just carry on
-            pass
+        if sys.stdin.isatty():
+            # dupe stdin, if we have one
+            self._new_stdin = sys.stdin
+            try:
+                fileno = sys.stdin.fileno()
+                if fileno is not None:
+                    try:
+                        self._new_stdin = os.fdopen(os.dup(fileno))
+                    except OSError:
+                        # couldn't dupe stdin, most likely because it's
+                        # not a valid file descriptor, so we just rely on
+                        # using the one that was passed in
+                        pass
+            except (AttributeError, ValueError):
+                # couldn't get stdin's fileno, so we just carry on
+                pass
+        else:
+            # set to /dev/null
+            self._new_stdin = os.devnull
 
     def run(self):
         '''
diff --git a/lib/ansible/plugins/action/pause.py b/lib/ansible/plugins/action/pause.py
--- a/lib/ansible/plugins/action/pause.py
+++ b/lib/ansible/plugins/action/pause.py
@@ -125,8 +125,9 @@ def run(self, tmp=None, task_vars=None):
             fd = None
             try:
                 fd = self._connection._new_stdin.fileno()
-            except ValueError:
-                # someone is using a closed file descriptor as stdin
+            except (ValueError, AttributeError):
+                # ValueError: someone is using a closed file descriptor as stdin
+                # AttributeError: someone is using a null file descriptor as stdin on windoez
                 pass
             if fd is not None:
                 if isatty(fd):


Test patch: