Repository: ansible/ansible

Issue:
ERROR! Unexpected Exception, this is probably a bug: [Errno 38] Function not implemented
<!---
Verify first that your issue/request is not already reported on GitHub.
THIS FORM WILL BE READ BY A MACHINE, COMPLETE ALL SECTIONS AS DESCRIBED.
Also test if the latest release, and devel branch are affected too.
ALWAYS add information AFTER (OUTSIDE) these html comments.
Otherwise it may end up being automatically closed by our bot. -->

##### SUMMARY
When trying to run as a Debian post-install process `ansible-playbook` dies with a confusing error message because `/dev/shm` is missing. I'm happy to file the bug upstream against the multiprocessing module if that makes more sense.
<!--- Explain the problem briefly -->

##### ISSUE TYPE
 - Bug Report

##### COMPONENT NAME
<!--- Insert, BELOW THIS COMMENT, the name of the module, plugin, task or feature.
Do not include extra details here, e.g. "vyos_command" not "the network module vyos_command" or the full path-->
ansible-playbook

##### ANSIBLE VERSION
<!--- Paste, BELOW THIS COMMENT, verbatim output from "ansible --version" between quotes below -->
```
ansible 2.6.1
  config file = None
  configured module search path = [u'/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/local/lib/python2.7/dist-packages/ansible
  executable location = /usr/local/bin/ansible
  python version = 2.7.13 (default, Nov 24 2017, 17:33:09) [GCC 6.3.0 20170516]
```

##### CONFIGURATION
<!--- If using Ansible 2.4 or above, paste, BELOW THIS COMMENT, the results of "ansible-config dump --only-changed"
Otherwise, mention any settings you have changed/added/removed in ansible.cfg
(or using the ANSIBLE_* environment variables).-->
n/a

##### OS / ENVIRONMENT
<!--- Mention, BELOW THIS COMMENT, the OS you are running Ansible from, and the OS you are
managing, or say "N/A" for anything that is not platform-specific.
Also mention the specific version of what you are trying to control,
e.g. if this is a network bug the version of firmware on the network device.-->
root@host:/tmp/ansible# uname -a
Linux port-01 4.9.0-6-amd64 #1 SMP Debian 4.9.82-1+deb9u3 (2018-03-02) x86_64 GNU/Linux
root@host:/tmp/ansible# more /etc/os-release 
PRETTY_NAME="Debian GNU/Linux 9 (stretch)"
NAME="Debian GNU/Linux"
VERSION_ID="9"
VERSION="9 (stretch)"
ID=debian
HOME_URL="https://www.debian.org/"
SUPPORT_URL="https://www.debian.org/support"
BUG_REPORT_URL="https://bugs.debian.org/"

This is a chrooted environment at the end of the debian install process
ie. `/usr/lib/finish-install.d/`

##### STEPS TO REPRODUCE
<!--- For bugs, show exactly how to reproduce the problem, using a minimal test-case.
For new features, show how the feature would be used. -->
This is a simpler [example of the problem](https://esolitos.com/items/2018/03/ansible-alpine-probably-bug) and the resolution when using ansible with alpine

<!--- Paste example playbooks or commands between quotes below -->
```yaml

```

<!--- You can also paste gist.github.com links for larger files -->

##### EXPECTED RESULTS
<!--- What did you expect to happen when running the steps above? -->
Check that `/dev/shm` exists before calling `multiprocessing/synchronize.py` and output an error message that mentions `/dev/shm`.

##### ACTUAL RESULTS
<!--- What actually happened? If possible run with extra verbosity (-vvvv) -->

<!--- Paste verbatim command output between quotes below -->
```
# ansible-playbook -vvv -i'localhost,' playbook_build.yml
ansible-playbook 2.6.1
  config file = None
  configured module search path = [u'/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/local/lib/python2.7/dist-packages/ansible
  executable location = /usr/local/bin/ansible-playbook
  python version = 2.7.13 (default, Nov 24 2017, 17:33:09) [GCC 6.3.0 20170516]
No config file found; using defaults
Parsed localhost, inventory source with host_list plugin
ERROR! Unexpected Exception, this is probably a bug: [Errno 38] Function not implemented
the full traceback was:

Traceback (most recent call last):
  File "/usr/local/bin/ansible-playbook", line 118, in <module>
    exit_code = cli.run()
  File "/usr/local/lib/python2.7/dist-packages/ansible/cli/playbook.py", line 120, in run
    passwords=passwords)
  File "/usr/local/lib/python2.7/dist-packages/ansible/executor/playbook_executor.py", line 60, in __init__
    self._tqm = TaskQueueManager(inventory=inventory, variable_manager=variable_manager, loader=loader, options=options, passwords=self.passwords)
  File "/usr/local/lib/python2.7/dist-packages/ansible/executor/task_queue_manager.py", line 104, in __init__
    self._final_q = multiprocessing.Queue()
  File "/usr/lib/python2.7/multiprocessing/__init__.py", line 218, in Queue
    return Queue(maxsize)
  File "/usr/lib/python2.7/multiprocessing/queues.py", line 63, in __init__
    self._rlock = Lock()
  File "/usr/lib/python2.7/multiprocessing/synchronize.py", line 147, in __init__
    SemLock.__init__(self, SEMAPHORE, 1, 1)
  File "/usr/lib/python2.7/multiprocessing/synchronize.py", line 75, in __init__
    sl = self._semlock = _multiprocessing.SemLock(kind, value, maxvalue)
OSError: [Errno 38] Function not implemented

```



Discussion:
Files identified in the description:
* [lib/ansible/playbook](https://github.com/ansible/ansible/blob/devel/lib/ansible/playbook)

If these files are inaccurate, please update the `component name` section of the description or use the `!component` bot command.

[click here for bot help](https://github.com/ansible/ansibullbot/blob/master/ISSUE_HELP.md)
<!--- boilerplate: components_banner --->
I've seen this report enough times now, that instead of just closing, I'll keep this open.

We just need to wrap `self._final_q = multiprocessing.Queue()` in a `try/except` block and provide a human friendly error stating that python multiprocessing requires access to `/dev/shm`.

Resolving patch:
diff --git a/lib/ansible/executor/task_queue_manager.py b/lib/ansible/executor/task_queue_manager.py
--- a/lib/ansible/executor/task_queue_manager.py
+++ b/lib/ansible/executor/task_queue_manager.py
@@ -29,7 +29,7 @@
 from ansible.executor.stats import AggregateStats
 from ansible.executor.task_result import TaskResult
 from ansible.module_utils.six import string_types
-from ansible.module_utils._text import to_text
+from ansible.module_utils._text import to_text, to_native
 from ansible.playbook.block import Block
 from ansible.playbook.play_context import PlayContext
 from ansible.plugins.loader import callback_loader, strategy_loader, module_loader
@@ -101,7 +101,10 @@ def __init__(self, inventory, variable_manager, loader, options, passwords, stdo
         self._failed_hosts = dict()
         self._unreachable_hosts = dict()
 
-        self._final_q = multiprocessing.Queue()
+        try:
+            self._final_q = multiprocessing.Queue()
+        except OSError as e:
+            raise AnsibleError("Unable to use multiprocessing, this is normally caused by lack of access to /dev/shm: %s" % to_native(e))
 
         # A temporary file (opened pre-fork) used by connection
         # plugins for inter-process locking.


Test patch: