Repository: ansible/ansible

Issue:
[v2] RuntimeWarning: Parent module 'ansible.plugins.action' not found while handling absolute import
**Ansible Version**: 2.0.0
**Summary**:

My custom ansible plugins cause warnings to be shown for every execution:

```
$ ANSIBLE_CONFIG=vagrant.cfg ansible-playbook test-changelog.yml 

PLAY ***************************************************************************

TASK [setup] *******************************************************************
ok: [trusty]

TASK [changelog start=yes] *****************************************************
/home/mg/src/deployments/provisioning/action_plugins/changelog.py:1: RuntimeWarning: Parent module 'ansible.plugins.action' not found while handling absolute import
  import time
/home/mg/src/deployments/provisioning/action_plugins/changelog.py:5: RuntimeWarning: Parent module 'ansible.plugins.action' not found while handling absolute import
  from ansible.plugins.action import ActionBase
ok: [trusty]

TASK [changelog msg=one] *******************************************************
/home/mg/src/deployments/provisioning/action_plugins/changelog.py:1: RuntimeWarning: Parent module 'ansible.plugins.action' not found while handling absolute import
  import time
/home/mg/src/deployments/provisioning/action_plugins/changelog.py:5: RuntimeWarning: Parent module 'ansible.plugins.action' not found while handling absolute import
  from ansible.plugins.action import ActionBase
changed: [trusty]

TASK [changelog msg=two] *******************************************************
/home/mg/src/deployments/provisioning/action_plugins/changelog.py:1: RuntimeWarning: Parent module 'ansible.plugins.action' not found while handling absolute import
  import time
/home/mg/src/deployments/provisioning/action_plugins/changelog.py:5: RuntimeWarning: Parent module 'ansible.plugins.action' not found while handling absolute import
  from ansible.plugins.action import ActionBase
changed: [trusty]

TASK [changelog msg=three] *****************************************************
/home/mg/src/deployments/provisioning/action_plugins/changelog.py:1: RuntimeWarning: Parent module 'ansible.plugins.action' not found while handling absolute import
  import time
/home/mg/src/deployments/provisioning/action_plugins/changelog.py:5: RuntimeWarning: Parent module 'ansible.plugins.action' not found while handling absolute import
  from ansible.plugins.action import ActionBase
changed: [trusty]
```

I've bisected this to commit 057712c1296794bacc4f8def85d6caae6f77738f.

You can look at [my action plugin](https://gist.github.com/mgedmin/aec4678b25c2734f0d57), if you're curious, but there's nothing special about it.  Note how even `import time` causes the warning!



Discussion:
**Steps to reproduce**:
- git clone https://github.com/mgedmin/ansible-bug-12979
- cd ansible-bug-12979
- make test

**Expected behavior**:

```
ansible --version
ansible 1.9.3
  configured module search path = /usr/share/ansible
ansible-playbook test.yml
 [WARNING]: provided hosts list is empty, only localhost is available


PLAY [localhost] ************************************************************** 

TASK: [example ] ************************************************************** 
ok: [localhost]

PLAY RECAP ******************************************************************** 
localhost                  : ok=1    changed=0    unreachable=0    failed=0   
```

**Actual behavior**:

```
ansible --version
ansible 2.0.0 (devel b97887ba41) last updated 2015/10/30 17:25:40 (GMT +300)
  lib/ansible/modules/core: (detached HEAD a3a7cdb9aa) last updated 2015/10/30 08:10:45 (GMT +300)
  lib/ansible/modules/extras: (detached HEAD b1db0e893c) last updated 2015/10/30 08:10:45 (GMT +300)
  config file = /etc/ansible/ansible.cfg
  configured module search path = /usr/share/ansible
ansible-playbook test.yml
 [WARNING]: provided hosts list is empty, only localhost is available


PLAY ***************************************************************************

TASK [example] *****************************************************************
/home/mg/tmp/ansible-bug-12979/action_plugins/example.py:3: RuntimeWarning: Parent module 'ansible.plugins.action' not found while handling absolute import
  from ansible.plugins.action.normal import ActionModule
ok: [localhost]

PLAY RECAP *********************************************************************
localhost                  : ok=1    changed=0    unreachable=0    failed=0   
```

That is really weird, we disabled this warning message to remove this, as really it's just complaining about not finding an `__init__.py` in the custom plugin location.

@mgedmin also not seeing this one as a problem any further. Running your example as checked out from git:

<pre>
# ansible-playbook -vv test.yml 
PLAY [localhost] ************************************************************** 
TASK: [example ] ************************************************************** 
<localhost> REMOTE_MODULE example
ok: [localhost] => {"changed": false, "msg": "Hello"}
PLAY RECAP ******************************************************************** 
localhost                  : ok=1    changed=0    unreachable=0    failed=0   
</pre>

Are you still seeing this? If not, we'll go ahead and close it, especially as we haven't heard any further reports of this occuring.

Yeah, I'm still seeing RuntimeWarnings:

```
ansible --version
ansible 2.0.0 (devel 09994c18f3) last updated 2015/11/10 08:56:11 (GMT +300)
  lib/ansible/modules/core: (detached HEAD fde6ee15d6) last updated 2015/11/10 09:01:29 (GMT +300)
  lib/ansible/modules/extras: (detached HEAD 6cc43bbd56) last updated 2015/11/10 09:01:29 (GMT +300)
  config file = /etc/ansible/ansible.cfg
  configured module search path = /usr/share/ansible
ansible-playbook test.yml
 [WARNING]: provided hosts list is empty, only localhost is available


PLAY ***************************************************************************

TASK [example] *****************************************************************
/home/mg/src/bugs/ansible-bug-12979/action_plugins/example.py:3: RuntimeWarning: Parent module 'ansible.plugins.action' not found while handling absolute import
  from ansible.plugins.action.normal import ActionModule
ok: [localhost]

PLAY RECAP *********************************************************************
localhost                  : ok=1    changed=0    unreachable=0    failed=0   
```

I've tried the test in a very clean environment (because normally my $PATH contains ~/.venv/bin, where ~/.venv is a virtualenv that has ansible 1.9.4 installed):

```
$ export ANSIBLE_HOME=/home/mg/src/ansible
$ export PATH=$ANSIBLE_HOME/bin:/usr/bin:/bin
$ export PYTHONPATH=$ANSIBLE_HOME/lib
$ mkdir /tmp/home
$ env -i PATH=$PATH ANSIBLE_HOME=$ANSIBLE_HOME PYTHONPATH=$PYTHONPATH HOME=/tmp/home make
ansible --version
ansible 2.0.0 (devel d8fb5efa90) last updated 2015/11/10 12:12:55 (GMT +300)
  lib/ansible/modules/core: (detached HEAD fde6ee15d6) last updated 2015/11/10 09:01:29 (GMT +300)
  lib/ansible/modules/extras: (detached HEAD 6cc43bbd56) last updated 2015/11/10 09:01:29 (GMT +300)
  config file = /etc/ansible/ansible.cfg
  configured module search path = /usr/share/ansible
ansible-playbook test.yml
 [WARNING]: provided hosts list is empty, only localhost is available

PLAY ***************************************************************************

TASK [example] *****************************************************************
/home/mg/src/bugs/ansible-bug-12979/action_plugins/example.py:3: RuntimeWarning: Parent module 'ansible.plugins.action' not found while handling absolute import
  from ansible.plugins.action.normal import ActionModule
ok: [localhost]

PLAY RECAP *********************************************************************
localhost                  : ok=1    changed=0    unreachable=0    failed=0   
```

BTW /usr/share/ansible does not exist on my system.  I used to have ansible apt-get installed, but then apt-get removed it, because it was too old.

Hmmm.  apt-get removed.  Not purged.  Which means

```
$ dpkg -L ansible
/etc
/etc/ansible
/etc/ansible/ansible.cfg
/etc/ansible/hosts
```

But no, removing those stale config files doesn't affect anything:

```
$ apt-get purge ansible
...

$ env -i PATH=$ANSIBLE_HOME/bin:/usr/bin:/bin ANSIBLE_HOME=$ANSIBLE_HOME PYTHONPATH=$ANSIBLE_HOME/lib HOME=/tmp/home make
ansible --version
ansible 2.0.0 (devel d8fb5efa90) last updated 2015/11/10 12:12:55 (GMT +300)
  lib/ansible/modules/core: (detached HEAD fde6ee15d6) last updated 2015/11/10 09:01:29 (GMT +300)
  lib/ansible/modules/extras: (detached HEAD 6cc43bbd56) last updated 2015/11/10 09:01:29 (GMT +300)
  config file = 
  configured module search path = Default w/o overrides
ansible-playbook test.yml
 [WARNING]: provided hosts list is empty, only localhost is available

PLAY ***************************************************************************

TASK [example] *****************************************************************
/home/mg/src/bugs/ansible-bug-12979/action_plugins/example.py:3: RuntimeWarning: Parent module 'ansible.plugins.action' not found while handling absolute import
  from ansible.plugins.action.normal import ActionModule
ok: [localhost]

PLAY RECAP *********************************************************************
localhost                  : ok=1    changed=0    unreachable=0    failed=0   
```

Clue: the warning disappears if I add a task like `- command: echo hi` above my custom module.

I've instrumented the Ansible source like this:

``` diff
diff --git a/lib/ansible/plugins/__init__.py b/lib/ansible/plugins/__init__.py
index 4def393..81c7230 100644
--- a/lib/ansible/plugins/__init__.py
+++ b/lib/ansible/plugins/__init__.py
@@ -304,6 +304,8 @@ class PluginLoader:
     __contains__ = has_plugin

     def _load_module_source(self, name, path):
+        import os
+        print(os.getpid(), "LOADING", name, "FROM", path)
         with open(path, 'r') as module_file:
             module = imp.load_source(name, path, module_file)
         return module
```

I see this:

```
ansible-playbook test.yml
19164 LOADING ansible.plugins.cache.memory FROM /home/mg/src/ansible/lib/ansible/plugins/cache/memory.py
 [WARNING]: provided hosts list is empty, only localhost is available

19164 LOADING /home/mg/src/ansible/lib/ansible/plugins/callback/context_demo FROM /home/mg/src/ansible/lib/ansible/plugins/callback/context_demo.py
19164 LOADING /home/mg/src/ansible/lib/ansible/plugins/callback/default FROM /home/mg/src/ansible/lib/ansible/plugins/callback/default.py
19164 LOADING /home/mg/src/ansible/lib/ansible/plugins/callback/hipchat FROM /home/mg/src/ansible/lib/ansible/plugins/callback/hipchat.py
19164 LOADING /home/mg/src/ansible/lib/ansible/plugins/callback/log_plays FROM /home/mg/src/ansible/lib/ansible/plugins/callback/log_plays.py
19164 LOADING /home/mg/src/ansible/lib/ansible/plugins/callback/mail FROM /home/mg/src/ansible/lib/ansible/plugins/callback/mail.py
19164 LOADING /home/mg/src/ansible/lib/ansible/plugins/callback/minimal FROM /home/mg/src/ansible/lib/ansible/plugins/callback/minimal.py
19164 LOADING /home/mg/src/ansible/lib/ansible/plugins/callback/oneline FROM /home/mg/src/ansible/lib/ansible/plugins/callback/oneline.py
19164 LOADING /home/mg/src/ansible/lib/ansible/plugins/callback/osx_say FROM /home/mg/src/ansible/lib/ansible/plugins/callback/osx_say.py
19164 LOADING /home/mg/src/ansible/lib/ansible/plugins/callback/profile_tasks FROM /home/mg/src/ansible/lib/ansible/plugins/callback/profile_tasks.py
19164 LOADING /home/mg/src/ansible/lib/ansible/plugins/callback/skippy FROM /home/mg/src/ansible/lib/ansible/plugins/callback/skippy.py
19164 LOADING /home/mg/src/ansible/lib/ansible/plugins/callback/syslog_json FROM /home/mg/src/ansible/lib/ansible/plugins/callback/syslog_json.py
19164 LOADING /home/mg/src/ansible/lib/ansible/plugins/callback/timer FROM /home/mg/src/ansible/lib/ansible/plugins/callback/timer.py
19164 LOADING /home/mg/src/ansible/lib/ansible/plugins/callback/tree FROM /home/mg/src/ansible/lib/ansible/plugins/callback/tree.py

PLAY ***************************************************************************
19164 LOADING ansible.plugins.strategy.linear FROM /home/mg/src/ansible/lib/ansible/plugins/strategy/linear.py
19164 LOADING ansible.plugins.action.example FROM /home/mg/src/bugs/ansible-bug-12979/action_plugins/example.py

TASK [example] *****************************************************************
19167 LOADING ansible.plugins.connection.local FROM /home/mg/src/ansible/lib/ansible/plugins/connection/local.py
19167 LOADING ansible.plugins.shell.sh FROM /home/mg/src/ansible/lib/ansible/plugins/shell/sh.py
19167 LOADING ansible.plugins.action.example FROM /home/mg/src/bugs/ansible-bug-12979/action_plugins/example.py
/home/mg/src/bugs/ansible-bug-12979/action_plugins/example.py:3: RuntimeWarning: Parent module 'ansible.plugins.action' not found while handling absolute import
  from ansible.plugins.action.normal import ActionModule
ok: [localhost]

PLAY RECAP *********************************************************************
localhost                  : ok=1    changed=0    unreachable=0    failed=0   
```

And if I change the test.yml as follows:

``` diff
diff --git a/test.yml b/test.yml
index 7128b19..11cfff3 100644
--- a/test.yml
+++ b/test.yml
@@ -2,4 +2,5 @@
 - hosts: localhost
   gather_facts: no
   tasks:
+    - command: echo hi
     - example:
```

Then I see this:

```
ansible --version
ansible 2.0.0 (devel d8fb5efa90) last updated 2015/11/10 12:12:55 (GMT +300)
  lib/ansible/modules/core: (detached HEAD fde6ee15d6) last updated 2015/11/10 09:01:29 (GMT +300)
  lib/ansible/modules/extras: (detached HEAD 6cc43bbd56) last updated 2015/11/10 09:01:29 (GMT +300)
  config file = 
  configured module search path = Default w/o overrides
ansible-playbook test.yml
19288 LOADING ansible.plugins.cache.memory FROM /home/mg/src/ansible/lib/ansible/plugins/cache/memory.py
 [WARNING]: provided hosts list is empty, only localhost is available

19288 LOADING /home/mg/src/ansible/lib/ansible/plugins/callback/context_demo FROM /home/mg/src/ansible/lib/ansible/plugins/callback/context_demo.py
19288 LOADING /home/mg/src/ansible/lib/ansible/plugins/callback/default FROM /home/mg/src/ansible/lib/ansible/plugins/callback/default.py
19288 LOADING /home/mg/src/ansible/lib/ansible/plugins/callback/hipchat FROM /home/mg/src/ansible/lib/ansible/plugins/callback/hipchat.py
19288 LOADING /home/mg/src/ansible/lib/ansible/plugins/callback/log_plays FROM /home/mg/src/ansible/lib/ansible/plugins/callback/log_plays.py
19288 LOADING /home/mg/src/ansible/lib/ansible/plugins/callback/mail FROM /home/mg/src/ansible/lib/ansible/plugins/callback/mail.py
19288 LOADING /home/mg/src/ansible/lib/ansible/plugins/callback/minimal FROM /home/mg/src/ansible/lib/ansible/plugins/callback/minimal.py
19288 LOADING /home/mg/src/ansible/lib/ansible/plugins/callback/oneline FROM /home/mg/src/ansible/lib/ansible/plugins/callback/oneline.py
19288 LOADING /home/mg/src/ansible/lib/ansible/plugins/callback/osx_say FROM /home/mg/src/ansible/lib/ansible/plugins/callback/osx_say.py
19288 LOADING /home/mg/src/ansible/lib/ansible/plugins/callback/profile_tasks FROM /home/mg/src/ansible/lib/ansible/plugins/callback/profile_tasks.py
19288 LOADING /home/mg/src/ansible/lib/ansible/plugins/callback/skippy FROM /home/mg/src/ansible/lib/ansible/plugins/callback/skippy.py
19288 LOADING /home/mg/src/ansible/lib/ansible/plugins/callback/syslog_json FROM /home/mg/src/ansible/lib/ansible/plugins/callback/syslog_json.py
19288 LOADING /home/mg/src/ansible/lib/ansible/plugins/callback/timer FROM /home/mg/src/ansible/lib/ansible/plugins/callback/timer.py
19288 LOADING /home/mg/src/ansible/lib/ansible/plugins/callback/tree FROM /home/mg/src/ansible/lib/ansible/plugins/callback/tree.py

PLAY ***************************************************************************
19288 LOADING ansible.plugins.strategy.linear FROM /home/mg/src/ansible/lib/ansible/plugins/strategy/linear.py

TASK [command] *****************************************************************
19291 LOADING ansible.plugins.connection.local FROM /home/mg/src/ansible/lib/ansible/plugins/connection/local.py
19291 LOADING ansible.plugins.shell.sh FROM /home/mg/src/ansible/lib/ansible/plugins/shell/sh.py
19291 LOADING ansible.plugins.action.normal FROM /home/mg/src/ansible/lib/ansible/plugins/action/normal.py
changed: [localhost]
19288 LOADING ansible.plugins.action.example FROM /home/mg/src/bugs/ansible-bug-12979/action_plugins/example.py

TASK [example] *****************************************************************
19291 LOADING ansible.plugins.action.example FROM /home/mg/src/bugs/ansible-bug-12979/action_plugins/example.py
ok: [localhost]

PLAY RECAP *********************************************************************
localhost                  : ok=2    changed=1    unreachable=0    failed=0   
```

No warning.

More clues:

```
diff --git a/lib/ansible/plugins/__init__.py b/lib/ansible/plugins/__init__.py
index 4def393..c021702 100644
--- a/lib/ansible/plugins/__init__.py
+++ b/lib/ansible/plugins/__init__.py
@@ -304,8 +304,17 @@ class PluginLoader:
     __contains__ = has_plugin

     def _load_module_source(self, name, path):
+        if 'action' in name:
+            import os, sys
+            print(os.getpid(), "sys.modules['ansible.plugins.action'] is", sys.modules.get('ansible.plugins.action', 'missing'))
+            print(os.getpid(), "sys.modules['ansible.plugins.action.normal'] is", sys.modules.get('ansible.plugins.action.normal', 'missing'))
+            print(os.getpid(), "LOADING", name, "FROM", path)
         with open(path, 'r') as module_file:
             module = imp.load_source(name, path, module_file)
+        if 'action' in name:
+            import os, sys
+            print(os.getpid(), "sys.modules['ansible.plugins.action'] is", sys.modules.get('ansible.plugins.action', 'missing'))
+            print(os.getpid(), "sys.modules['ansible.plugins.action.normal'] is", sys.modules.get('ansible.plugins.action.normal', 'missing'))
         return module

     def get(self, name, *args, **kwargs):
```

With this I see

```
PLAY ***************************************************************************
20019 sys.modules['ansible.plugins.action'] is <module 'ansible.plugins.action' from '/home/mg/src/ansible/lib/ansible/plugins/action/__init__.pyc'>
20019 sys.modules['ansible.plugins.action.normal'] is missing
20019 LOADING ansible.plugins.action.example FROM /home/mg/src/bugs/ansible-bug-12979/action_plugins/example.py
20019 sys.modules['ansible.plugins.action'] is <module 'ansible.plugins.action' from '/home/mg/src/ansible/lib/ansible/plugins/action/__init__.pyc'>
20019 sys.modules['ansible.plugins.action.normal'] is <module 'ansible.plugins.action.normal' from '/home/mg/src/ansible/lib/ansible/plugins/action/normal.pyc'>

TASK [example] *****************************************************************
20022 sys.modules['ansible.plugins.action'] is missing
20022 sys.modules['ansible.plugins.action.normal'] is missing
20022 LOADING ansible.plugins.action.example FROM /home/mg/src/bugs/ansible-bug-12979/action_plugins/example.py
/home/mg/src/bugs/ansible-bug-12979/action_plugins/example.py:3: RuntimeWarning: Parent module 'ansible.plugins.action' not found while handling absolute import
  from ansible.plugins.action.normal import ActionModule
20022 sys.modules['ansible.plugins.action'] is <module 'ansible.plugins.action' from '/home/mg/src/ansible/lib/ansible/plugins/action/__init__.pyc'>
20022 sys.modules['ansible.plugins.action.normal'] is <module 'ansible.plugins.action.normal' from '/home/mg/src/ansible/lib/ansible/plugins/action/normal.pyc'>
ok: [localhost]

PLAY RECAP *********************************************************************
localhost                  : ok=1    changed=0    unreachable=0    failed=0   
```

The warning shows up when `sys.modules['ansible.plugins.action']` does not exist prior to calling `imp.load_source()`.

Control group (with the extra `command` task above `example`):

```
PLAY ***************************************************************************

TASK [command] *****************************************************************
20199 sys.modules['ansible.plugins.action'] is <module 'ansible.plugins.action' from '/home/mg/src/ansible/lib/ansible/plugins/action/__init__.pyc'>
20199 sys.modules['ansible.plugins.action.normal'] is missing
20199 LOADING ansible.plugins.action.normal FROM /home/mg/src/ansible/lib/ansible/plugins/action/normal.py
20199 sys.modules['ansible.plugins.action'] is <module 'ansible.plugins.action' from '/home/mg/src/ansible/lib/ansible/plugins/action/__init__.pyc'>
20199 sys.modules['ansible.plugins.action.normal'] is <module 'ansible.plugins.action.normal' from '/home/mg/src/ansible/lib/ansible/plugins/action/normal.pyc'>
changed: [localhost]
20196 sys.modules['ansible.plugins.action'] is <module 'ansible.plugins.action' from '/home/mg/src/ansible/lib/ansible/plugins/action/__init__.pyc'>
20196 sys.modules['ansible.plugins.action.normal'] is missing
20196 LOADING ansible.plugins.action.example FROM /home/mg/src/bugs/ansible-bug-12979/action_plugins/example.py
20196 sys.modules['ansible.plugins.action'] is <module 'ansible.plugins.action' from '/home/mg/src/ansible/lib/ansible/plugins/action/__init__.pyc'>
20196 sys.modules['ansible.plugins.action.normal'] is <module 'ansible.plugins.action.normal' from '/home/mg/src/ansible/lib/ansible/plugins/action/normal.pyc'>

TASK [example] *****************************************************************
20199 sys.modules['ansible.plugins.action'] is <module 'ansible.plugins.action' from '/home/mg/src/ansible/lib/ansible/plugins/action/__init__.pyc'>
20199 sys.modules['ansible.plugins.action.normal'] is <module 'ansible.plugins.action.normal' from '/home/mg/src/ansible/lib/ansible/plugins/action/normal.pyc'>
20199 LOADING ansible.plugins.action.example FROM /home/mg/src/bugs/ansible-bug-12979/action_plugins/example.py
20199 sys.modules['ansible.plugins.action'] is <module 'ansible.plugins.action' from '/home/mg/src/ansible/lib/ansible/plugins/action/__init__.pyc'>
20199 sys.modules['ansible.plugins.action.normal'] is <module 'ansible.plugins.action.normal' from '/home/mg/src/ansible/lib/ansible/plugins/action/normal.pyc'>
ok: [localhost]

PLAY RECAP *********************************************************************
localhost                  : ok=2    changed=1    unreachable=0    failed=0   
```

So when Ansible loads regular action plugins, it pre-imports the `ansible.plugins.action` package.
When Ansible loads custom action plugins, it doesn't, and that somehow causes this error.


Resolving patch:
diff --git a/lib/ansible/plugins/__init__.py b/lib/ansible/plugins/__init__.py
--- a/lib/ansible/plugins/__init__.py
+++ b/lib/ansible/plugins/__init__.py
@@ -304,6 +304,9 @@ def has_plugin(self, name):
     __contains__ = has_plugin
 
     def _load_module_source(self, name, path):
+        if name in sys.modules:
+            # See https://github.com/ansible/ansible/issues/13110
+            return sys.modules[name]
         with open(path, 'r') as module_file:
             module = imp.load_source(name, path, module_file)
         return module


Test patch: