Repository: StackStorm/st2

Issue:
Cannot create action with a name that collides with a Python library
We created an action called "elasticsearch.py" (with the appropriate yaml). When we try to run it we get an ST2 exception. We are able to reproduce with another action called "test.py". Our hypothesis is that if the action name is the same as a Python library, it fails to run it. This happens after migrating to 1.4.0 from 0.13.2 (it works fine in 0.13.2).

Here's the stacktrace:

```
$ st2 run netflix_samples.elasticsearch app_region=us-east-1
.
id: 57211d9b7279e6326f6b025f
status: failed
parameters: 
 app_region: us-east-1
result: 
 exit_code: 1
 result: null
 stderr: "Traceback (most recent call last):
 File "/opt/stackstorm/st2/local/lib/python2.7/site-packages/st2actions/runners/python_action_wrapper.py", line 164, in <module>
   obj.run()
 File "/opt/stackstorm/st2/local/lib/python2.7/site-packages/st2actions/runners/python_action_wrapper.py", line 102, in run
   action = self._get_action_instance()
 File "/opt/stackstorm/st2/local/lib/python2.7/site-packages/st2actions/runners/python_action_wrapper.py", line 116, in _get_action_instance
   actions_cls = action_loader.register_plugin(Action, self._file_path)
 File "/opt/stackstorm/st2/lib/python2.7/site-packages/st2common/util/loader.py", line 163, in register_plugin
   (plugin_abs_file_path))
Exception: Found no classes in plugin file "/opt/stackstorm/packs/netflix_samples/actions/elasticsearch.py" matching requirements.
"
 stdout: ''
```



Discussion:
Same issue with  pack names
Example: can't have a pack named datadog if did "pip install datadog"

@jsjeannotte Can you share contents of `test.py` and `test.yaml` where you see this issue?

https://github.com/StackStorm/st2/blob/master/st2common/st2common/util/loader.py#L55

This is because of how we use some python module loading  hackery to import classes with actions in them. I can point you to the code but fixing it would require us to move to stevedore like entry_point approach which then makes the whole pack story horrendous IMO.

Suspect it might originate https://github.com/StackStorm/st2/blob/master/st2common/st2common/util/loader.py#L130.

We might need to pass in the pack_name as per https://docs.python.org/2/library/importlib.html#importlib.import_module although there is no guarantee of a package being present from the python perspective. This might be in @Kami wheelhouse.

Any updates on this issue? 

@skarmark Since there are work arounds for this and really it seems like a trivial problem to me, can we wait on this? 

We can wait on this. It is not super urgent but problematic nonetheless. We give freedom to our consumers to select their action and action file names and we have to catch these errors at runtime and ask them to change the action name which is not a very good user experience. Also, changing the action name which is deployed and possibly running is tricky. 

I will look into it some time in the next couple of days and see if it's possible to do an easy fix.

After looking into the issue again more in-depth I don't think we should change this behavior - right now it behaves the same as any other code in Python.

In regular Python there are some ways around it (e.g. using from future import absolute_import), but in our case only possible work-around to get it to work would be to to manipulate `PYTHONPATH` for the Python action sub-process, but this would cause problems since it would mask other imports.

Another option would be to make action/ directory a package (this would require user to add **init**.py file to the directory), but I'm not a big fan of this approach since actions are not necessary Python only.

In any case, I will still play around with it a bit and see if we can come up with a compromise - perhaps make a package thing optional and if **init**.py is not present, it will behave as it does right now.


Resolving patch:
diff --git a/st2common/st2common/util/loader.py b/st2common/st2common/util/loader.py
--- a/st2common/st2common/util/loader.py
+++ b/st2common/st2common/util/loader.py
@@ -13,7 +13,7 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 
-import importlib
+import imp
 import inspect
 import json
 import os
@@ -127,7 +127,7 @@ def register_plugin_class(base_class, file_path, class_name):
     if module_name is None:
         return None
 
-    module = importlib.import_module(module_name)
+    module = imp.load_source(module_name, file_path)
     klass = getattr(module, class_name, None)
 
     if not klass:
@@ -142,10 +142,12 @@ def register_plugin(plugin_base_class, plugin_abs_file_path):
     registered_plugins = []
     plugin_dir = os.path.dirname(os.path.realpath(plugin_abs_file_path))
     _register_plugin_path(plugin_dir)
+
     module_name = _get_plugin_module(plugin_abs_file_path)
     if module_name is None:
         return None
-    module = importlib.import_module(module_name)
+
+    module = imp.load_source(module_name, plugin_abs_file_path)
     klasses = _get_plugin_classes(module)
 
     # Try registering classes in plugin file. Some may fail.


Test patch:
diff --git a/st2actions/tests/unit/test_pythonrunner.py b/st2actions/tests/unit/test_pythonrunner.py
--- a/st2actions/tests/unit/test_pythonrunner.py
+++ b/st2actions/tests/unit/test_pythonrunner.py
@@ -34,6 +34,9 @@
 
 PASCAL_ROW_ACTION_PATH = os.path.join(tests_base.get_resources_path(), 'packs',
                                       'pythonactions/actions/pascal_row.py')
+TEST_ACTION_PATH = os.path.join(tests_base.get_resources_path(), 'packs',
+                                'pythonactions/actions/test.py')
+
 # Note: runner inherits parent args which doesn't work with tests since test pass additional
 # unrecognized args
 mock_sys = mock.Mock()
@@ -366,6 +369,18 @@ def run(self):
         self.assertEqual(action3.config, config)
         self.assertEqual(action3.action_service, action_service)
 
+    def test_action_with_same_module_name_as_module_in_stdlib(self):
+        runner = pythonrunner.get_runner()
+        runner.action = self._get_mock_action_obj()
+        runner.runner_parameters = {}
+        runner.entry_point = TEST_ACTION_PATH
+        runner.container_service = service.RunnerContainerService()
+        runner.pre_run()
+        (status, output, _) = runner.run({})
+        self.assertEqual(status, LIVEACTION_STATUS_SUCCEEDED)
+        self.assertTrue(output is not None)
+        self.assertEqual(output['result'], 'test action')
+
     def _get_mock_action_obj(self):
         """
         Return mock action object.
diff --git a/st2tests/st2tests/resources/packs/pythonactions/actions/test.py b/st2tests/st2tests/resources/packs/pythonactions/actions/test.py
new file mode 100644
--- /dev/null
+++ b/st2tests/st2tests/resources/packs/pythonactions/actions/test.py
@@ -0,0 +1,9 @@
+# Python action which uses the same module name as built-in Python module
+# (test)
+
+from st2actions.runners.pythonrunner import Action
+
+
+class TestAction(Action):
+    def run(self):
+        return 'test action'