Repository: conan-io/conan

Issue:
[python_requires] imports inside python_required modules
Title: [python_requires] imports inside python_required modules

- [x] I've read the [CONTRIBUTING guide](https://raw.githubusercontent.com/conan-io/conan/develop/.github/CONTRIBUTING.md).
- [x] I've specified the Conan version, operating system version and any tool that can be relevant.
- [x] I've explained the steps to reproduce the error or the motivation/use case of the question/suggestion.

Conan 1.8.2
Python 3.6.5
Windows 7

according to the official Conan documentation for python_requires, we created a base class package with helper functions in a separate python file (helper.py) which is imported from the base class with a regular Python import statement like described in the documentation.

This worked perfectly for us as long as we had only one version of the base package. The problem occured after we created version 2.0 of the base package with one module using version 2.0 and another module still using version 1.0. The helper.py import statement inside the base package cannot distinguish from which version of the base package it is called and therefore always helper.py from the first version mentioned in a Conan file is used.

Here are the steps to reproduce this problem. I hope it gets a little bit more clear then:

**base/conanfile.py**
 ```python
from conans import ConanFile
import helper
class Base(ConanFile):
    exports = "*.py"
```

**base/helper.py**
```python
def getVersion():
    return "1.0"
```
Conan command: `conan export . Base/1.0@user/channel`
This exports Base/1.0@user/channel correctly.

**module1/conanfile.py**
```python
from conans import ConanFile, python_requires
base = python_requires("Base/1.0@user/channel")
class Module1(ConanFile):
    name = "module1"
    version = base.helper.getVersion()
```
Conan command: `conan export . user/channel`.
This exports module1/1.0@user/channel correctly.

**module2/conanfile.py**
```python
from conans import ConanFile, python_requires
base = python_requires("Base/1.0@user/channel")
module1 = python_requires("module1/1.0@user/channel")
class MyPackage(ConanFile):
    name = "module2"
    version = base.helper.getVersion()
```
Conan command: `conan export . user/channel`.
This exports module2/1.0@user/channel correctly.

So far everthing works well. Now we create a new version 2.0 of the Base package. In the new version we rename the helper.py method getVersion() to getVersionInADifferentWay():

**base/helper.py**
```python
def getVersionInADifferentWay():
    return "2.0"
```
Conan command: `conan export . Base/2.0@user/channel`
This exports Base/2.0@user/channel correctly.

Now module2 should use the new Base package 2.0 whereas module1 should still use the old version Base/1.0:

**module2/conanfile.py**
```python
from conans import ConanFile, python_requires
base = python_requires("Base/2.0@user/channel")
module1 = python_requires("module1/1.0@user/channel")
class MyPackage(ConanFile):
    name = "module2"
    version = base.helper.getVersionInADifferentWay()
```
Conan command: `conan export . user/channel`.
This leads to the following error:
```
ERROR: Unable to load conanfile in module2/V2.0/conanfile.py
KeyError: 'module1/1.0@user/channel'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "Python3/lib/site-packages/conans/client/loader.py", line 235, in _parse_file
    loaded = imp.load_source(filename, conan_file_path)
  File "Python3/lib/imp.py", line 172, in load_source
    module = _load(spec)
  File "<frozen importlib._bootstrap>", line 684, in _load
  File "<frozen importlib._bootstrap>", line 665, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 678, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "module2/V2.0/conanfile.py", line 3, in <module>
    module1 = python_requires("module1/1.0@user/channel")
  File "Python3/lib/site-packages/conans/client/graph/python_requires.py", line 41, in __call__
    module = imp.load_source(str(r).replace(".", "*"), path)
  File "Python3/lib/imp.py", line 172, in load_source
    module = _load(spec)
  File "<frozen importlib._bootstrap>", line 684, in _load
  File "<frozen importlib._bootstrap>", line 665, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 678, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File ".conan/data/module1/1.0/user/channel/export/conanfile.py", line 3, in <module>
    class Module1(ConanFile):
  File ".conan/data/module1/1.0/user/channel/export/conanfile.py", line 5, in Module1
    version = base.helper.getVersion()
AttributeError: module 'helper' has no attribute 'getVersion'
```

Could you please give us a hint or suggestions how we could solve this problem with Conan

Thank You!



Discussion:

Resolving patch:
diff --git a/conans/client/graph/python_requires.py b/conans/client/graph/python_requires.py
--- a/conans/client/graph/python_requires.py
+++ b/conans/client/graph/python_requires.py
@@ -1,11 +1,9 @@
-import imp
-import sys
-import os
+from collections import namedtuple
 
-from conans.model.ref import ConanFileReference
+from conans.client.loader import parse_conanfile
 from conans.client.recorder.action_recorder import ActionRecorder
+from conans.model.ref import ConanFileReference
 from conans.model.requires import Requirement
-from collections import namedtuple
 
 
 PythonRequire = namedtuple("PythonRequire", "conan_ref module")
@@ -36,13 +34,7 @@ def __call__(self, require):
             result = self._proxy.get_recipe(r, False, False, remote_name=None,
                                             recorder=ActionRecorder())
             path, _, _, reference = result
-            try:
-                dirname = os.path.dirname(path)
-                sys.path.append(dirname)
-                # replace avoid warnings in Py2 with dots
-                module = imp.load_source(str(r).replace(".", "*"), path)
-            finally:
-                sys.path.pop()
+            module, _ = parse_conanfile(path)
             python_require = PythonRequire(reference, module)
             self._cached_requires[require] = python_require
         self._requires.append(python_require)
diff --git a/conans/client/loader.py b/conans/client/loader.py
--- a/conans/client/loader.py
+++ b/conans/client/loader.py
@@ -43,7 +43,7 @@ def __init__(self, runner, output, python_requires):
         sys.modules["conans"].python_requires = python_requires
 
     def load_class(self, conanfile_path):
-        loaded, filename = _parse_file(conanfile_path)
+        loaded, filename = parse_conanfile(conanfile_path)
         try:
             conanfile = _parse_module(loaded, filename)
             conanfile.python_requires = self._python_requires.requires
@@ -218,17 +218,17 @@ def _invalid_python_requires(require):
     raise ConanException("Invalid use of python_requires(%s)" % require)
 
 
-def _parse_file(conan_file_path):
+def parse_conanfile(conan_file_path):
     """ From a given path, obtain the in memory python import module
     """
 
     if not os.path.exists(conan_file_path):
         raise NotFoundException("%s not found!" % conan_file_path)
 
+    module_id = str(uuid.uuid1())
+    current_dir = os.path.dirname(conan_file_path)
+    sys.path.insert(0, current_dir)
     try:
-        module_id = str(uuid.uuid1())
-        current_dir = os.path.dirname(conan_file_path)
-        sys.path.append(current_dir)
         old_modules = list(sys.modules.keys())
         with chdir(current_dir):
             sys.dont_write_bytecode = True
@@ -256,6 +256,6 @@ def _parse_file(conan_file_path):
         raise ConanException("Unable to load conanfile in %s\n%s" % (conan_file_path,
                                                                      '\n'.join(trace[3:])))
     finally:
-        sys.path.pop()
+        sys.path.pop(0)
 
     return loaded, module_id


Test patch:
diff --git a/conans/test/client/cmd/export_test.py b/conans/test/client/cmd/export_test.py
--- a/conans/test/client/cmd/export_test.py
+++ b/conans/test/client/cmd/export_test.py
@@ -1,4 +1,3 @@
-
 import os
 import shutil
 import unittest
@@ -9,7 +8,7 @@
 from conans.test.utils.tools import TestServer, TestClient
 from conans.util.files import save, load
 from conans.client.cmd.export import _replace_scm_data_in_conanfile
-from conans.client.loader import _parse_file
+from conans.client.loader import parse_conanfile
 from conans.test.utils.test_files import temp_folder
 from conans.model.scm import SCMData
 
@@ -24,7 +23,7 @@ class ConanLib(ConanFile):
     scm = {{"revision": "{revision}",
            "type": "git",
            "url": "{url}"}}
-    
+
     {after_scm}
 
 {after_recipe}
@@ -49,7 +48,7 @@ def _do_actual_test(self, scm_data, after_scm, after_recipe):
         scm_data = SCMData(conanfile=namedtuple('_', 'scm')(scm=scm_data))
         _replace_scm_data_in_conanfile(self.conanfile_path, scm_data)
         self.assertEqual(load(self.conanfile_path), target_conanfile)
-        _parse_file(self.conanfile_path)  # Check that the resulting file is valid python code.
+        parse_conanfile(self.conanfile_path)  # Check that the resulting file is valid python code.
 
     def test_conanfile_after_scm(self):
         scm_data = {'type': 'git',
diff --git a/conans/test/functional/scm_test.py b/conans/test/functional/scm_test.py
--- a/conans/test/functional/scm_test.py
+++ b/conans/test/functional/scm_test.py
@@ -2,6 +2,7 @@
 import os
 import unittest
 from collections import namedtuple
+from nose.plugins.attrib import attr
 
 from conans.client.tools.scm import Git, SVN
 from conans.model.ref import ConanFileReference, PackageReference
@@ -36,6 +37,7 @@ def build(self):
 base_svn = base % "svn"
 
 
+@attr('git')
 class GitSCMTest(unittest.TestCase):
 
     def setUp(self):
@@ -477,6 +479,7 @@ def scm_serialization_test(self):
         self.assertEquals(data, data2)
 
 
+@attr('svn')
 class SVNSCMTest(SVNLocalRepoTestCase):
 
     def setUp(self):
@@ -484,7 +487,6 @@ def setUp(self):
         self.client = TestClient()
 
     def _commit_contents(self):
-        # self.client.runner('svn co "{url}" "{path}"'.format(url=self.repo_url, path=self.client.current_folder))
         self.client.runner("svn add *", cwd=self.client.current_folder)
         self.client.runner('svn commit -m  "commiting"', cwd=self.client.current_folder)
 
diff --git a/conans/test/integration/python_build_test.py b/conans/test/integration/python_build_test.py
--- a/conans/test/integration/python_build_test.py
+++ b/conans/test/integration/python_build_test.py
@@ -1,12 +1,12 @@
-import unittest
-from conans.test.utils.tools import TestClient, TestServer,\
-    create_local_git_repo
-from conans.paths import CONANFILE, BUILD_INFO
-from conans.util.files import load, save
 import os
-from conans.test.utils.test_files import temp_folder
+import unittest
+
 from conans.model.info import ConanInfo
 from conans.model.ref import ConanFileReference
+from conans.paths import CONANFILE, BUILD_INFO
+from conans.test.utils.test_files import temp_folder
+from conans.test.utils.tools import TestClient, TestServer, create_local_git_repo
+from conans.util.files import load, save
 
 
 conanfile = """from conans import ConanFile
@@ -104,8 +104,8 @@ class PkgTest(base.MyConanfileBase):
         self.assertIn("Pkg/0.1@lasote/testing: My cool package_info!", client.out)
         client.run("remove * -f")
         client.run("download Pkg/0.1@lasote/testing")
-        self.assertIn("Pkg/0.1@lasote/testing: Package installed 5ab84d6acfe1f23c4fae0ab88f26e3a396351ac9",
-                      client.out)
+        self.assertIn("Pkg/0.1@lasote/testing: Package installed "
+                      "5ab84d6acfe1f23c4fae0ab88f26e3a396351ac9", client.out)
 
     def reuse_version_ranges_test(self):
         client = TestClient()
@@ -138,7 +138,8 @@ def source(self):
         self.assertTrue(error)
         self.assertIn("ERROR: Pkg/0.1@lasote/testing: Error in source() method, line 4", client.out)
         self.assertIn('base = python_requires("MyConanfileBase/1.0@lasote/testing', client.out)
-        self.assertIn("Invalid use of python_requires(MyConanfileBase/1.0@lasote/testing)", client.out)
+        self.assertIn("Invalid use of python_requires(MyConanfileBase/1.0@lasote/testing)",
+                      client.out)
 
     def transitive_multiple_reuse_test(self):
         client = TestClient()
@@ -257,8 +258,8 @@ class PkgTest(base.MyConanfileBase):
         self.assertIn("Pkg/0.1@lasote/testing: My cool package_info!", client.out)
         client.run("remove * -f")
         client.run("download Pkg/0.1@lasote/testing")
-        self.assertIn("Pkg/0.1@lasote/testing: Package installed 5ab84d6acfe1f23c4fae0ab88f26e3a396351ac9",
-                      client.out)
+        self.assertIn("Pkg/0.1@lasote/testing: Package installed "
+                      "5ab84d6acfe1f23c4fae0ab88f26e3a396351ac9", client.out)
 
     def reuse_scm_test(self):
         client = TestClient()
@@ -383,6 +384,45 @@ def build(self):
         client.run("create . Pkg/0.1@user/testing")
         self.assertIn("Pkg/0.1@user/testing: HEADER CONTENT!: my header Pkg!!", client.out)
 
+    def transitive_imports_conflicts_test(self):
+        # https://github.com/conan-io/conan/issues/3874
+        client = TestClient()
+        conanfile = """from conans import ConanFile
+import myhelper
+class SourceBuild(ConanFile):
+    exports = "*.py"
+"""
+        helper = """def myhelp(output):
+    output.info("MyHelperOutput!")
+"""
+        client.save({"conanfile.py": conanfile,
+                     "myhelper.py": helper})
+        client.run("export . base1/1.0@user/channel")
+        client.save({"myhelper.py": helper.replace("MyHelperOutput!", "MyOtherHelperOutput!")})
+        client.run("export . base2/1.0@user/channel")
+
+        conanfile = """from conans import ConanFile, python_requires
+base2 = python_requires("base2/1.0@user/channel")
+base1 = python_requires("base1/1.0@user/channel")
+
+class MyConanfileBase(ConanFile):
+    def build(self):
+        base1.myhelper.myhelp(self.output)
+        base2.myhelper.myhelp(self.output)
+"""
+        # This should work, even if there is a local "myhelper.py" file, which could be
+        # accidentaly imported (and it was, it was a bug)
+        client.save({"conanfile.py": conanfile})
+        client.run("create . Pkg/0.1@lasote/testing")
+        self.assertIn("Pkg/0.1@lasote/testing: MyHelperOutput!", client.out)
+        self.assertIn("Pkg/0.1@lasote/testing: MyOtherHelperOutput!", client.out)
+
+        # Now, the same, but with "clean_first=True", should keep working
+        client.save({"conanfile.py": conanfile}, clean_first=True)
+        client.run("create . Pkg/0.1@lasote/testing")
+        self.assertIn("Pkg/0.1@lasote/testing: MyHelperOutput!", client.out)
+        self.assertIn("Pkg/0.1@lasote/testing: MyOtherHelperOutput!", client.out)
+
 
 class PythonBuildTest(unittest.TestCase):
 
@@ -594,7 +634,8 @@ def package_info(self):
         client.run("export . lasote/stable")
 
         # We can't build the package without our PYTHONPATH
-        self.assertRaises(Exception, client.run, "install conantool/1.0@lasote/stable --build missing")
+        self.assertRaises(Exception, client.run,
+                          "install conantool/1.0@lasote/stable --build missing")
 
         # But we can inject the PYTHONPATH
         client.run("install conantool/1.0@lasote/stable -e PYTHONPATH=['%s']" % external_dir)
@@ -648,4 +689,5 @@ def external_baz():
         client.save({CONANFILE: conanfile_simple})
         client.run("export . lasote/stable")
         # Should work even if PYTHONPATH is not declared as [], only external resource needed
-        client.run('install Hello/0.1@lasote/stable --build missing -e PYTHONPATH="%s"' % external_dir)
+        client.run('install Hello/0.1@lasote/stable --build missing -e PYTHONPATH="%s"'
+                   % external_dir)
diff --git a/conans/test/util/tools_test.py b/conans/test/util/tools_test.py
--- a/conans/test/util/tools_test.py
+++ b/conans/test/util/tools_test.py
@@ -1331,6 +1331,7 @@ def build(self):
         client.run("create . user/channel")
 
 
+@attr('git')
 class GitToolTest(unittest.TestCase):
 
     def test_repo_root(self):
@@ -1572,6 +1573,7 @@ def build(self):
         self.assertIn("specify a branch to checkout", client.out)
 
 
+@attr('svn')
 class SVNToolTestsBasic(SVNLocalRepoTestCase):
     def test_clone(self):
         project_url, _ = self.create_project(files={'myfile': "contents"})
@@ -1767,6 +1769,7 @@ def test_branch(self):
         self.assertEqual("tags/v12.3.4", svn.get_branch())
 
 
+@attr('svn')
 class SVNToolTestsBasicOldVersion(SVNToolTestsBasic):
     def run(self, *args, **kwargs):
         try:
@@ -1781,6 +1784,7 @@ def run(self, *args, **kwargs):
     # Do not add tests to this class, all should be compatible with new version of SVN
 
 
+@attr('svn')
 @unittest.skipUnless(SVN.get_version() >= SVN.API_CHANGE_VERSION, "SVN::is_pristine not implemented")
 class SVNToolTestsPristine(SVNLocalRepoTestCase):
 
@@ -1996,6 +2000,7 @@ def test_ignored_external(self):
         self.assertTrue(self.svn.is_pristine())
 
 
+@attr('svn')
 class SVNToolsTestsRecipe(SVNLocalRepoTestCase):
 
     conanfile = """