Repository: conan-io/conan
Pull request: Issue/5814 fix python_requires with short_paths enabled
Changelog: Bugfix: Use imported python requires' `short_path` value instead of the defined in the `conanfile` that imports it. 
Docs: omit

Closes #5814

- [X] Refer to the issue that supports this Pull Request.
- [X] If the issue has missing info, explain the purpose/use case/pain/need that covers this Pull Request.
- [X] I've read the [Contributing guide](https://github.com/conan-io/conan/blob/develop/.github/CONTRIBUTING.md).
- [X] I've followed the PEP8 style guides for Python code.
- [ ] I've opened another PR in the Conan docs repo to the ``develop`` branch, documenting this one. 

<sup>**Note:** By default this PR will skip the slower tests and will use a limited set of python versions. Check [here](https://github.com/conan-io/conan/blob/develop/.github/PR_INCREASE_TESTING.md) how to increase the testing level by writing some tags in the current PR body text.</sup>


Problem statement:
Use of python_requires with enabled short_paths fails
In Windows conan fails when a recipe uses `python_requires` and `short_paths = True` at the same time.

To reproduce the issue, it's enough to just add `short_paths = True` into the consumer conanfile of the `python_requires` example.

Conan will end with
```
ERROR: Error while trying to get recipe sources for pyreq/version@user/channel. No remote defined
```

Hints from issue discussion:
Hi @kdsx,
I was able to reproduce the issue when using `export_sources` in the `python_requires`
While we have a look at it, did you try to set the `short_paths = True` in the python requires as well?

Thanks a lot for the feedback
Hi @czoido,
I've already tried it. Unfortunately with no effect. Conan fails with the same message.
Hi!

I haven't been able to reproduce it, please @czoido share a unit test that can reproduce it.

That error message typically appears when a package is downloaded from a remote, then the remote is removed (and the necessary sources aren't there), so when the package need the sources it fails. It is not that directly related to short_paths, but to the remote removal. So any help with more detailed steps to reproduce will help. Thanks!
In my case it happened for a package which exists only in the local cache and has never been to a remote server. Then for simplicity I've reproduced it on the mentioned conan example. In contrast to the example my package does not use `exports_sources`, only `exports` which appear in the local cache in the same directory as `conanfile.py`, thus it should not require any additional sources to download.

I use the latest conan version at the moment (1.18.5) on Windows 10. I doubt it's important, but I run conan from cygwin environment, however conan itself installed by `pip` into `venv` generated by a native (non-cygwin) version of Python 3.
I've looked into code a bit and found that when conan installs the dependencies (including pyreq) it calls `_build_package` and then `complete_recipe_sources` for every dependency and the package itself. In the very beginning it tries to find the sources folder and then retrieve it if it is not available. The problem is probably here:
```
sources_folder = cache.package_layout(ref, conanfile.short_paths).export_sources()
```
It requests the source folder for the package `ref` (which is `pyreq`), but provides `short_paths` flag for the package `conanfile` which is `consumer`. Since `pyreq` does not have `shot_paths` flag, its sources were exported to the normal location, but `sources_folder` here points to a short path which even does not exist.

What is interesting, when I specify `short_paths` for `pyreq` too, it still does not work! When I create `pyreq` package, its `complete_recipe_sources` gets `conanfile.short_paths == False` and sources are still exported to their normal (non-short) location. Furthermore its "short" folder is created, but contains only `real_path.txt` which points to "long" folder. Thus when `complete_recipe_sources` is called from `consumer` context for `pyreq` it still can't find the sources directory.
Thanks a lot for reporting the bug @kdsx 
I have just made a PR that should fix the problem :)

Commit message: remove line

Patch to review:
diff --git a/conans/client/installer.py b/conans/client/installer.py
index 1bd24f6aab7..dac9c92a717 100644
--- a/conans/client/installer.py
+++ b/conans/client/installer.py
@@ -427,7 +427,7 @@ def _build_package(self, node, output, keep_build, remotes):
             assert python_require.ref.revision is not None, \
                 "Installer should receive python_require.ref always"
             complete_recipe_sources(self._remote_manager, self._cache,
-                                    conanfile, python_require.ref, remotes)
+                                    python_require.conanfile, python_require.ref, remotes)
 
         builder = _PackageBuilder(self._cache, output, self._hook_manager, self._remote_manager)
         pref = builder.build_package(node, keep_build, self._recorder, remotes)
diff --git a/conans/test/functional/python_requires/python_requires_test.py b/conans/test/functional/python_requires/python_requires_test.py
index aa238d65553..b96e2e8dfbf 100644
--- a/conans/test/functional/python_requires/python_requires_test.py
+++ b/conans/test/functional/python_requires/python_requires_test.py
@@ -576,6 +576,36 @@ class Lib(ConanFile):
         self.assertIn("Same python_requires with different versions not allowed for a conanfile",
                       t.out)
 
+    def short_paths_test(self):
+        # https://github.com/conan-io/conan/issues/5814
+        client = TestClient(servers={"default": TestServer()},
+                            users={"default": [("lasote", "mypass")]})
+        conanfile = """from conans import ConanFile
+class MyConanfileBase(ConanFile):
+    license = "MyLicense"
+    author = "author@company.com"
+    exports = "*.txt"
+    exports_sources = "*.h"
+    generators = "cmake"
+        """
+        client.save({"conanfile.py": conanfile,
+                     "file.h": "header",
+                     "other.txt": "text"})
+        client.run("create . Base/1.2@lasote/testing")
+
+        reuse = """from conans import python_requires
+base = python_requires("Base/1.2@lasote/testing")
+class PkgTest(base.MyConanfileBase):
+    short_paths = True
+    name = "consumer"
+    version = "1.0.0"
+    def build(self):
+        self.output.info("Package built successfully!")
+        """
+        client.save({"conanfile.py": reuse}, clean_first=True)
+        client.run("create . lasote/testing")
+        self.assertIn("Package built successfully!", client.out)
+
 
 class PythonRequiresNestedTest(unittest.TestCase):
 


Code review comments:
diff_hunk: @@ -576,6 +576,36 @@ class Lib(ConanFile):
         self.assertIn("Same python_requires with different versions not allowed for a conanfile",
                       t.out)
 
+    def short_paths_test(self):
+        # https://github.com/conan-io/conan/issues/5814
+        client = TestClient(servers={"default": TestServer()},
+                            users={"default": [("lasote", "mypass")]})
+        conanfile = """from conans import ConanFile
+class MyConanfileBase(ConanFile):
+    license = "MyLicense"
+    author = "author@company.com"
+    exports = "*.txt"
+    exports_sources = "*.h"
+    generators = "cmake"
+        """
+        client.save({"conanfile.py": conanfile,
+                     "file.h": "header",
+                     "other.txt": "text"})
+        client.run("create . Base/1.2@lasote/testing")
+
+        reuse = """from conans import python_requires
+base = python_requires("Base/1.2@lasote/testing")
+class PkgTest(base.MyConanfileBase):
+    short_paths = True
+    name = "consumer"
+    version = "1.0.0"
+    def build(self):
path: conans/test/functional/python_requires/python_requires_test.py
text: @user1:
Dont need build method, just use the normal output.

diff_hunk: @@ -576,6 +576,36 @@ class Lib(ConanFile):
         self.assertIn("Same python_requires with different versions not allowed for a conanfile",
                       t.out)
 
+    def short_paths_test(self):
+        # https://github.com/conan-io/conan/issues/5814
+        client = TestClient(servers={"default": TestServer()},
path: conans/test/functional/python_requires/python_requires_test.py
text: @user1:
use TestClient(default_server_user=True)

diff_hunk: @@ -576,6 +576,36 @@ class Lib(ConanFile):
         self.assertIn("Same python_requires with different versions not allowed for a conanfile",
                       t.out)
 
+    def short_paths_test(self):
+        # https://github.com/conan-io/conan/issues/5814
+        client = TestClient(servers={"default": TestServer()},
+                            users={"default": [("lasote", "mypass")]})
+        conanfile = """from conans import ConanFile
path: conans/test/functional/python_requires/python_requires_test.py
text: @user1:
Keep it simple, remove unused things:
- license, author, generator

Use textwrap.dedent() for cleaner layout