Repository: conan-io/conan

Issue:
[bug] `conan test` doesn't work with python_requires packages
### Environment details

* Operating System+version: Arch Linux
* Compiler+version: clang 16
* Conan version: 2.0.17
* Python version: 3.11.1


### Steps to reproduce

Using the following files:

* `conanfile.py`

```python
from conan import ConanFile

class FuncsPackage(ConanFile):
    name = "funcs"
    version = "0.1.0"
    package_type = "python-require"

def func_one(args):
    if args:
        return "Args passed"
    return "No arguments"
```

* `test_package/conanfile.py`

```python
from conan import ConanFile
from conan.tools.layout import basic_layout
from conan.tools.build import cross_building

class PyvenvTestConan(ConanFile):
    settings = "os", "build_type", "arch", "compiler"
    apply_env = False
    test_type = "explicit"
    build_policy = "missing"

    def config_options(self):
        del self.settings.build_type
        del self.settings.arch
        del self.settings.compiler

    def layout(self):
        basic_layout(self)

    def test(self):
        if not cross_building(self):
            func_one = self.python_requires["funcs"].module.func_one
            result = func_one("These are the arguments")
            self.run("echo 'Result: {}'".format(result), env="conanrun")
```

1. Execute `conan create . --user user` - this will work correctly
2. Execute `conan test test_package funcs/0.1.0@user`, the below error message will be thrown.

### Logs

```bash
$ conan create . --user user

======== Exporting recipe to the cache ========
funcs/0.1.0@user: Exporting package recipe: /tmp/pyrequires/conanfile.py
funcs/0.1.0@user: Copied 1 '.py' file: conanfile.py
funcs/0.1.0@user: Exported to cache folder: /home/user/.conan2/p/funcs9df5001c34428/e
funcs/0.1.0@user: Exported: funcs/0.1.0@user#8fb741c2645c0dd35d1db5c5b7cc49c3 (2024-01-20 01:38:45 UTC)

======== Input profiles ========
Profile host:
[settings]
arch=x86_64
build_type=Release
compiler=clang
compiler.cppstd=20
compiler.libcxx=libstdc++11
compiler.version=16
os=Linux
[conf]
tools.build:compiler_executables={'c': '/usr/bin/clang', 'cpp': '/usr/bin/clang++'}

Profile build:
[settings]
arch=x86_64
build_type=Release
compiler=clang
compiler.cppstd=20
compiler.libcxx=libstdc++11
compiler.version=16
os=Linux
[conf]
tools.build:compiler_executables={'c': '/usr/bin/clang', 'cpp': '/usr/bin/clang++'}


======== Computing dependency graph ========

======== Launching test_package ========

======== Computing dependency graph ========
Graph root
    funcs/0.1.0@user (test package): /tmp/pyrequires/test_package/conanfile.py
Python requires
    funcs/0.1.0@user#8fb741c2645c0dd35d1db5c5b7cc49c3 - Cache

======== Computing necessary packages ========

======== Installing packages ========

======== Testing the package ========
Removing previously existing 'test_package' build folder: /tmp/pyrequires/test_package/build
funcs/0.1.0@user (test package): Test package build: build
funcs/0.1.0@user (test package): Test package build folder: /tmp/pyrequires/test_package/build
funcs/0.1.0@user (test package): Generating aggregated env files
funcs/0.1.0@user (test package): Generated aggregated env files: ['conanbuild.sh', 'conanrun.sh']

======== Testing the package: Building ========

======== Testing the package: Executing test ========
funcs/0.1.0@user (test package): Running test()
funcs/0.1.0@user (test package): RUN: echo 'Result: Args passed'
Result: Args passed
```

```bash
$ conan test test_package funcs/0.1.0@user
======== Input profiles ========
Profile host:
[settings]
arch=x86_64
build_type=Release
compiler=clang
compiler.cppstd=20
compiler.libcxx=libstdc++11
compiler.version=16
os=Linux
[conf]
tools.build:compiler_executables={'c': '/usr/bin/clang', 'cpp': '/usr/bin/clang++'}

Profile build:
[settings]
arch=x86_64
build_type=Release
compiler=clang
compiler.cppstd=20
compiler.libcxx=libstdc++11
compiler.version=16
os=Linux
[conf]
tools.build:compiler_executables={'c': '/usr/bin/clang', 'cpp': '/usr/bin/clang++'}


======== Launching test_package ========

======== Computing dependency graph ========
Graph root
    funcs/0.1.0@user (test package): /tmp/pyrequires/test_package/conanfile.py

======== Computing necessary packages ========

======== Installing packages ========

======== Testing the package ========
ERROR: The conanfile at '/tmp/pyrequires/test_package/conanfile.py' doesn't declare any requirement, use `self.tested_reference_str` to require the package being created.
```


Discussion:
Additionally, if I modify the `test_package/conanfile.py` to include `tested_reference_str` (in this only way I could work out how to - this works with a `conan create` command):

```diff
diff --git a/test_package/conanfile.py b/test_package/conanfile.py
index c4cc2cc..4f6092b 100644
--- a/test_package/conanfile.py
+++ b/test_package/conanfile.py
@@ -7,6 +7,7 @@ class PyvenvTestConan(ConanFile):
     settings = "os", "build_type", "arch", "compiler"
     apply_env = False
     test_type = "explicit"
+    python_requires = ConanFile.tested_reference_str
 
     def config_options(self):
         del self.settings.build_type
```

This gives the following error:

```bash
$ conan test test_package funcs/0.1.0@user

======== Input profiles ========
Profile host:
[settings]
arch=x86_64
build_type=Release
compiler=clang
compiler.cppstd=20
compiler.libcxx=libstdc++11
compiler.version=16
os=Linux
[conf]
tools.build:compiler_executables={'c': '/usr/bin/clang', 'cpp': '/usr/bin/clang++'}

Profile build:
[settings]
arch=x86_64
build_type=Release
compiler=clang
compiler.cppstd=20
compiler.libcxx=libstdc++11
compiler.version=16
os=Linux
[conf]
tools.build:compiler_executables={'c': '/usr/bin/clang', 'cpp': '/usr/bin/clang++'}


======== Launching test_package ========

======== Computing dependency graph ========
ERROR: Traceback (most recent call last):
  File "/home/user/.local/pipx/venvs/conan/lib/python3.11/site-packages/conan/cli/cli.py", line 272, in main
    cli.run(args)
  File "/home/user/.local/pipx/venvs/conan/lib/python3.11/site-packages/conan/cli/cli.py", line 172, in run
    command.run(self._conan_api, args[0][1:])
  File "/home/user/.local/pipx/venvs/conan/lib/python3.11/site-packages/conan/cli/command.py", line 125, in run
    info = self._method(conan_api, parser, *args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/user/.local/pipx/venvs/conan/lib/python3.11/site-packages/conan/cli/commands/test.py", line 40, in test
    deps_graph = run_test(conan_api, path, ref, profile_host, profile_build, remotes, lockfile,
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/user/.local/pipx/venvs/conan/lib/python3.11/site-packages/conan/cli/commands/test.py", line 67, in run_test
    print_graph_basic(deps_graph)
  File "/home/user/.local/pipx/venvs/conan/lib/python3.11/site-packages/conan/cli/printers/graph.py", line 20, in print_graph_basic
    for r in node.conanfile.python_requires._pyrequires.values():  # TODO: improve interface
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: 'NoneType' object has no attribute '_pyrequires'

ERROR: 'NoneType' object has no attribute '_pyrequires'

```
using

```diff
diff --git a/test_package/conanfile.py b/test_package/conanfile.py
index c4cc2cc..640bf0b 100644
--- a/test_package/conanfile.py
+++ b/test_package/conanfile.py
@@ -7,6 +7,7 @@ class PyvenvTestConan(ConanFile):
     settings = "os", "build_type", "arch", "compiler"
     apply_env = False
     test_type = "explicit"
+    python_requires = self.tested_reference_str
 
     def config_options(self):
         del self.settings.build_type
```

Does not work with `conan create`, and gives:

```bash
$ conan test test_package funcs/0.1.0@user

======== Input profiles ========
Profile host:
[settings]
arch=x86_64
build_type=Release
compiler=clang
compiler.cppstd=20
compiler.libcxx=libstdc++11
compiler.version=16
os=Linux
[conf]
tools.build:compiler_executables={'c': '/usr/bin/clang', 'cpp': '/usr/bin/clang++'}

Profile build:
[settings]
arch=x86_64
build_type=Release
compiler=clang
compiler.cppstd=20
compiler.libcxx=libstdc++11
compiler.version=16
os=Linux
[conf]
tools.build:compiler_executables={'c': '/usr/bin/clang', 'cpp': '/usr/bin/clang++'}

ERROR: Error loading conanfile at '/tmp/pyrequires/test_package/conanfile.py': Unable to load conanfile in /tmp/pyrequires/test_package/conanfile.py
  File "<frozen importlib._bootstrap_external>", line 940, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/tmp/pyrequires/test_package/conanfile.py", line 6, in <module>
    class PyvenvTestConan(ConanFile):
  File "/tmp/pyrequires/test_package/conanfile.py", line 10, in PyvenvTestConan
    python_requires = self.tested_reference_str
                      ^^^^
NameError: name 'self' is not defined

```
Hi @samuel-emrys 

Thanks for the report.

The ``python_requires`` is injected, cannot be defined in ``test_package/conanfile.py`` recipe, because they work at the class level, so ``self.`` wont work here. This is why it only works with ``conan create`` but not with the independent ``conan test`` command. I am having a look to see if it is possible to avoid this issue.
Thanks @memsharded. Would it be feasible to just avoid raising when there are no `requires` or `tool_requires`, but still a `python_requires` statement?

```
ERROR: The conanfile at '/tmp/pyrequires/test_package/conanfile.py' doesn't declare any requirement, use `self.tested_reference_str` to require the package being created.
```
> Thanks @memsharded. Would it be feasible to just avoid raising when there are no requires or tool_requires, but still a python_requires statement?

But hardcoding the ``python_requires`` is also inconvenient, it has to change for every version, etc. I am exploring other possibilities now. The thing is that there is missing context for ``conan test`` command, I am trying to add it explicitly.
Proposing https://github.com/conan-io/conan/pull/15485, seems explicit better than implicit, but can't guarantee, we need to make sure we don't break
Yes, agree. As someone who doesn't look at the mechanics of how this works super frequently, being explicit is better to immediately understand it. After not looking at my python requires recipes for a few months, it took me some time to realise that I didn't need to specify the `python_requires` explicitly for it to be available to the test package - it was very confusing initially.



Patch:
diff --git a/conan/cli/commands/create.py b/conan/cli/commands/create.py
index a81d99ce2cc..3e08b77c892 100644
--- a/conan/cli/commands/create.py
+++ b/conan/cli/commands/create.py
@@ -9,7 +9,7 @@
 from conan.cli.printers import print_profiles
 from conan.cli.printers.graph import print_graph_packages, print_graph_basic
 from conan.errors import ConanException
-from conans.util.files import mkdir
+from conans.util.files import mkdir, load
 
 
 @conan_command(group="Creator", formatters={"json": format_graph_json})
@@ -98,7 +98,9 @@ def create(conan_api, parser, *args):
     if test_conanfile_path:
         # TODO: We need arguments for:
         #  - decide update policy "--test_package_update"
-        tested_python_requires = ref.repr_notime() if is_python_require else None
+        # If it is a string, it will be injected always, if it is a RecipeReference, then it will
+        # be replaced only if ``python_requires = "tested_reference_str"``
+        tested_python_requires = ref.repr_notime() if is_python_require else ref
         from conan.cli.commands.test import run_test
         # The test_package do not make the "conan create" command return a different graph or
         # produce a different lockfile. The result is always the same, irrespective of test_package
@@ -124,16 +126,16 @@ def _check_tested_reference_matches(deps_graph, tested_ref, out):
                     "tested is '{}'".format(missmatch[0], tested_ref))
 
 
-def test_package(conan_api, deps_graph, test_conanfile_path, tested_python_requires=None):
+def test_package(conan_api, deps_graph, test_conanfile_path):
     out = ConanOutput()
     out.title("Testing the package")
     # TODO: Better modeling when we are testing a python_requires
-    if len(deps_graph.nodes) == 1 and not tested_python_requires:
+    conanfile = deps_graph.root.conanfile
+    if len(deps_graph.nodes) == 1 and not hasattr(conanfile, "python_requires"):
         raise ConanException("The conanfile at '{}' doesn't declare any requirement, "
                              "use `self.tested_reference_str` to require the "
                              "package being created.".format(test_conanfile_path))
     conanfile_folder = os.path.dirname(test_conanfile_path)
-    conanfile = deps_graph.root.conanfile
     # To make sure the folders are correct
     conanfile.folders.set_base_folders(conanfile_folder, output_folder=None)
     if conanfile.build_folder and conanfile.build_folder != conanfile.source_folder:
diff --git a/conans/client/loader.py b/conans/client/loader.py
index 28be532d387..66086f71197 100644
--- a/conans/client/loader.py
+++ b/conans/client/loader.py
@@ -54,7 +54,10 @@ def load_basic_module(self, conanfile_path, graph_lock=None, display="", remotes
 
         try:
             module, conanfile = _parse_conanfile(conanfile_path)
-            if tested_python_requires:
+            if isinstance(tested_python_requires, RecipeReference):
+                if getattr(conanfile, "python_requires", None) == "tested_reference_str":
+                    conanfile.python_requires = tested_python_requires.repr_notime()
+            elif tested_python_requires:
                 conanfile.python_requires = tested_python_requires
 
             if self._pyreq_loader:


Test patch:
diff --git a/conan/cli/commands/test.py b/conan/cli/commands/test.py
index d8f2093fb78..eda4ebafc75 100644
--- a/conan/cli/commands/test.py
+++ b/conan/cli/commands/test.py
@@ -38,7 +38,7 @@ def test(conan_api, parser, *args):
     print_profiles(profile_host, profile_build)
 
     deps_graph = run_test(conan_api, path, ref, profile_host, profile_build, remotes, lockfile,
-                          args.update, build_modes=args.build)
+                          args.update, build_modes=args.build, tested_python_requires=ref)
     lockfile = conan_api.lockfile.update_lockfile(lockfile, deps_graph, args.lockfile_packages,
                                                   clean=args.lockfile_clean)
     conan_api.lockfile.save_lockfile(lockfile, args.lockfile_out, cwd)
@@ -55,6 +55,11 @@ def run_test(conan_api, path, ref, profile_host, profile_build, remotes, lockfil
                                                          update=update,
                                                          lockfile=lockfile,
                                                          tested_python_requires=tested_python_requires)
+    if isinstance(tested_python_requires, str):  # create python-require
+        conanfile = root_node.conanfile
+        if not getattr(conanfile, "python_requires", None) == "tested_reference_str":
+            ConanOutput().warning("test_package/conanfile.py should declare 'python_requires"
+                                  " = \"tested_reference_str\"'", warn_tag="deprecated")
 
     out = ConanOutput()
     out.title("Launching test_package")
@@ -74,5 +79,5 @@ def run_test(conan_api, path, ref, profile_host, profile_build, remotes, lockfil
 
     conan_api.install.install_binaries(deps_graph=deps_graph, remotes=remotes)
     _check_tested_reference_matches(deps_graph, ref, out)
-    test_package(conan_api, deps_graph, path, tested_python_requires)
+    test_package(conan_api, deps_graph, path)
     return deps_graph
diff --git a/conans/test/integration/py_requires/python_requires_test.py b/conans/test/integration/py_requires/python_requires_test.py
index ef86ee11765..9b2342bef17 100644
--- a/conans/test/integration/py_requires/python_requires_test.py
+++ b/conans/test/integration/py_requires/python_requires_test.py
@@ -1013,6 +1013,7 @@ class Common(ConanFile):
             from conan import ConanFile
 
             class Tool(ConanFile):
+                python_requires = "tested_reference_str"
                 def test(self):
                     self.output.info("{}!!!".format(self.python_requires["common"].module.mycommon()))
             """)
@@ -1021,6 +1022,9 @@ def test(self):
         c.run("create .")
         assert "common/0.1 (test package): 42!!!" in c.out
 
+        c.run("test test_package common/0.1")
+        assert "common/0.1 (test package): 42!!!" in c.out
+
     def test_test_package_python_requires_configs(self):
         """ test how to test_package a python_require with various configurations
         """
@@ -1048,6 +1052,7 @@ def test(self):
                 "test_package/conanfile.py": test})
         c.run("create . ")
         assert "common/0.1 (test package): RELEASEOK!!!" in c.out
+        assert "WARN: deprecated: test_package/conanfile.py should declare 'python_requires" in c.out
         c.run("create . -s build_type=Debug")
         assert "common/0.1 (test package): DEBUGOK!!!" in c.out