Repository: conan-io/conan
Pull request: Multiple CMake User Toolchain
Close #9724

Changelog: Feature: The `CMakeToolchain` is now prepared to apply several user toolchains.
Docs: https://github.com/conan-io/docs/pull/2340


Problem statement:
[feature] How CMakeToolchain could handle multiple user toolchains
Lets say that we have 2 different build_requires, one ``sdk`` and one ``compiler``. Building a package could use both, and each one would want to inject their own ``user_toolchain.cmake``. 

Or they want to propagate information to the consumer in other ways. Challenges:

- Passing information like CC/CXX via environment requires using VirtualBuildEnv, and activating env scripts, which is not the most convenient for IDE developers. Env-vars might not be enough for some cases, and still a user toolchain might be necessary.
- Passing information via ``self.conf_info`` to consumer can work, but at the moment can only provide 1 value for ``user_toolchain``. More than one value would require list aggregation or equivalent functionality

cc/ @Blejdfist

Hints from issue discussion:
The ability to provide multiple files to be included in the `UserToolchain` block when using the `CMakeToolchain` generator would provide a lot of flexibility.
I am thinking that we might need a bit more of power in the ``Conf`` class to handle some scenarios. So far the ``Conf`` class is a pure ``key=value`` store, very simple and convenient. But there are current ``conan.conf`` cases that might not be covered:

- The list of hooks to apply
- If we want to evolve it and be able to pass general compilation flags, like CXX_FLAGS to the whole build.
- To be able to aggregate information coming from multiple ``build_requires``, in the consumer.

This would mean in practice that ``Conf`` would be an instance of the ``Environment`` class (and kind o makes sense now that we removed the dependency of ``Environment`` to ``conanfile`` instance, becoming fully abstract). Some questions:
- Should we evolve ``Conf`` to be equal or very similar to ``Environment``?
- Should it be an ``Environment`` instance? What is the overlap now with ``[buildenv]`` profile section?
- Does it make sense to have an ``Environment`` with "un-realizable" env-vars, like ``tools.cmake.cmake_toolchain:xxxx``?

Wdyt @lasote?
Lets try this approach of using something like ``Environment`` to extend the capabilities of current ``Conf``
We will try the following approach:

- Make explicit in a generator like ``ConfAggregator`` the behavior of accummulating conf from build-requires
- This generator can be explicitly instantiated to customize priorities and behavior
- The generator will be enabled in 1.X with ``auto_use`` as VirtualBuildEnv and automatic in 2.0
- CMakeToolchain will listen to that property as a potential list (both for profile, and from deps, agnostic in this point, just move the list to a list of ``include(xxxx)``)

Commit message: Removed ordered generators

Patch to review:
diff --git a/conan/tools/cmake/toolchain.py b/conan/tools/cmake/toolchain.py
index 52c8c1b130d..832e0251b64 100644
--- a/conan/tools/cmake/toolchain.py
+++ b/conan/tools/cmake/toolchain.py
@@ -1,3 +1,4 @@
+import json
 import os
 import re
 import textwrap
@@ -457,20 +458,22 @@ def context(self):
 
 class UserToolchain(Block):
     template = textwrap.dedent("""
-        {% if user_toolchain %}
+        {% for user_toolchain in paths %}
         include("{{user_toolchain}}")
-        {% endif %}
+        {% endfor %}
         """)
 
-    user_toolchain = None
+    user_toolchains = None
 
     def context(self):
-        # This is global [conf] injection of extra toolchain files
-        user_toolchain = self._conanfile.conf["tools.cmake.cmaketoolchain:user_toolchain"]
-        user_toolchain = user_toolchain or self.user_toolchain
-        if user_toolchain:
-            user_toolchain = user_toolchain.replace("\\", "/")
-        return {"user_toolchain": user_toolchain}
+        # Priority "self.user_toolchains", otherwise it will take a single conf value if set
+        toolchains = self.user_toolchains
+        if not toolchains:
+            # This is global [conf] injection of extra toolchain files
+            user_toolchain = self._conanfile.conf["tools.cmake.cmaketoolchain:user_toolchain"]
+            toolchains = [user_toolchain.replace("\\", "/")] if user_toolchain else []
+
+        return {"paths": toolchains if toolchains else []}
 
 
 class CMakeFlagsInitBlock(Block):
diff --git a/conans/test/functional/toolchains/cmake/test_cmake_toolchain.py b/conans/test/functional/toolchains/cmake/test_cmake_toolchain.py
index bd878beaf51..054483c8e46 100644
--- a/conans/test/functional/toolchains/cmake/test_cmake_toolchain.py
+++ b/conans/test/functional/toolchains/cmake/test_cmake_toolchain.py
@@ -115,3 +115,69 @@ def test_cmake_toolchain_without_build_type():
     toolchain = client.load("conan_toolchain.cmake")
     assert "CMAKE_MSVC_RUNTIME_LIBRARY" not in toolchain
     assert "CMAKE_BUILD_TYPE" not in toolchain
+
+
+@pytest.mark.parametrize("with_context", ["False", "True"])
+def test_cmake_toolchain_multiple_user_toolchain(with_context):
+    """ A consumer consuming two packages that declare:
+            self.conf_info["tools.cmake.cmaketoolchain:user_toolchain"]
+        The consumer wants to use apply both toolchains in the CMakeToolchain.
+        There are two ways to customize the CMakeToolchain (parametrized):
+                1. Altering the context of the block (with_context = True)
+                2. Using the t.blocks["user_toolchain"].user_toolchains = [] (with_context = False)
+    """
+    client = TestClient()
+    conanfile = textwrap.dedent("""
+        import os
+        from conans import ConanFile
+        class Pkg(ConanFile):
+            exports_sources = "*"
+            def package(self):
+                self.copy("*")
+            def package_info(self):
+                f = os.path.join(self.package_folder, "mytoolchain.cmake")
+                self.conf_info["tools.cmake.cmaketoolchain:user_toolchain"] = f
+        """)
+    client.save({"conanfile.py": conanfile,
+                 "mytoolchain.cmake": 'message(STATUS "mytoolchain1.cmake !!!running!!!")'})
+    client.run("create . toolchain1/0.1@")
+    client.save({"conanfile.py": conanfile,
+                 "mytoolchain.cmake": 'message(STATUS "mytoolchain2.cmake !!!running!!!")'})
+    client.run("create . toolchain2/0.1@")
+
+    conanfile = textwrap.dedent("""
+        from conans import ConanFile
+        from conan.tools.cmake import CMake, CMakeToolchain
+        class Pkg(ConanFile):
+            settings = "os", "compiler", "arch", "build_type"
+            exports_sources = "CMakeLists.txt"
+            tool_requires = "toolchain1/0.1", "toolchain2/0.1"
+
+
+            def generate(self):
+                # Get the toolchains from "tools.cmake.cmaketoolchain:user_toolchain" conf at the
+                # tool_requires
+                user_toolchains = []
+                for dep in self.dependencies.direct_build.values():
+                    ut = dep.conf_info["tools.cmake.cmaketoolchain:user_toolchain"]
+                    if ut:
+                        user_toolchains.append(ut)
+
+                # Modify the context of the user_toolchain block
+                t = CMakeToolchain(self)
+                if {}:
+                    t.blocks["user_toolchain"].values["paths"] = user_toolchains
+                else:
+                    t.blocks["user_toolchain"].user_toolchains = user_toolchains
+                t.generate()
+
+            def build(self):
+                cmake = CMake(self)
+                cmake.configure()
+        """).format(with_context)
+
+    client.save({"conanfile.py": conanfile,
+                 "CMakeLists.txt": gen_cmakelists()}, clean_first=True)
+    client.run("create . pkg/0.1@")
+    assert "mytoolchain1.cmake !!!running!!!" in client.out
+    assert "mytoolchain2.cmake !!!running!!!" in client.out
diff --git a/conans/test/unittests/tools/cmake/test_cmaketoolchain.py b/conans/test/unittests/tools/cmake/test_cmaketoolchain.py
index 4a0032d95a6..1711c530f3a 100644
--- a/conans/test/unittests/tools/cmake/test_cmaketoolchain.py
+++ b/conans/test/unittests/tools/cmake/test_cmaketoolchain.py
@@ -134,7 +134,8 @@ def context(self):
 
 def test_user_toolchain(conanfile):
     toolchain = CMakeToolchain(conanfile)
-    toolchain.blocks["user_toolchain"].user_toolchain = "myowntoolchain.cmake"
+    toolchain.blocks["user_toolchain"].user_toolchains = ["myowntoolchain.cmake"]
+    # toolchain.blocks["user_toolchain"].values["paths"] = ["myowntoolchain.cmake"]
     content = toolchain.content
     assert 'include("myowntoolchain.cmake")' in content
 


Code review comments:
diff_hunk: @@ -115,3 +115,69 @@ def test_cmake_toolchain_without_build_type():
     toolchain = client.load("conan_toolchain.cmake")
     assert "CMAKE_MSVC_RUNTIME_LIBRARY" not in toolchain
     assert "CMAKE_BUILD_TYPE" not in toolchain
+
+
+@pytest.mark.parametrize("with_context", ["False", "True"])
+def test_cmake_toolchain_multiple_user_toolchain(with_context):
+    """ A consumer consuming two packages that declare:
+            self.conf_info["tools.cmake.cmaketoolchain:user_toolchain"]
+        The consumer wants to use apply both toolchains in the CMakeToolchain.
+        There are two ways to customize the CMakeToolchain (parametrized):
+                1. Altering the context of the block (with_context = True)
+                2. Using the t.blocks["user_toolchain"].user_toolchains = [] (with_context = False)
+    """
+    client = TestClient()
+    conanfile = textwrap.dedent("""
+        import os
+        from conans import ConanFile
+        class Pkg(ConanFile):
+            exports_sources = "*"
+            def package(self):
+                self.copy("*")
+            def package_info(self):
+                f = os.path.join(self.package_folder, "mytoolchain.cmake")
+                self.conf_info["tools.cmake.cmaketoolchain:user_toolchain"] = f
+        """)
+    client.save({"conanfile.py": conanfile,
+                 "mytoolchain.cmake": 'message(STATUS "mytoolchain1.cmake !!!running!!!")'})
+    client.run("create . toolchain1/0.1@")
+    client.save({"conanfile.py": conanfile,
+                 "mytoolchain.cmake": 'message(STATUS "mytoolchain2.cmake !!!running!!!")'})
+    client.run("create . toolchain2/0.1@")
+
+    conanfile = textwrap.dedent("""
+        from conans import ConanFile
+        from conan.tools.cmake import CMake, CMakeToolchain
+        class Pkg(ConanFile):
+            settings = "os", "compiler", "arch", "build_type"
+            exports_sources = "CMakeLists.txt"
+            tool_requires = "toolchain1/0.1", "toolchain2/0.1"
+
+
+            def generate(self):
+                # Get the toolchains from "tools.cmake.cmaketoolchain:user_toolchain" conf at the
+                # tool_requires
+                user_toolchains = []
+                for dep in self.dependencies.direct_build.values():
+                    ut = dep.conf_info["tools.cmake.cmaketoolchain:user_toolchain"]
+                    if ut:
+                        user_toolchains.append(ut)
+
+                # Modify the context of the user_toolchain block
+                t = CMakeToolchain(self)
+                if {}:
+                    t.blocks["user_toolchain"].values["paths"] = user_toolchains
path: conans/test/functional/toolchains/cmake/test_cmake_toolchain.py
text: @user1:
Maybe if the block has a explicit ``.user_toolchains`` attribute, the generic context one doesn't need to be documented (or even tested?)

@author:
My point is, you can always do that because of the blocks, right? even if it is not documented you can still watch it, see the key, and alter it.
So, Is it worth it to have the `.user_toolchains` or just the context? As I said in the PR description, I don't think we should keep both.

@user1:
Oh, sorry, I missed the updated description of the PR. 
Yes, I agree we should probably do only 1 way to do this. As the context one is very built-in and cannot be removed, maybe we just want to remove the ``.user_toolchains`` attribute. The truth is that those attributes were added before we had the ``block.values`` helper, that greatly simplify to modify contexts.