Repository: conan-io/conan

Issue:
[question] Using custom cross-compilation toolchains with requirements and CMakeDeps
### What is your question?

Updating to conan 2.3.2

I'm building packages via a Yocto based x-compilation toolchain, and the SDK generates this toolchain for us. It is vital to use it as it sets `CMAKE_FIND_ROOT_PATH` among other things, to find packages that are in the SDK and does not come from Conan. Therefore I override the generated toolchain with the `tools.cmake.cmaketoolchain:toolchain_file`  option.

However this poses a problem when we also have `requirements `statements for our Conan requirements. Conan generates the correct `<dep>-config.cmake` files for each requirement, but they end up under `.../build/Release/generators`. As we are using a custom toolchain, we have no way of knowing this path in beforehand, and therefore we dont know how to set the `CMAKE_MODULE_PATH`.

If I instead do NOT use the custom toolchain file, but rely on the generated one from Conan with CMakeToolchain, I find the requirement packages as the module path is calculated and added to the toolchain file. But then I get problems for packages in our cross-compilation SDK that we cannot find.

The solution we had in Conan 1.63 was to use the old syntax without toolchain etc, and using `cmake_find_package`. This worked, even with a custom toolchain file, because the `Find<dep>.cmake` files ended up in the build folder. As `cmake_find_package `is deprecated, I want to move away from this.

What is the best practice here?

### Have you read the CONTRIBUTING guide?

- [X] I've read the CONTRIBUTING guide


Discussion:
Hi @rollebolle 

Thanks for your question.

The ``self.generators_folder`` is known in the recipe, and it will point to the folder that you want.
What is the mechanism that you need to have this folder communicated to your CMake? You can add any arbitrary variables to the toolchain and presets like:
```python
def generate(self):
     tc = CMakeToolchain(self)
     gen_folder = self.generators_folder.replace("\\", "/")  # Windows-CMake not happy
     tc.variables["MYPATH_GEN"] = gen_folder   # IT will be a variable in conan_toolchain.cmake
     tc.cache_variables["MYPATH_GEN"] = gen_folder  # It will be a cache variables in CMakePresets (and passed in CLI arg)
     tc.generate()
```

Does this make sense for your use case?
Thanks. This does work and I I'll use it. However it means my recipe now has adapations that is only needed in the special case of supplying a toolchain from outside. Ideally I would like the recipe to be agnostic to this detail. As I wrote above, this was not needed with the old generators as the FindXXX  files happened to all end up in the build folder. Let me know if I misunderstood.
Quick question: 

> it sets CMAKE_FIND_ROOT_PATH among other things, to find packages that are in the SDK and does not come from Conan. Therefore I override the generated toolchain with the tools.cmake.cmaketoolchain:toolchain_file option.

Why don't you use the generated ``conan_toolchain.cmake`` and also your own one via the ``tools.cmake.cmaketoolchain:user_toolchain``? That appends your own toolchain, but also uses the Conan one, so that should work?
I tried that but get some nasty conflicts - I think in many advanced cases you want to have complete control of the toolchain file, as in our case. 
I have been thinking a bit about this problem, and honestly I don't know yet what would be the best approach.

I think it might be possible to try to find the generated files from your toolchain. This might be done for example by looking for the location of the ``conandeps_legacy.cmake`` that will be in the vast majority of cases in a subfolder of the current "build" folder, containing all the other generated files. It might be a bit fragile, because the ``conandeps_legacy.cmake`` might be removed in the future, as its name indicates, but it might work. The ``CMakePresets.json`` is also generated inside the ``generators_folder`` with the other files, and if there is a CMakeLists.txt in the root, the ``CMakeUserPreset.json`` file that is generated will have an ``include`` to the ``CMakePresets.json`` file. Using this, it seems quite feasible to obtain the ``generators_folder`` from your own toolchain file.

What would be the best mechanism to communicate the ``generators_folder``, which might be different in different packages (if all use the same ``cmake_layout()``, then it would be direct to deduce the location), to a user ``toolchain_file`` otherwise?

Please let me know if this helps.




Thanks for taking the time to think about this. Would it make sense for Conan to always append a valid  CMAKE_MODULE_PATH to CMakeUserPresets.json, so that even if we use a custom toolchain we can find the dependencies?

I don't really know the details around this, but  in Conan 2, a dependency on using the generated toolchains to be able to find any requirement packages was introduced. This in combination with the common cross-compile scenario where we want to use a custom toolchain file certainly makes it a bit more complicated to "get it right".
> Thanks for taking the time to think about this. Would it make sense for Conan to always append a valid CMAKE_MODULE_PATH to CMakeUserPresets.json, so that even if we use a custom toolchain we can find the dependencies?

Yes, this might be something worth exploring.

> I don't really know the details around this, but in Conan 2, a dependency on using the generated toolchains to be able to find any requirement packages was introduced. This in combination with the common cross-compile scenario where we want to use a custom toolchain file certainly makes it a bit more complicated to "get it right".

yes, it is true that the ``toolchain_file`` conf, fully replacing the toolchain, hasn't been that used so far, it seems the injection of toolchains via ``include()`` with ``user_toolchain`` conf is way more popular, because there is some important configuration that Conan knows already how to map, that would require an extra effort from users. That is, the mapping from Conan settings to CMake variables, for example to set the VS runtime, to select BUILD_SHARED_LIBS or not, enable/disable fPIC, etc, is a variability that otherwise the user ``toolchain_file`` would have to implement, most likely requiring different ``toolchain_file`` files, and that is more difficult to maintain that using the ``include()`` mechanism of ``user_toolchain`` and letting Conan complete the information to adjust to Conan settings.

But in any case, I understand the use case, let me discuss with the team, it sounds that to minimize risks we could add those variables when the ``toolchain_file`` is defined. I am assigning this to have a look for Conan 2.5, thanks for the feedback!

I am exploring the idea in https://github.com/conan-io/conan/pull/16455, but it is a draft, I am not very convinced, I might explore other approaches too.
I have been syncing with the team about https://github.com/conan-io/conan/pull/16455 we are still not fully convinced about the approach. We are going to explore the possibility of fully controlling the ``CMakeToolchain`` blocks from the outside or inject the toolchain at the end so it has higher priority.



Patch:
diff --git a/conan/tools/cmake/toolchain/blocks.py b/conan/tools/cmake/toolchain/blocks.py
index 5d1f340518d..1f4509cb0f0 100644
--- a/conan/tools/cmake/toolchain/blocks.py
+++ b/conan/tools/cmake/toolchain/blocks.py
@@ -1326,6 +1326,15 @@ def __getitem__(self, name):
         return self._blocks[name]
 
     def process_blocks(self):
+        blocks = self._conanfile.conf.get("tools.cmake.cmaketoolchain:enabled_blocks",
+                                          check_type=list)
+        if blocks is not None:
+            try:
+                new_blocks = OrderedDict((b, self._blocks[b]) for b in blocks)
+            except KeyError as e:
+                raise ConanException(f"Block {e} defined in tools.cmake.cmaketoolchain"
+                                     f":enabled_blocks doesn't exist in {list(self._blocks.keys())}")
+            self._blocks = new_blocks
         result = []
         for b in self._blocks.values():
             content = b.get_rendered_content()
diff --git a/conans/model/conf.py b/conans/model/conf.py
index 49a27c3ff17..1a36d24a5f9 100644
--- a/conans/model/conf.py
+++ b/conans/model/conf.py
@@ -74,6 +74,7 @@
     "tools.cmake.cmaketoolchain:toolset_cuda": "(Experimental) Path to a CUDA toolset to use, or version if installed at the system level",
     "tools.cmake.cmaketoolchain:presets_environment": "String to define wether to add or not the environment section to the CMake presets. Empty by default, will generate the environment section in CMakePresets. Can take values: 'disabled'.",
     "tools.cmake.cmaketoolchain:extra_variables": "Dictionary with variables to be injected in CMakeToolchain (potential override of CMakeToolchain defined variables)",
+    "tools.cmake.cmaketoolchain:enabled_blocks": "Select the specific blocks to use in the conan_toolchain.cmake",
     "tools.cmake.cmake_layout:build_folder_vars": "Settings and Options that will produce a different build folder and different CMake presets names",
     "tools.cmake.cmake_layout:build_folder": "(Experimental) Allow configuring the base folder of the build for local builds",
     "tools.cmake.cmake_layout:test_folder": "(Experimental) Allow configuring the base folder of the build for test_package",


Test patch:
diff --git a/test/unittests/tools/cmake/test_cmaketoolchain.py b/test/unittests/tools/cmake/test_cmaketoolchain.py
index 8ba5ff7f03b..3eabef9b4b5 100644
--- a/test/unittests/tools/cmake/test_cmaketoolchain.py
+++ b/test/unittests/tools/cmake/test_cmaketoolchain.py
@@ -95,6 +95,34 @@ def test_select_blocks(conanfile):
     assert "########## 'preprocessor' block #############" not in content
 
 
+def test_enabled_blocks_conf(conanfile):
+    conanfile.conf.define("tools.cmake.cmaketoolchain:enabled_blocks", ["generic_system"])
+    toolchain = CMakeToolchain(conanfile)
+    content = toolchain.content
+    assert "########## 'generic_system' block #############" in content
+    assert "########## 'cmake_flags_init' block #############" not in content
+    assert "########## 'libcxx' block #############" not in content
+    assert "########## 'variables' block #############" not in content
+    assert "########## 'preprocessor' block #############" not in content
+
+    # remove multiple
+    conanfile.conf.define("tools.cmake.cmaketoolchain:enabled_blocks",
+                          ["generic_system", "cmake_flags_init"])
+    toolchain = CMakeToolchain(conanfile)
+    content = toolchain.content
+    assert "########## 'generic_system' block #############" in content
+    assert "########## 'cmake_flags_init' block #############" in content
+    assert "########## 'libcxx' block #############" not in content
+    assert "########## 'variables' block #############" not in content
+    assert "########## 'preprocessor' block #############" not in content
+
+    conanfile.conf.define("tools.cmake.cmaketoolchain:enabled_blocks", ["potato"])
+    toolchain = CMakeToolchain(conanfile)
+    with pytest.raises(ConanException) as e:
+        _ = toolchain.content
+    assert "Block 'potato' defined in tools.cmake.cmaketoolchain:enabled_blocks doesn't" in str(e)
+
+
 def test_dict_keys(conanfile):
     toolchain = CMakeToolchain(conanfile)
     assert "generic_system" in toolchain.blocks.keys()