Repository: conan-io/conan
Pull request: Feature/cmakedeps legacy adapter
Changelog: Feature: Make ``CMakeDeps`` generator create a ``conandeps.cmake`` that aggregates all direct dependencies in a ``cmake``-like generator style.
Docs: https://github.com/conan-io/docs/pull/3492


Close https://github.com/conan-io/conan/issues/14623

Problem statement:
[feature] Bring back CMake targets generator for Conan 2
### What is your suggestion?

The new way of injecting dependency information into CMake is a lot more verbose, complex and error prone on the CMake side than conan 1's `conan_basic_setup(TARGETS)`.

1. Need to use find_package for each dependency instead of a single include. (more verbose)
2. The name in find_package is often slightly different, e.g. ZLIB (CMake) instead of zlib (conan). (error prone, complex)
3. The find scripts often do things in a non package-manager friendly way via extra options, e.g. boosts autolinking option. Conan 1's way neatly sidestepped that unnecessary complexity. (error prone, complex, verbose)
4. find_package falls back to looking for system libs silently if not configured correctly (error prone, complex)

I'd very much like to have an option to import targets the conan 1 way with conan 2, be it as an option or a separate generator. I can see how the new way works better for 'unintrusive' conan support in libraries. But I maintain several applications that are specifically built on conan with 10-20 dependencies that are a huge pain to port to conan 2 right now.

### Have you read the CONTRIBUTING guide?

- [ ] I've read the CONTRIBUTING guide

Hints from issue discussion:
Hi @ltjax 

Thanks for your feedback.

Conan decided to follow the ``CMakeDeps`` "transparent integration" approach in Conan 1.X (this is not exclusive of Conan 2.0, the migration to ``CMakeDeps`` should be done while in Conan 1.X, then move to 2.0), based on massive feedback against the non-transparent integration of the legacy ``cmake`` integration.

> Need to use find_package for each dependency instead of a single include. (more verbose)

It is not more verbose, it is standard, CMake recommended and Conan-agnostic way to do it. The idea is that it is not more verbose because it is not necessary to modify your existing ``CMakeLists.txt`` at all, using the standard existing ``find_package()`` calls

> The name in find_package is often slightly different, e.g. ZLIB (CMake) instead of zlib (conan). (error prone, complex)

Another massive push of the community, to have target names that matches the "upstream" target names (the one provided by the original projects or CMake). This also means that you can use Conan with existing projects without changing a single line of your ``CMakeLists.txt``


Then, there was a thing that was also super confusing for users, is having more than 1, different integrations. That was causing lots of questions, problems, support tickets, etc. People was complaining in social, reddit, etc, because Conan CMake integration was intrusive.

Finally, there is the issue that resources are not infinite. Every generator requires a lot of development, testing, documentation, support and further maintenance along years. We simply cannot have 2 different CMake integrations, it is too much effort. So we had to decide in one direction, and it was the transparent, standard CMake ``find_package()`` integration. We understand that some users preferred the explicit, custom Conan targets one, actually I liked it, but it is impossible in practice to satisfy those 2 radically opposite approaches with the resources we have.

All the necessary mechanisms for users to use custom generators are there, from using ``python-requires``, to global custom generators in the cache.

> But I maintain several applications that are specifically built on conan with 10-20 dependencies that are a huge pain to port to conan 2 right now.

in ConanCenter we had to migrate many hundreds of recipes to the new generators, we can understand the pain to migrate recipes. Still, this upgrade is independent of Conan 2.0 migration, this should happen first, (it can be done gradually, one at a time) in Conan 1.X, and once things are ready, migrating to Conan 2.0 will be way easier. We are willing to help as much as possible in the migration, don't hesitate to ask as many questions and report issues as you need.

Thanks very much for your feedback!

I might not have been explicit enough what my use case is: I'm mostly (but not exclusively) building applications that are consuming conan packages. Most of them even just use a `conanfile.txt`, not a `conanfile.py`. When I add a dependency, I have to change the CMake and the C++ code anyways to use it, so it is intrusive anyways. I want my builds to be deterministic (which `find_package` is horrible for) and I prefer to have less code and less complexity to maintain. I was very happy with the old generator.

> It is not more verbose, it is standard, CMake recommended and Conan-agnostic way to do it. The idea is that it is not more verbose because it is not necessary to modify your existing `CMakeLists.txt` at all, using the standard existing `find_package()` calls

I don't see how 'being standard' makes it not more verbose. Less code makes it less verbose, and the old way was typically less code to write/read and less code executed. I just had to add the dependency to my `conanfile.txt` and my `target_link_libraries`. Now I need `find_package`s and an additional translation step too.
Again, I can see how all this does makes sense from a recipe-maintainer's perspective, but it does not for my application developer's.

The targets generator does not even have to be another generator at all. Maybe it can be done via one or two options to CMakeDeps? 



Thanks for your feedback.

I totally understand you, I created that ``cmake`` generator, I was an advocate of that integration and I think it had advantages, and I think you make valid points about the verbosity of the code. However, I'm afraid this approach was not necessarily well received by the vast majority of the wider C++ community as it made adopting Conan more challenging for most existing projects, and would have caused new projects to converge to a Conan-only approach, with many projects and companies not willing to take this risk while Conan was in its infancy. Modifying the ``CMakeLists.txt`` to integrate Conan was a blocker and deal-breaker for the largest part of the C++ community. I really tried to advocate for the advantages of the other approach, but there was a massive push back from the community. It is even very possible that Conan wouldn't existed today if we had decided to stuck with that approach.


> The targets generator does not even have to be another generator at all. Maybe it can be done via one or two options to CMakeDeps?

There are some hybrid options that we are exploring that could emulate the legacy behavior of the ``cmake`` generator, and we will consult with the team how best to proceed. Can't promise anything, as there many challenges and we want to avoid fragmentation in our user base, but we are investigating this and will update this issue accordingly. 
We are investigating the impact of migrating our SW projects (with many small modules) from Conan 1.x to 2.x and this topic is one of the bigger "oh crap" moments we have experienced.

I am making this post to point out, that you may now have satisfied a vocal group of Conan users, which was unhappy, but on the other hand have upset a previously silent group of happy users.

For what it's worth, our developers don't care about CMake "purity", they just want things to work with low effort. From their perspective, when they want to consume a new package/library, they now have to add three seemingly redundant pieces of code in separate locations (requires, find_package, target_link_libraries), where it was previously enough to add it in just one place (requires). Some even opted to use conandata.yml so that a) recipes are not cluttered with a large number of requires entries and b) (more importantly) developers don't have to touch the "scary parts" i.e. conan recipe, cmakelists.txt.

I hope that I will be able to write a custom generator that just iterates over the dependencies and creates the appropriate find_package and target_link_libraries statements, but I haven't thought too much about the feasibility/pitfalls of that plan yet.

Commit message: Merge branch 'release/2.0' into feature/cmakedeps_legacy_adapter

Patch to review:
diff --git a/conan/tools/cmake/cmakedeps/cmakedeps.py b/conan/tools/cmake/cmakedeps/cmakedeps.py
index 9ab2816c060..dda5b3dab97 100644
--- a/conan/tools/cmake/cmakedeps/cmakedeps.py
+++ b/conan/tools/cmake/cmakedeps/cmakedeps.py
@@ -1,5 +1,10 @@
 import os
+import textwrap
 
+import jinja2
+from jinja2 import Template
+
+from conan.api.output import Color
 from conan.internal import check_duplicated_generator
 from conan.tools.cmake.cmakedeps import FIND_MODE_CONFIG, FIND_MODE_NONE, FIND_MODE_BOTH, \
     FIND_MODE_MODULE
@@ -43,6 +48,7 @@ def generate(self):
         generator_files = self.content
         for generator_file, content in generator_files.items():
             save(self._conanfile, generator_file, content)
+        self.generate_aggregator()
 
     @property
     def content(self):
@@ -66,6 +72,7 @@ def content(self):
                                      "generator.".format(common_name))
 
         # Iterate all the transitive requires
+        direct_configs = []
         for require, dep in list(host_req.items()) + list(build_req.items()) + list(test_req.items()):
             # Require is not used at the moment, but its information could be used,
             # and will be used in Conan 2.0
@@ -87,6 +94,23 @@ def content(self):
             if cmake_find_mode in (FIND_MODE_MODULE, FIND_MODE_BOTH):
                 self._generate_files(require, dep, ret, find_module_mode=True)
 
+            if require.direct:  # aggregate config information for user convenience
+                find_module_mode = True if cmake_find_mode == FIND_MODE_MODULE else False
+                config = ConfigTemplate(self, require, dep, find_module_mode)
+                direct_configs.append(config)
+
+        if direct_configs:
+            # Some helpful verbose messages about generated files
+            msg = ["CMakeDeps necessary find_package() and targets for your CMakeLists.txt"]
+            for config in direct_configs:
+                msg.append(f"    find_package({config.file_name})")
+            targets = ' '.join(c.root_target_name for c in direct_configs)
+            msg.append(f"    target_link_libraries(... {targets})")
+            if self._conanfile._conan_is_consumer:
+                self._conanfile.output.info("\n".join(msg), fg=Color.CYAN)
+            else:
+                self._conanfile.output.verbose("\n".join(msg))
+
         return ret
 
     def _generate_files(self, require, dep, ret, find_module_mode):
@@ -162,3 +186,38 @@ def get_find_mode(self, dep):
         if tmp is None:
             return "config"
         return tmp.lower()
+
+    def generate_aggregator(self):
+        host = self._conanfile.dependencies.host
+        build_req = self._conanfile.dependencies.direct_build
+        test_req = self._conanfile.dependencies.test
+
+        configs = []
+        for require, dep in list(host.items()) + list(build_req.items()) + list(test_req.items()):
+            if not require.direct:
+                continue
+            if require.build and dep.ref.name not in self.build_context_activated:
+                continue
+            cmake_find_mode = self.get_property("cmake_find_mode", dep)
+            cmake_find_mode = cmake_find_mode or FIND_MODE_CONFIG
+            cmake_find_mode = cmake_find_mode.lower()
+            find_module_mode = True if cmake_find_mode == FIND_MODE_MODULE else False
+            config = ConfigTemplate(self, require, dep, find_module_mode)
+            configs.append(config)
+
+        template = textwrap.dedent("""\
+            message(STATUS "Conan: Using CMakeDeps conandeps.cmake aggregator via include()")
+            message(STATUS "Conan: It is recommended to use explicit find_package() per dependency instead")
+
+            {% for config in configs %}
+            find_package({{config.file_name}})
+            {% endfor %}
+
+            add_library(conandeps INTERFACE)
+            target_link_libraries(conandeps INTERFACE {% for t in configs %} {{t.root_target_name}} {% endfor %})
+            """)
+
+        template = Template(template, trim_blocks=True, lstrip_blocks=True,
+                            undefined=jinja2.StrictUndefined)
+        conandeps = template.render({"configs": configs})
+        save(self._conanfile, "conandeps.cmake", conandeps)
diff --git a/conans/test/functional/toolchains/cmake/cmakedeps/test_cmakedeps_aggregator.py b/conans/test/functional/toolchains/cmake/cmakedeps/test_cmakedeps_aggregator.py
new file mode 100644
index 00000000000..75b9851cb4c
--- /dev/null
+++ b/conans/test/functional/toolchains/cmake/cmakedeps/test_cmakedeps_aggregator.py
@@ -0,0 +1,44 @@
+import textwrap
+
+from conans.test.assets.sources import gen_function_cpp
+
+
+def test_aggregator(transitive_libraries):
+    c = transitive_libraries
+
+    conanfile = textwrap.dedent("""
+        import os
+        from conan import ConanFile
+        from conan.tools.cmake import CMake, cmake_layout
+        class Pkg(ConanFile):
+            settings = "os", "arch", "compiler", "build_type"
+            requires = "engine/1.0"
+            generators = "CMakeDeps", "CMakeToolchain"
+            exports_sources = "*"
+            def layout(self):
+                cmake_layout(self)
+            def build(self):
+                cmake = CMake(self)
+                cmake.configure()
+                cmake.build()
+                self.run(os.path.join(self.cpp.build.bindir, "app"))
+        """)
+
+    cmakelists = textwrap.dedent("""
+        cmake_minimum_required(VERSION 3.15)
+        set(CMAKE_CXX_COMPILER_WORKS 1)
+        set(CMAKE_CXX_ABI_COMPILED 1)
+        project(app CXX)
+        include(${CMAKE_BINARY_DIR}/generators/conandeps.cmake)
+        add_executable(app main.cpp)
+        target_link_libraries(app conandeps)
+        """)
+
+    c.save({
+        "conanfile.py": conanfile,
+        "main.cpp": gen_function_cpp(name="main", includes=["engine"], calls=["engine"]),
+        "CMakeLists.txt": cmakelists
+    }, clean_first=True)
+    c.run("build .")
+    assert "matrix/1.0: Hello World Release!" in c.out
+    assert "engine/1.0: Hello World Release!" in c.out


Code review comments:
diff_hunk: @@ -162,3 +186,38 @@ def get_find_mode(self, dep):
         if tmp is None:
             return "config"
         return tmp.lower()
+
+    def generate_aggregator(self):
+        host = self._conanfile.dependencies.host
+        build_req = self._conanfile.dependencies.direct_build
+        test_req = self._conanfile.dependencies.test
+
+        configs = []
+        for require, dep in list(host.items()) + list(build_req.items()) + list(test_req.items()):
+            if not require.direct:
+                continue
+            if require.build and dep.ref.name not in self.build_context_activated:
+                continue
+            cmake_find_mode = self.get_property("cmake_find_mode", dep)
+            cmake_find_mode = cmake_find_mode or FIND_MODE_CONFIG
+            cmake_find_mode = cmake_find_mode.lower()
+            find_module_mode = True if cmake_find_mode == FIND_MODE_MODULE else False
+            config = ConfigTemplate(self, require, dep, find_module_mode)
+            configs.append(config)
+
+        template = textwrap.dedent("""\
+            message(STATUS "Conan: Using CMakeDeps conandeps.cmake aggregator via include()")
+            message(STATUS "Conan: It is recommended to use explicit find_package() per dependency instead")
+
+            {% for config in configs %}
+            find_package({{config.file_name}})
+            {% endfor %}
+
+            add_library(conandeps INTERFACE)
path: conan/tools/cmake/cmakedeps/cmakedeps.py
text: @user1:
Consider using a variable like the original generator, where the value of the variable is a list of targets
https://docs.conan.io/1/integrations/build_system/cmake/cmake_generator.html#global-variables-approach

(I'd also consider using a name like `CONAN_DEPS_LEGACY` or something like that)

diff_hunk: @@ -0,0 +1,44 @@
+import textwrap
+
+from conans.test.assets.sources import gen_function_cpp
+
+
+def test_aggregator(transitive_libraries):
+    c = transitive_libraries
+
+    conanfile = textwrap.dedent("""
+        import os
+        from conan import ConanFile
+        from conan.tools.cmake import CMake, cmake_layout
+        class Pkg(ConanFile):
+            settings = "os", "arch", "compiler", "build_type"
+            requires = "engine/1.0"
+            generators = "CMakeDeps", "CMakeToolchain"
+            exports_sources = "*"
+            def layout(self):
+                cmake_layout(self)
+            def build(self):
+                cmake = CMake(self)
+                cmake.configure()
+                cmake.build()
+                self.run(os.path.join(self.cpp.build.bindir, "app"))
+        """)
+
+    cmakelists = textwrap.dedent("""
+        cmake_minimum_required(VERSION 3.15)
+        set(CMAKE_CXX_COMPILER_WORKS 1)
+        set(CMAKE_CXX_ABI_COMPILED 1)
+        project(app CXX)
+        include(${CMAKE_BINARY_DIR}/generators/conandeps.cmake)
path: conans/test/functional/toolchains/cmake/cmakedeps/test_cmakedeps_aggregator.py
text: @user1:
I would probably suggest renaming this to `conandeps_legacy.cmake` or something similar - to reinforce the message that this is compatible with a legacy behaviour.