Repository: conan-io/conan

Issue:
[bug] dependency cycle can't be avoided when bootstrapping a compiler
### Describe the bug

I'm not sure if this ought to be a bug report or a feature request.

I'm working on a recipe for gcc, and I'm running into a bootstrapping 
issue that I don't know how to solve.


My goals are:

* Build gcc cross compilers for linux and package them with conan.

* The minimum versions of the linux kernel and glibc that are being
 targeted by the cross compiler should be specified in the settings.

* Build a bootstrap gcc using whatever C compiler is available in path, and
then use that to build the main cross compiler.

I thought I had the whole scheme working.

This command does indeed succeed.  It creates a bootstrap gcc, and then uses 
the bootstrap gcc to build a final gcc.

``` 
$ conan create ./cc -pr:b ./Linux-bootstrap -pr:h ./Linux-x86_64
```

But then when I try to use it, it all falls apart.

```
$ conan create foo -pr:h ./Linux-x86_64 -pr:b ./Linux-x86_64
...
ERROR: There is a cycle/loop in the graph:
    Initial ancestor: cc/1.0
    Require: cc/1.0
    Dependency: cc/1.0
```

This dependency cycle would be a problem if I really was trying to build `cc`
using `Linux-x86_64` as the build profile.   But I don't need to build it.  I 
built it already using `Linux-bootstrap` as the build profile.   The `package_id` 
has not changed.   Now I just want to use it.  But because there'd be a dependency 
cycle *if* I tried to build it this way, it doesn't even get that far.

What can I do?

Thanks for reading.



### How to reproduce it

example code is here https://github.com/smoofra/conan-cc-bootstrap


Discussion:
Hi @smoofra 

Thanks for your report.

I have just tried it and it works, I can do your above steps and it doesn't raise any error.

You didn't report the Conan version, so maybe you are using Conan 1? I was trying with Conan 2. 
One of the aspects that Conan 2 greatly improve over Conan 1 is the graph resolution capabilities, allowing more advanced cases like this kind of bootstrapping.
I'm using conan 2.5.0 
I just tried it again in a fresh environment and I still get the erorr

ubuntu 24.04,  python 3.12.3

```
certifi==2024.7.4
charset-normalizer==3.3.2
colorama==0.4.6
conan==2.5.0
distro==1.8.0
fasteners==0.19
idna==3.7
Jinja2==3.1.4
MarkupSafe==2.1.5
patch-ng==1.17.4
python-dateutil==2.9.0.post0
PyYAML==6.0.1
requests==2.32.3
six==1.16.0
urllib3==1.26.19
```
I have added a dockerfile to the example 
run `docker build .` and it should reproduce
well, I found a workaround

```
    def build_requirements(self):
        # when this recipe is being used as a build tool, hide all of its
        # requirements to avoid graph cycle errors due to bootstrapping
        if self.context == "build":
            return
        ....
```

It means that the gcc recipe can't be built using `--build=missing`, because the dependencies would be missing from conan.    

But my gcc recipe is already only possible to build inside its own special CI pipeline anyway for [reasons](https://github.com/conan-io/conan/issues/13469) so this is no great loss to me
Sorry, I have limited connectivity now, can't really use docker now, but assigned it to have a look at my return.
It is possible that the ``if self.context == "build":`` is not a workaround, it sounds like a valid and correct way to break an infinite loop that would happen otherwise (I need to check it properly first, last time I couldn't reproduce)
I have been investigating this.

At the moment, it is not possible to have a dependency on the same ``pkg/version`` as the current package, being the only exception the one that does something like:
```python
def build_requirements(self):
     if cross_building(self):
           self.tool_requires("pkg/version")
```

The reason it is this way is because it is very easy to fall in an infinite loop which results in the Conan application hanging forever until it exhausts and fail, which is very bad UX.

The ``cross_building()`` condition acts very much like the ``if self.context == "build"`` in your case, pre-building it in the way you do it, as it is not possible to build it as part of a larger graph with ``--build=missing`` if the package is already in the "build" context.

It is not possible then to use ``settings`` to condition in the ``build_requirements()``, because it doesn't really break the loop, the loop is checked before that and it fails, except for the case of the change of context from host to build.

I'd say that your approach is good enough at the moment.

I am not saying that we will not try to expand the bootstrapping capabilities, I will do some experiments to check what might be possible.
This would be an experimental branch to investigate the possibilities: https://github.com/conan-io/conan/pull/16870



Patch:
diff --git a/conans/client/graph/graph.py b/conans/client/graph/graph.py
index 284a05b202b..a2834789a9d 100644
--- a/conans/client/graph/graph.py
+++ b/conans/client/graph/graph.py
@@ -135,7 +135,7 @@ def check_downstream_exists(self, require):
             if require.build and (self.context == CONTEXT_HOST or  # switch context
                                   require.ref.version != self.ref.version):  # or different version
                 pass
-            elif require.visible is False and require.ref.version != self.ref.version:
+            elif require.visible is False: #  and require.ref.version != self.ref.version:
                 # Experimental, to support repackaging of openssl previous versions FIPS plugins
                 pass  # An invisible require doesn't conflict with itself
             else:
@@ -177,15 +177,17 @@ def check_downstream_exists(self, require):
         source_node = dependant.src
         return source_node.check_downstream_exists(down_require) or result
 
-    def check_loops(self, new_node):
+    def check_loops(self, new_node, count=0):
         if self.ref == new_node.ref and self.context == new_node.context:
-            return self
+            if count >= 1:
+                return self
+            count += 1
         if not self.dependants:
             return
         assert len(self.dependants) == 1
         dependant = self.dependants[0]
         source_node = dependant.src
-        return source_node.check_loops(new_node)
+        return source_node.check_loops(new_node, count)
 
     @property
     def package_id(self):


Test patch:
diff --git a/test/integration/build_requires/test_install_test_build_require.py b/test/integration/build_requires/test_install_test_build_require.py
index 148dfd2a35d..8892dc0325f 100644
--- a/test/integration/build_requires/test_install_test_build_require.py
+++ b/test/integration/build_requires/test_install_test_build_require.py
@@ -222,3 +222,37 @@ def build_requirements(self):
     assert package1["depends"] == []
     assert package2["package_id"] == linux_pkg_id
     assert package2["depends"] == [win_pkg_id]
+
+
+def test_bootstrap_cc():
+    # https://github.com/conan-io/conan/issues/16758
+    cc = textwrap.dedent("""
+        from conan import ConanFile
+
+        class CcConanfile(ConanFile):
+            name = "cc"
+            version = "1.0"
+            options = {"bootstrap": [True, False]}
+            default_options = {"bootstrap": True}
+
+            def build_requirements(self):
+                if self.options.bootstrap:
+                    self.tool_requires("cc/1.0", options={"*:bootstrap": False})
+        """)
+    foo = GenConanfile("foo", "1.0").with_tool_requirement("cc/1.0")
+    c = TestClient()
+    c.save({"cc/conanfile.py": cc,
+            "foo/conanfile.py": foo})
+    c.run("create cc --build-require")
+    # Both are build-requires
+    c.assert_listed_binary({"cc/1.0": ("826727aac60b5956d1df5121a3921f26a6984f15", "Build")},
+                           build=True)
+    c.assert_listed_binary({"cc/1.0": ("96ae1d965ce6f2e2256ac0cfc3a35dcd4860c389", "Build")},
+                           build=True)
+    # This works fine, being able to build the 2 different binaries both bootstrapped and not
+    c.run("list cc/1.0:*")
+    assert "bootstrap: False" in c.out
+    assert "bootstrap: True" in c.out
+    c.run("create foo")
+    c.assert_listed_binary({"cc/1.0": ("96ae1d965ce6f2e2256ac0cfc3a35dcd4860c389", "Cache")},
+                           build=True)
diff --git a/test/integration/graph/core/test_build_requires.py b/test/integration/graph/core/test_build_requires.py
index 7e396044b72..a3945956a9f 100644
--- a/test/integration/graph/core/test_build_requires.py
+++ b/test/integration/graph/core/test_build_requires.py
@@ -883,12 +883,15 @@ def test_direct_loop_error(self):
         assert type(deps_graph.error) == GraphLoopError
 
         # Build requires always apply to the consumer
-        self.assertEqual(2, len(deps_graph.nodes))
+        self.assertEqual(4, len(deps_graph.nodes))
         app = deps_graph.root
         tool = app.dependencies[0].dst
+        tool2 = tool.dependencies[0].dst
+        tool3 = tool2.dependencies[0].dst
 
         self._check_node(app, "app/0.1", deps=[tool], dependents=[])
-        self._check_node(tool, "cmake/0.1#123", deps=[], dependents=[app])
+        self._check_node(tool, "cmake/0.1#123", deps=[tool2], dependents=[app])
+        self._check_node(tool2, "cmake/0.1#123", deps=[tool3], dependents=[tool])
 
     def test_indirect_loop_error(self):
         # app -(br)-> gtest/0.1 -(br)-> cmake/0.1 -(br)->gtest/0.1 ....
@@ -902,14 +905,16 @@ def test_indirect_loop_error(self):
         assert type(deps_graph.error) == GraphLoopError
 
         # Build requires always apply to the consumer
-        self.assertEqual(4, len(deps_graph.nodes))
+        self.assertEqual(6, len(deps_graph.nodes))
         app = deps_graph.root
         cmake = app.dependencies[0].dst
         gtest = cmake.dependencies[0].dst
         cmake2 = gtest.dependencies[0].dst
+        gtest2 = cmake2.dependencies[0].dst
+        cmake3 = gtest2.dependencies[0].dst
 
         assert deps_graph.error.ancestor == cmake
-        assert deps_graph.error.node == cmake2
+        assert deps_graph.error.node == cmake3
 
     def test_infinite_recursion_test(self):
         """Ensure that direct conflicts in a node are properly reported and Conan does not loop"""