Repository: pantsbuild/pants

Issue:
Allow for me to _eject_ dependencies in an Infer/Inject dependencies request
**Is your feature request related to a problem? Please describe.**
Twice now, I've found myself wanting to write a plugin which implements infer/inject dependencies and in addition to adding dependencies, I want to "boot" some as well.

The booting would be transitive, too.

---

The maybe simplest use-case is the python package(s) `mxnet`, which comes in GPU CUDA flavors as well (`mxnet-cu*`). Both provide the module `mxnet`. I have both deps in my resolve, and the CPU-bound package gets used by default by nature of the name and `module_mapper`'s sane default.

I want a CLI toggle to inject the GPU flavor, not hard at all. However now my requirements PEX has _both_ the CPU `mxnet` and the GPU `mxnet-cu*` packages, and that results in the CPU package being loaded (which I'm sure is an implementation detail). In cases where I _inject_ the GPU dep, I need to _eject_ the CPU one.

**Describe the solution you'd like**
Provide me a way as a plugin author to eject dependencies (perhaps as simple as allowing me to inject a `!!...` dependency address).

**Describe alternatives you've considered**
`sys.path` hacks, which is what I'm gonna do until I have a better way.

**Additional context**
N/A



Discussion:
> (perhaps as simple as allowing me to inject a !!... dependency address).

This is much easier to implement if you only remove deps for the target in question, rather than transitively.

I don't think this would be very hard to add: you'll define a new `@union` like `RemoveDependenciesPlugin` or something, and in the rule for `DependenciesRequest`, use set addition to subtract out everything relevant.
> This is much easier to implement if you only remove deps for the target in question, rather than transitively.

Ah, and if I code it right, every relevant target will have it removed, so it wouldn't appear anywhere due to transitive deps? Very good point!

Relevant ticket if we're adding a new union: https://github.com/pantsbuild/pants/issues/15400
> Relevant ticket if we're adding a new union: https://github.com/pantsbuild/pants/issues/15400

Indeed, but not blocking this at all.

Resolving patch:
diff --git a/src/python/pants/backend/java/dependency_inference/rules.py b/src/python/pants/backend/java/dependency_inference/rules.py
--- a/src/python/pants/backend/java/dependency_inference/rules.py
+++ b/src/python/pants/backend/java/dependency_inference/rules.py
@@ -64,7 +64,7 @@ async def infer_java_dependencies_via_source_analysis(
         JavaInferredDependencies,
         JavaInferredDependenciesAndExportsRequest(request.field_set.source),
     )
-    return InferredDependencies(dependencies=jids.dependencies)
+    return InferredDependencies(jids.dependencies)
 
 
 @rule(desc="Inferring Java dependencies and exports by source analysis")
diff --git a/src/python/pants/engine/internals/graph.py b/src/python/pants/engine/internals/graph.py
--- a/src/python/pants/engine/internals/graph.py
+++ b/src/python/pants/engine/internals/graph.py
@@ -1158,6 +1158,9 @@ async def resolve_dependencies(
             for addr in special_cased_field.to_unparsed_address_inputs().values
         )
 
+    excluded = explicitly_provided.ignores.union(
+        *itertools.chain(deps.exclude for deps in inferred)
+    )
     result = Addresses(
         sorted(
             {
@@ -1165,10 +1168,10 @@ async def resolve_dependencies(
                 for addr in (
                     *generated_addresses,
                     *explicitly_provided_includes,
-                    *itertools.chain.from_iterable(inferred),
+                    *itertools.chain.from_iterable(deps.include for deps in inferred),
                     *special_cased,
                 )
-                if addr not in explicitly_provided.ignores
+                if addr not in excluded
             }
         )
     )
diff --git a/src/python/pants/engine/target.py b/src/python/pants/engine/target.py
--- a/src/python/pants/engine/target.py
+++ b/src/python/pants/engine/target.py
@@ -2553,17 +2553,18 @@ def rules():
 @frozen_after_init
 @dataclass(unsafe_hash=True)
 class InferredDependencies:
-    dependencies: FrozenOrderedSet[Address]
+    include: FrozenOrderedSet[Address]
+    exclude: FrozenOrderedSet[Address]
 
-    def __init__(self, dependencies: Iterable[Address]) -> None:
+    def __init__(
+        self,
+        include: Iterable[Address],
+        *,
+        exclude: Iterable[Address] = (),
+    ) -> None:
         """The result of inferring dependencies."""
-        self.dependencies = FrozenOrderedSet(sorted(dependencies))
-
-    def __bool__(self) -> bool:
-        return bool(self.dependencies)
-
-    def __iter__(self) -> Iterator[Address]:
-        return iter(self.dependencies)
+        self.include = FrozenOrderedSet(sorted(include))
+        self.exclude = FrozenOrderedSet(sorted(exclude))
 
 
 @union


Test patch:
diff --git a/src/python/pants/backend/go/target_type_rules_test.py b/src/python/pants/backend/go/target_type_rules_test.py
--- a/src/python/pants/backend/go/target_type_rules_test.py
+++ b/src/python/pants/backend/go/target_type_rules_test.py
@@ -298,7 +298,7 @@ def get_main(addr: Address) -> Address:
                 )
             ],
         )
-        assert [main_addr] == list(inferred_addresses)
+        assert inferred_addresses == InferredDependencies([main_addr])
         return main_addr
 
     assert get_main(Address("explicit")) == Address("explicit", target_name="pkg")
diff --git a/src/python/pants/backend/helm/dependency_inference/chart_test.py b/src/python/pants/backend/helm/dependency_inference/chart_test.py
--- a/src/python/pants/backend/helm/dependency_inference/chart_test.py
+++ b/src/python/pants/backend/helm/dependency_inference/chart_test.py
@@ -147,10 +147,12 @@ def test_infer_chart_dependencies(rule_runner: RuleRunner) -> None:
         InferredDependencies,
         [InferHelmChartDependenciesRequest(HelmChartDependenciesInferenceFieldSet.create(tgt))],
     )
-    assert set(inferred_deps.dependencies) == {
-        Address("3rdparty/helm/jetstack", target_name="cert-manager"),
-        Address("src/bar", target_name="bar"),
-    }
+    assert inferred_deps == InferredDependencies(
+        [
+            Address("3rdparty/helm/jetstack", target_name="cert-manager"),
+            Address("src/bar", target_name="bar"),
+        ]
+    )
 
 
 def test_disambiguate_chart_dependencies(rule_runner: RuleRunner) -> None:
@@ -190,9 +192,11 @@ def test_disambiguate_chart_dependencies(rule_runner: RuleRunner) -> None:
         InferredDependencies,
         [InferHelmChartDependenciesRequest(HelmChartDependenciesInferenceFieldSet.create(tgt))],
     )
-    assert set(inferred_deps.dependencies) == {
-        Address("src/bar", target_name="bar"),
-    }
+    assert inferred_deps == InferredDependencies(
+        [
+            Address("src/bar", target_name="bar"),
+        ]
+    )
 
 
 def test_raise_error_when_unknown_dependency_is_found(rule_runner: RuleRunner) -> None:
diff --git a/src/python/pants/backend/helm/dependency_inference/unittest_test.py b/src/python/pants/backend/helm/dependency_inference/unittest_test.py
--- a/src/python/pants/backend/helm/dependency_inference/unittest_test.py
+++ b/src/python/pants/backend/helm/dependency_inference/unittest_test.py
@@ -71,8 +71,7 @@ def test_infers_single_chart(rule_runner: RuleRunner) -> None:
         ],
     )
 
-    assert len(inferred_deps.dependencies) == 1
-    assert list(inferred_deps.dependencies)[0] == chart_tgt.address
+    assert inferred_deps == InferredDependencies([chart_tgt.address])
 
 
 def test_injects_parent_chart(rule_runner: RuleRunner) -> None:
@@ -119,8 +118,5 @@ def test_injects_parent_chart(rule_runner: RuleRunner) -> None:
         ],
     )
 
-    assert len(chart1_inferred_deps.dependencies) == 1
-    assert len(chart2_inferred_deps.dependencies) == 1
-
-    assert list(chart1_inferred_deps.dependencies)[0] == chart1_tgt.address
-    assert list(chart2_inferred_deps.dependencies)[0] == chart2_tgt.address
+    assert chart1_inferred_deps == InferredDependencies([chart1_tgt.address])
+    assert chart2_inferred_deps == InferredDependencies([chart2_tgt.address])
diff --git a/src/python/pants/backend/java/dependency_inference/rules_test.py b/src/python/pants/backend/java/dependency_inference/rules_test.py
--- a/src/python/pants/backend/java/dependency_inference/rules_test.py
+++ b/src/python/pants/backend/java/dependency_inference/rules_test.py
@@ -98,12 +98,12 @@ def test_infer_java_imports_same_target(rule_runner: RuleRunner) -> None:
     assert rule_runner.request(
         InferredDependencies,
         [InferJavaSourceDependencies(JavaSourceDependenciesInferenceFieldSet.create(target_a))],
-    ) == InferredDependencies(dependencies=[])
+    ) == InferredDependencies([])
 
     assert rule_runner.request(
         InferredDependencies,
         [InferJavaSourceDependencies(JavaSourceDependenciesInferenceFieldSet.create(target_b))],
-    ) == InferredDependencies(dependencies=[])
+    ) == InferredDependencies([])
 
 
 @maybe_skip_jdk_test
@@ -150,12 +150,12 @@ def test_infer_java_imports(rule_runner: RuleRunner) -> None:
     assert rule_runner.request(
         InferredDependencies,
         [InferJavaSourceDependencies(JavaSourceDependenciesInferenceFieldSet.create(target_a))],
-    ) == InferredDependencies(dependencies=[target_b.address])
+    ) == InferredDependencies([target_b.address])
 
     assert rule_runner.request(
         InferredDependencies,
         [InferJavaSourceDependencies(JavaSourceDependenciesInferenceFieldSet.create(target_b))],
-    ) == InferredDependencies(dependencies=[])
+    ) == InferredDependencies([])
 
 
 @maybe_skip_jdk_test
@@ -205,12 +205,12 @@ def test_infer_java_imports_with_cycle(rule_runner: RuleRunner) -> None:
     assert rule_runner.request(
         InferredDependencies,
         [InferJavaSourceDependencies(JavaSourceDependenciesInferenceFieldSet.create(target_a))],
-    ) == InferredDependencies(dependencies=[target_b.address])
+    ) == InferredDependencies([target_b.address])
 
     assert rule_runner.request(
         InferredDependencies,
         [InferJavaSourceDependencies(JavaSourceDependenciesInferenceFieldSet.create(target_b))],
-    ) == InferredDependencies(dependencies=[target_a.address])
+    ) == InferredDependencies([target_a.address])
 
 
 @maybe_skip_jdk_test
@@ -260,7 +260,7 @@ def test_infer_java_imports_ambiguous(rule_runner: RuleRunner, caplog) -> None:
     assert rule_runner.request(
         InferredDependencies,
         [InferJavaSourceDependencies(JavaSourceDependenciesInferenceFieldSet.create(target_b))],
-    ) == InferredDependencies(dependencies=[])
+    ) == InferredDependencies([])
     assert len(caplog.records) == 1
     assert (
         "The target b/B.java imports `org.pantsbuild.a.A`, but Pants cannot safely" in caplog.text
@@ -269,7 +269,7 @@ def test_infer_java_imports_ambiguous(rule_runner: RuleRunner, caplog) -> None:
     assert rule_runner.request(
         InferredDependencies,
         [InferJavaSourceDependencies(JavaSourceDependenciesInferenceFieldSet.create(target_c))],
-    ) == InferredDependencies(dependencies=[Address("a_one", relative_file_path="A.java")])
+    ) == InferredDependencies([Address("a_one", relative_file_path="A.java")])
 
 
 @maybe_skip_jdk_test
@@ -304,9 +304,7 @@ def test_infer_java_imports_unnamed_package(rule_runner: RuleRunner) -> None:
     assert rule_runner.request(
         InferredDependencies,
         [InferJavaSourceDependencies(JavaSourceDependenciesInferenceFieldSet.create(target_a))],
-    ) == InferredDependencies(
-        dependencies=[Address("", target_name="a", relative_file_path="Lib.java")]
-    )
+    ) == InferredDependencies([Address("", target_name="a", relative_file_path="Lib.java")])
 
 
 @maybe_skip_jdk_test
@@ -348,12 +346,12 @@ def test_infer_java_imports_same_target_with_cycle(rule_runner: RuleRunner) -> N
     assert rule_runner.request(
         InferredDependencies,
         [InferJavaSourceDependencies(JavaSourceDependenciesInferenceFieldSet.create(target_a))],
-    ) == InferredDependencies(dependencies=[target_b.address])
+    ) == InferredDependencies([target_b.address])
 
     assert rule_runner.request(
         InferredDependencies,
         [InferJavaSourceDependencies(JavaSourceDependenciesInferenceFieldSet.create(target_b))],
-    ) == InferredDependencies(dependencies=[target_a.address])
+    ) == InferredDependencies([target_a.address])
 
 
 @maybe_skip_jdk_test
diff --git a/src/python/pants/backend/kotlin/dependency_inference/rules_test.py b/src/python/pants/backend/kotlin/dependency_inference/rules_test.py
--- a/src/python/pants/backend/kotlin/dependency_inference/rules_test.py
+++ b/src/python/pants/backend/kotlin/dependency_inference/rules_test.py
@@ -91,12 +91,12 @@ class B {}
     assert rule_runner.request(
         InferredDependencies,
         [InferKotlinSourceDependencies(KotlinSourceDependenciesInferenceFieldSet.create(target_a))],
-    ) == InferredDependencies(dependencies=[])
+    ) == InferredDependencies([])
 
     assert rule_runner.request(
         InferredDependencies,
         [InferKotlinSourceDependencies(KotlinSourceDependenciesInferenceFieldSet.create(target_b))],
-    ) == InferredDependencies(dependencies=[])
+    ) == InferredDependencies([])
 
 
 @maybe_skip_jdk_test
@@ -140,12 +140,12 @@ class B {}
     assert rule_runner.request(
         InferredDependencies,
         [InferKotlinSourceDependencies(KotlinSourceDependenciesInferenceFieldSet.create(target_a))],
-    ) == InferredDependencies(dependencies=[target_b.address])
+    ) == InferredDependencies([target_b.address])
 
     assert rule_runner.request(
         InferredDependencies,
         [InferKotlinSourceDependencies(KotlinSourceDependenciesInferenceFieldSet.create(target_b))],
-    ) == InferredDependencies(dependencies=[target_a.address])
+    ) == InferredDependencies([target_a.address])
 
 
 @maybe_skip_jdk_test
@@ -195,14 +195,14 @@ class C {}
     assert rule_runner.request(
         InferredDependencies,
         [InferKotlinSourceDependencies(KotlinSourceDependenciesInferenceFieldSet.create(target_b))],
-    ) == InferredDependencies(dependencies=[])
+    ) == InferredDependencies([])
     assert len(caplog.records) == 1
     assert "The target b/B.kt imports `org.pantsbuild.a.A`, but Pants cannot safely" in caplog.text
 
     assert rule_runner.request(
         InferredDependencies,
         [InferKotlinSourceDependencies(KotlinSourceDependenciesInferenceFieldSet.create(target_c))],
-    ) == InferredDependencies(dependencies=[Address("a_one", relative_file_path="A.kt")])
+    ) == InferredDependencies([Address("a_one", relative_file_path="A.kt")])
 
 
 @maybe_skip_jdk_test
@@ -242,7 +242,7 @@ def main(args: Array<String>) {
     assert rule_runner.request(
         InferredDependencies,
         [InferKotlinSourceDependencies(KotlinSourceDependenciesInferenceFieldSet.create(target_a))],
-    ) == InferredDependencies(dependencies=[])
+    ) == InferredDependencies([])
 
     assert rule_runner.request(
         InferredDependencies,
@@ -251,4 +251,4 @@ def main(args: Array<String>) {
                 KotlinSourceDependenciesInferenceFieldSet.create(target_main)
             )
         ],
-    ) == InferredDependencies(dependencies=[target_a.address])
+    ) == InferredDependencies([target_a.address])
diff --git a/src/python/pants/backend/python/dependency_inference/rules_test.py b/src/python/pants/backend/python/dependency_inference/rules_test.py
--- a/src/python/pants/backend/python/dependency_inference/rules_test.py
+++ b/src/python/pants/backend/python/dependency_inference/rules_test.py
@@ -375,9 +375,9 @@ def check(address: Address, expected: list[Address]) -> None:
             [InferInitDependencies(InitDependenciesInferenceFieldSet.create(target))],
         )
         if behavior == InitFilesInference.never:
-            assert not result
-        else:
-            assert result == InferredDependencies(expected)
+            expected = []
+
+        assert result == InferredDependencies(expected)
 
     check(
         Address("src/python/root/mid/leaf", relative_file_path="f.py"),
diff --git a/src/python/pants/backend/scala/dependency_inference/rules_test.py b/src/python/pants/backend/scala/dependency_inference/rules_test.py
--- a/src/python/pants/backend/scala/dependency_inference/rules_test.py
+++ b/src/python/pants/backend/scala/dependency_inference/rules_test.py
@@ -90,12 +90,12 @@ def test_infer_scala_imports_same_target(rule_runner: RuleRunner) -> None:
     assert rule_runner.request(
         InferredDependencies,
         [InferScalaSourceDependencies(ScalaSourceDependenciesInferenceFieldSet.create(target_a))],
-    ) == InferredDependencies(dependencies=[])
+    ) == InferredDependencies([])
 
     assert rule_runner.request(
         InferredDependencies,
         [InferScalaSourceDependencies(ScalaSourceDependenciesInferenceFieldSet.create(target_b))],
-    ) == InferredDependencies(dependencies=[])
+    ) == InferredDependencies([])
 
 
 @maybe_skip_jdk_test
@@ -143,12 +143,12 @@ class B {}
     assert rule_runner.request(
         InferredDependencies,
         [InferScalaSourceDependencies(ScalaSourceDependenciesInferenceFieldSet.create(target_a))],
-    ) == InferredDependencies(dependencies=[target_b.address])
+    ) == InferredDependencies([target_b.address])
 
     assert rule_runner.request(
         InferredDependencies,
         [InferScalaSourceDependencies(ScalaSourceDependenciesInferenceFieldSet.create(target_b))],
-    ) == InferredDependencies(dependencies=[target_a.address])
+    ) == InferredDependencies([target_a.address])
 
 
 @maybe_skip_jdk_test
@@ -198,7 +198,7 @@ class C {}
     assert rule_runner.request(
         InferredDependencies,
         [InferScalaSourceDependencies(ScalaSourceDependenciesInferenceFieldSet.create(target_b))],
-    ) == InferredDependencies(dependencies=[])
+    ) == InferredDependencies([])
     assert len(caplog.records) == 1
     assert (
         "The target b/B.scala imports `org.pantsbuild.a.A`, but Pants cannot safely" in caplog.text
@@ -207,7 +207,7 @@ class C {}
     assert rule_runner.request(
         InferredDependencies,
         [InferScalaSourceDependencies(ScalaSourceDependenciesInferenceFieldSet.create(target_c))],
-    ) == InferredDependencies(dependencies=[Address("a_one", relative_file_path="A.scala")])
+    ) == InferredDependencies([Address("a_one", relative_file_path="A.scala")])
 
 
 def test_infer_unqualified_symbol_from_intermediate_scope(rule_runner: RuleRunner) -> None:
@@ -443,9 +443,9 @@ class D {
     assert rule_runner.request(
         InferredDependencies,
         [InferScalaSourceDependencies(ScalaSourceDependenciesInferenceFieldSet.create(target_c))],
-    ) == InferredDependencies(dependencies=[target_b.address])
+    ) == InferredDependencies([target_b.address])
 
     assert rule_runner.request(
         InferredDependencies,
         [InferScalaSourceDependencies(ScalaSourceDependenciesInferenceFieldSet.create(target_d))],
-    ) == InferredDependencies(dependencies=[target_b.address])
+    ) == InferredDependencies([target_b.address])
diff --git a/src/python/pants/engine/internals/graph_test.py b/src/python/pants/engine/internals/graph_test.py
--- a/src/python/pants/engine/internals/graph_test.py
+++ b/src/python/pants/engine/internals/graph_test.py
@@ -1638,14 +1638,27 @@ async def infer_smalltalk_dependencies(request: InferSmalltalkDependencies) -> I
     # addresses, one per line.
     hydrated_sources = await Get(HydratedSources, HydrateSourcesRequest(request.field_set.source))
     digest_contents = await Get(DigestContents, Digest, hydrated_sources.snapshot.digest)
-    all_lines = itertools.chain.from_iterable(
-        file_content.content.decode().splitlines() for file_content in digest_contents
-    )
-    resolved = await MultiGet(
+    all_lines = [
+        line.strip()
+        for line in itertools.chain.from_iterable(
+            file_content.content.decode().splitlines() for file_content in digest_contents
+        )
+    ]
+    include = await MultiGet(
         Get(Address, AddressInput, AddressInput.parse(line, description_of_origin="smalltalk rule"))
         for line in all_lines
+        if not line.startswith("!")
+    )
+    exclude = await MultiGet(
+        Get(
+            Address,
+            AddressInput,
+            AddressInput.parse(line[1:], description_of_origin="smalltalk rule"),
+        )
+        for line in all_lines
+        if line.startswith("!")
     )
-    return InferredDependencies(resolved)
+    return InferredDependencies(include, exclude=exclude)
 
 
 @pytest.fixture
@@ -1808,6 +1821,13 @@ def test_dependency_inference(dependencies_rule_runner: RuleRunner) -> None:
                 //:inferred_but_ignored2
                 """
             ),
+            "demo/f3.st": dedent(
+                """\
+                //:inferred1
+                !:inferred_and_provided1
+                !//:inferred_and_provided2
+                """
+            ),
             "demo/BUILD": dedent(
                 """\
                 smalltalk_libraries(
@@ -1830,6 +1850,7 @@ def test_dependency_inference(dependencies_rule_runner: RuleRunner) -> None:
         expected=[
             Address("demo", relative_file_path="f1.st"),
             Address("demo", relative_file_path="f2.st"),
+            Address("demo", relative_file_path="f3.st"),
         ],
     )
 
@@ -1857,6 +1878,13 @@ def test_dependency_inference(dependencies_rule_runner: RuleRunner) -> None:
             ),
         ],
     )
+    assert_dependencies_resolved(
+        dependencies_rule_runner,
+        Address("demo", relative_file_path="f3.st", target_name="demo"),
+        expected=[
+            Address("", target_name="inferred1"),
+        ],
+    )
 
 
 def test_depends_on_generated_targets(dependencies_rule_runner: RuleRunner) -> None: