Repository: dbt-labs/dbt-core

Issue:
[CT-2381] [Bug] `state:modified+` selection doesn't behave as expected for `dbt build` when tests are modified
### Is this a new bug in dbt-core?

- [X] I believe this is a new bug in dbt-core
- [X] I have searched the existing issues, and I could not find an existing issue for this bug

### Current Behavior

`dbt build --select state:modified+` should behave like the other commands.

Specifically the following scenario doesn't yield the expected selection:
- modify a single/bunch of tests (but only tests)
- `PREVIOUS_TARGET=~/previous_target`
- `cp -r $DBT_PROJECT_DIR/target/* $PREVIOUS_TARGET`
- run `dbt list --select state:modified+ --state $PREVIOUS_TARGET`  => correctly lists only the changed tests (nothing downstream)
- run `dbt build --select state:modified+ --state $PREVIOUS_TARGET`  => runs the changed tests but also ALL downstream nodes!!
- while `dbt run --select state:modified+ --state $PREVIOUS_TARGET` does as expected: "Nothing to run"
- and `dbt test --select state:modified+ --state $PREVIOUS_TARGET` as well: only runs the changed tests.

### Expected Behavior

`dbt build --select state:modified+` should NOT run models downstream from modified tests (if only the tests have been modified, not the tests' model(s) )

### Steps To Reproduce

- modify a single/bunch of tests (but only tests)
- `PREVIOUS_TARGET=~/previous_target`
- `cp -r $DBT_PROJECT_DIR/target/* $PREVIOUS_TARGET`
- run `dbt list --select state:modified+ --state $PREVIOUS_TARGET`  => correctly lists only the changed tests (nothing downstream)
- run `dbt build --select state:modified+ --state $PREVIOUS_TARGET`  => runs the changed tests but also ALL downstream nodes!!
- while `dbt run --select state:modified+ --state $PREVIOUS_TARGET` does as expected: "Nothing to run"
- and `dbt test --select state:modified+ --state $PREVIOUS_TARGET` as well: only runs the changed tests.

### Relevant log output

_No response_

### Environment

```markdown
- OS: PopOS 22.04
- Python: 3.7
- dbt: 1.2.5
```


### Which database adapter are you using with dbt?

snowflake

### Additional Context

_No response_


Discussion:
Thanks for reporting this @b-luu !

Excellent write-up -- I was able to reproduce what you described:
- If we change test X anything downstream of it's parent node gets executed during `dbt build --select state:modified+ ...`


<details>
<summary>Full details of the reprex I used.
</summary>

`models/my_model_1.sql`

```
select 1 as id
```

`models/my_model_2.sql`

```
select * from {{ ref("my_model_1") }}
```

`models/my_model_3.sql`

```
select * from {{ ref("my_model_2") }}
```

`tests/my_test.sql`

```sql
select * from {{ ref("my_model_2") }}
where 0=1

```

Prime the pump:
```shell
dbt build
```

Alter the test and save the state:
```shell
echo "-- $(date +%T)" >> tests/singular/my_test.sql
PREVIOUS_TARGET=~/previous_target
mkdir -p $PREVIOUS_TARGET
cp -r target/* $PREVIOUS_TARGET
```

Try a few different commands:
```shell
dbt list --select state:modified+ --state $PREVIOUS_TARGET
dbt run --select state:modified+ --state $PREVIOUS_TARGET
dbt test --select state:modified+ --state $PREVIOUS_TARGET
dbt build --select state:modified+ --state $PREVIOUS_TARGET
```

</details>
Intuitively, I'd look into the order of execution between [BuildTask.get_node_selector](https://github.com/dbt-labs/dbt-core/blob/fe62ab8ec5df334985505bf03c5dd10b465ef995/core/dbt/task/build.py#L45) and [BuildTask.compile_manifest](https://github.com/dbt-labs/dbt-core/blob/fe62ab8ec5df334985505bf03c5dd10b465ef995/core/dbt/task/build.py#L67).

I've gotta admit I've stopped looking/lost myself in the [CompilerProtocol](https://github.com/dbt-labs/dbt-core/blob/fe62ab8ec5df334985505bf03c5dd10b465ef995/core/dbt/adapters/protocol.py#L53) because it lacks the `add_test_edges: bool` kwarg that's passed in [`BuildTask`'s `.compile_manifest()` method](https://github.com/dbt-labs/dbt-core/blob/fe62ab8ec5df334985505bf03c5dd10b465ef995/core/dbt/task/build.py#L72)... :thinking: :shrug: 
In its current implementation, the `build` task does actually add an edge between a test on upstream model A, and model B depending on model A. Meaning that, even though model B doesn't directly depend on that test, there is an edge between them in the DAG, and so it's selected _for the `build` task only_.

I think it will be tricky to resolve this issue, given that's our current approach to achieving the `build` behavior. In practice, we don't see this issue come up so frequently. I agree it's confusing when it does happen, though, and "leaks" the underlying implementation.
Thanks @jtcohen6 for the confirmation that it is indeed the expected behavior.

True the solution would be tricky, but here it's preventing me from using `dbt build` and forcing me to fallback to executing `run`, `test`, `snapshot`, and `seed` which isn't optimal because each type of task only runs after all the previous type have succeeded.

Anyway, as a quicker solution, do you think we could add a note on this caveat in [the documentation](https://docs.getdbt.com/reference/commands/build)? Or would it add more confusion than it would solve?

Actually, looking more deeply into it, I think the definitive solution isn't too complicated:
- because the edges added between parent models' tests and children models are only really useful for the resolution of execution order during the actual execution (i.e not at compile time)
- we could just add an attribute to these edges in the [add_test_edges method the Compiler](https://github.com/dbt-labs/dbt-core/blob/main/core/dbt/compilation.py#L477)
- and then filter out on this attribute in [dbt.graph.Graph.descendants()](https://github.com/dbt-labs/dbt-core/blob/main/core/dbt/graph/graph.py#L40)

I could give it a go if you want?
@b-luu You're more than welcome to give a go! Conceptually, that approach makes sense to me. It would be worth a quick check to see if the added attribute and filtering step results in a meaningful slowdown at scale

Resolving patch:
diff --git a/core/dbt/compilation.py b/core/dbt/compilation.py
--- a/core/dbt/compilation.py
+++ b/core/dbt/compilation.py
@@ -474,7 +474,7 @@ def add_test_edges(self, linker: Linker, manifest: Manifest) -> None:
                     # is a subset of all upstream nodes of the current node,
                     # add an edge from the upstream test to the current node.
                     if test_depends_on.issubset(upstream_nodes):
-                        linker.graph.add_edge(upstream_test, node_id)
+                        linker.graph.add_edge(upstream_test, node_id, edge_type="parent_test")
 
     def compile(self, manifest: Manifest, write=True, add_test_edges=False) -> Graph:
         self.initialize()
diff --git a/core/dbt/graph/graph.py b/core/dbt/graph/graph.py
--- a/core/dbt/graph/graph.py
+++ b/core/dbt/graph/graph.py
@@ -28,16 +28,29 @@ def ancestors(self, node: UniqueId, max_depth: Optional[int] = None) -> Set[Uniq
         """Returns all nodes having a path to `node` in `graph`"""
         if not self.graph.has_node(node):
             raise DbtInternalError(f"Node {node} not found in the graph!")
+        filtered_graph = self.exclude_edge_type("parent_test")
         return {
             child
-            for _, child in nx.bfs_edges(self.graph, node, reverse=True, depth_limit=max_depth)
+            for _, child in nx.bfs_edges(filtered_graph, node, reverse=True, depth_limit=max_depth)
         }
 
     def descendants(self, node: UniqueId, max_depth: Optional[int] = None) -> Set[UniqueId]:
         """Returns all nodes reachable from `node` in `graph`"""
         if not self.graph.has_node(node):
             raise DbtInternalError(f"Node {node} not found in the graph!")
-        return {child for _, child in nx.bfs_edges(self.graph, node, depth_limit=max_depth)}
+        filtered_graph = self.exclude_edge_type("parent_test")
+        return {child for _, child in nx.bfs_edges(filtered_graph, node, depth_limit=max_depth)}
+
+    def exclude_edge_type(self, edge_type_to_exclude):
+        return nx.restricted_view(
+            self.graph,
+            nodes=[],
+            edges=(
+                (a, b)
+                for a, b in self.graph.edges
+                if self.graph[a][b].get("edge_type") == edge_type_to_exclude
+            ),
+        )
 
     def select_childrens_parents(self, selected: Set[UniqueId]) -> Set[UniqueId]:
         ancestors_for = self.select_children(selected) | selected


Test patch:
diff --git a/tests/functional/build/fixtures.py b/tests/functional/build/fixtures.py
--- a/tests/functional/build/fixtures.py
+++ b/tests/functional/build/fixtures.py
@@ -206,6 +206,27 @@
           - not_null
 """
 
+models_triple_blocking__test_yml = """
+version: 2
+
+models:
+  - name: model_a
+    columns:
+      - name: id
+        tests:
+          - not_null
+  - name: model_b
+    columns:
+      - name: id
+        tests:
+          - not_null
+  - name: model_c
+    columns:
+      - name: id
+        tests:
+          - not_null
+"""
+
 models_interdependent__model_a_sql = """
 select 1 as id
 """
diff --git a/tests/functional/build/test_build.py b/tests/functional/build/test_build.py
--- a/tests/functional/build/test_build.py
+++ b/tests/functional/build/test_build.py
@@ -18,6 +18,7 @@
     models_simple_blocking__model_a_sql,
     models_simple_blocking__model_b_sql,
     models_simple_blocking__test_yml,
+    models_triple_blocking__test_yml,
     models_interdependent__test_yml,
     models_interdependent__model_a_sql,
     models_interdependent__model_b_sql,
@@ -196,3 +197,34 @@ def test_interdependent_models_fail(self, project):
         actual = [str(r.status) for r in results]
         expected = ["error"] * 4 + ["skipped"] * 7 + ["pass"] * 2 + ["success"] * 3
         assert sorted(actual) == sorted(expected)
+
+
+class TestDownstreamSelection:
+    @pytest.fixture(scope="class")
+    def models(self):
+        return {
+            "model_a.sql": models_simple_blocking__model_a_sql,
+            "model_b.sql": models_simple_blocking__model_b_sql,
+            "test.yml": models_simple_blocking__test_yml,
+        }
+
+    def test_downstream_selection(self, project):
+        """Ensure that selecting test+ does not select model_a's other children"""
+        results = run_dbt(["build", "--select", "model_a not_null_model_a_id+"], expect_pass=True)
+        assert len(results) == 2
+
+
+class TestLimitedUpstreamSelection:
+    @pytest.fixture(scope="class")
+    def models(self):
+        return {
+            "model_a.sql": models_interdependent__model_a_sql,
+            "model_b.sql": models_interdependent__model_b_sql,
+            "model_c.sql": models_interdependent__model_c_sql,
+            "test.yml": models_triple_blocking__test_yml,
+        }
+
+    def test_limited_upstream_selection(self, project):
+        """Ensure that selecting 1+model_c only selects up to model_b (+ tests of both)"""
+        results = run_dbt(["build", "--select", "1+model_c"], expect_pass=True)
+        assert len(results) == 4
diff --git a/tests/functional/defer_state/test_run_results_state.py b/tests/functional/defer_state/test_run_results_state.py
--- a/tests/functional/defer_state/test_run_results_state.py
+++ b/tests/functional/defer_state/test_run_results_state.py
@@ -216,9 +216,9 @@ def test_build_run_results_state(self, project):
         results = run_dbt(
             ["build", "--select", "result:fail+", "--state", "./state"], expect_pass=False
         )
-        assert len(results) == 2
+        assert len(results) == 1
         nodes = set([elem.node.name for elem in results])
-        assert nodes == {"table_model", "unique_view_model_id"}
+        assert nodes == {"unique_view_model_id"}
 
         results = run_dbt(["ls", "--select", "result:fail+", "--state", "./state"])
         assert len(results) == 1
@@ -240,9 +240,9 @@ def test_build_run_results_state(self, project):
         results = run_dbt(
             ["build", "--select", "result:warn+", "--state", "./state"], expect_pass=True
         )
-        assert len(results) == 2  # includes table_model to be run
+        assert len(results) == 1
         nodes = set([elem.node.name for elem in results])
-        assert nodes == {"table_model", "unique_view_model_id"}
+        assert nodes == {"unique_view_model_id"}
 
         results = run_dbt(["ls", "--select", "result:warn+", "--state", "./state"])
         assert len(results) == 1
@@ -483,12 +483,11 @@ def test_concurrent_selectors_build_run_results_state(self, project):
             ],
             expect_pass=False,
         )
-        assert len(results) == 5
+        assert len(results) == 4
         nodes = set([elem.node.name for elem in results])
         assert nodes == {
             "error_model",
             "downstream_of_error_model",
             "table_model_modified_example",
-            "table_model",
             "unique_view_model_id",
         }