Repository: dbt-labs/dbt-core

Issue:
[CT-677] [Bug] Downstream models do not run correctly after NetworkX 2.8.1
### Is there an existing issue for this?

- [X] I have searched the existing issues

### Current Behavior

Any 1.0.x version of dbt installed after NetworkX 2.8.1 was released does not correctly run downstream models when using selection syntax.  For some reason this does not seem to effect 1.1

### Expected Behavior

All downstream models are run

### Steps To Reproduce

Install dbt-<whatever> @ v1.0.x
Create a project with a `model_foo` and downstream `model_bar`
Run dbt with selection `dbt run --select +model_foo+`
Observe that only `model_foo` is run

### Relevant log output

_No response_

### Environment

```markdown
- OS: MacOS
- Python: 3.10
- dbt: 1.0.6
```


### What database are you using dbt with?

postgres

### Additional Context

_No response_


Discussion:
https://github.com/networkx/networkx/milestone/21?closed=1 -- looks like 2.8.2 of networkx is incoming...
2.8.1 broke, but 2.8.2 did not fix whatever is now wrong with the graph stuff and newtorkx.
After quite a few hours of investigation, this is proving a VERY difficult bug to track down.
Here's a synopsis of the work so far.

### We're doing this wrong
The issue can be traced back to our use of `nx.single_source_shortest_path_length` in dbt-core <`1.1.x` [here](https://github.com/dbt-labs/dbt-core/blob/6a772f185d4617461a64e9efee59270396a06cdb/core/dbt/graph/graph.py#L35) and [here](https://github.com/dbt-labs/dbt-core/blob/6a772f185d4617461a64e9efee59270396a06cdb/core/dbt/graph/graph.py#L47).

Specifically, it's used to return a list of a given node's descendants, and by reversing the graph first, it's ancestors.  However, that isn't really what that function is [supposed to do](https://networkx.org/documentation/stable/reference/algorithms/generated/networkx.algorithms.shortest_paths.unweighted.single_source_shortest_path_length.html)!  Using the DAG-specific `nx.ancestors` and `nx.descendants` works correctly in all versions and should have probably been what we did in the first place.  The only downside to this approach is that `nx.desc` / `nx.ances` don't offer a max depth option but it's a simple enough thing to implement.

### There's still a bug, even when we do it the "right way"
In researching this it was discovered that doing any kind of iteration over `self.graph` immediately before applying `nx.single_source_shortest_path_length` seems to "fix" it.  This explains why `dbt-core>=1.1` works correctly because we changed the method used to reverse the graph to one that actually iterates over it.  **This makes absolutely no sense** since iterating over the graph should not mutate it in any way!!!

I suspect that either we are constructing the initial graph in such a way that causes problems unless it's iterated over OR there's a bug in networkx somewhere very very very not obvious.  Unfortunately, I wasn't able to definitively pin down the source of this bug despite several days worth of effort.





Resolving patch:
diff --git a/core/dbt/graph/graph.py b/core/dbt/graph/graph.py
--- a/core/dbt/graph/graph.py
+++ b/core/dbt/graph/graph.py
@@ -28,20 +28,16 @@ def ancestors(self, node: UniqueId, max_depth: Optional[int] = None) -> Set[Uniq
         """Returns all nodes having a path to `node` in `graph`"""
         if not self.graph.has_node(node):
             raise InternalException(f"Node {node} not found in the graph!")
-        # This used to use nx.utils.reversed(self.graph), but that is deprecated,
-        # so changing to use self.graph.reverse(copy=False) as recommeneded
-        G = self.graph.reverse(copy=False) if self.graph.is_directed() else self.graph
-        anc = nx.single_source_shortest_path_length(G=G, source=node, cutoff=max_depth).keys()
-        return anc - {node}
+        return {
+            child
+            for _, child in nx.bfs_edges(self.graph, node, reverse=True, depth_limit=max_depth)
+        }
 
     def descendants(self, node: UniqueId, max_depth: Optional[int] = None) -> Set[UniqueId]:
         """Returns all nodes reachable from `node` in `graph`"""
         if not self.graph.has_node(node):
             raise InternalException(f"Node {node} not found in the graph!")
-        des = nx.single_source_shortest_path_length(
-            G=self.graph, source=node, cutoff=max_depth
-        ).keys()
-        return des - {node}
+        return {child for _, child in nx.bfs_edges(self.graph, node, depth_limit=max_depth)}
 
     def select_childrens_parents(self, selected: Set[UniqueId]) -> Set[UniqueId]:
         ancestors_for = self.select_children(selected) | selected


Test patch: