Repository: dbt-labs/dbt-core

Issue:
[CT-1984] [Regression] In v1.4, seeds with hooks calling macros raise an AttributeError
### Is this a regression in a recent version of dbt-core?

- [X] I believe this is a regression in dbt-core functionality
- [X] I have searched the existing issues, and I could not find an existing issue for this regression

### Current Behavior

After trying to upgrade both dbt-utils, dbt-core and dbt-snowflake to 1.0.0, 1.4.1 and 1.4.0 respectively, my project no-longer compiles. 

Downgrading just the dbt parts (core and snowflake) and it compiles. 


### Expected/Previous Behavior

I expect that after following migration guides etc. that the project will compile. 

It compiles as normal if I downgrade dbt to previous version (1.3.2)


### Steps To Reproduce

This is not easily reproducible. It did not affect a new project, just a current one. Something in 1.4.1 is different in the fact that downgrading dbt, the project does compile.

### Relevant log output

```shell
dbt compile  
18:42:38  Running with dbt=1.4.1
18:42:54  Encountered an error:
'SeedNode' object has no attribute 'depends_on'
18:42:54  Traceback (most recent call last):
  File "/Users/stephen.herron/.local/share/virtualenvs/snowflake_dbt-aabVNRJU/lib/python3.10/site-packages/dbt/main.py", line 135, in main
    results, succeeded = handle_and_check(args)
  File "/Users/stephen.herron/.local/share/virtualenvs/snowflake_dbt-aabVNRJU/lib/python3.10/site-packages/dbt/main.py", line 198, in handle_and_check
    task, res = run_from_args(parsed)
  File "/Users/stephen.herron/.local/share/virtualenvs/snowflake_dbt-aabVNRJU/lib/python3.10/site-packages/dbt/main.py", line 245, in run_from_args
    results = task.run()
  File "/Users/stephen.herron/.local/share/virtualenvs/snowflake_dbt-aabVNRJU/lib/python3.10/site-packages/dbt/task/runnable.py", line 454, in run
    self._runtime_initialize()
  File "/Users/stephen.herron/.local/share/virtualenvs/snowflake_dbt-aabVNRJU/lib/python3.10/site-packages/dbt/task/runnable.py", line 165, in _runtime_initialize
    super()._runtime_initialize()
  File "/Users/stephen.herron/.local/share/virtualenvs/snowflake_dbt-aabVNRJU/lib/python3.10/site-packages/dbt/task/runnable.py", line 94, in _runtime_initialize
    self.load_manifest()
  File "/Users/stephen.herron/.local/share/virtualenvs/snowflake_dbt-aabVNRJU/lib/python3.10/site-packages/dbt/task/runnable.py", line 81, in load_manifest
    self.manifest = ManifestLoader.get_full_manifest(self.config)
  File "/Users/stephen.herron/.local/share/virtualenvs/snowflake_dbt-aabVNRJU/lib/python3.10/site-packages/dbt/parser/manifest.py", line 203, in get_full_manifest
    manifest = loader.load()
  File "/Users/stephen.herron/.local/share/virtualenvs/snowflake_dbt-aabVNRJU/lib/python3.10/site-packages/dbt/parser/manifest.py", line 339, in load
    self.parse_project(
  File "/Users/stephen.herron/.local/share/virtualenvs/snowflake_dbt-aabVNRJU/lib/python3.10/site-packages/dbt/parser/manifest.py", line 467, in parse_project
    parser.parse_file(block)
  File "/Users/stephen.herron/.local/share/virtualenvs/snowflake_dbt-aabVNRJU/lib/python3.10/site-packages/dbt/parser/base.py", line 425, in parse_file
    self.parse_node(file_block)
  File "/Users/stephen.herron/.local/share/virtualenvs/snowflake_dbt-aabVNRJU/lib/python3.10/site-packages/dbt/parser/base.py", line 386, in parse_node
    self.render_update(node, config)
  File "/Users/stephen.herron/.local/share/virtualenvs/snowflake_dbt-aabVNRJU/lib/python3.10/site-packages/dbt/parser/base.py", line 363, in render_update
    self.update_parsed_node_config(node, config, context=context)
  File "/Users/stephen.herron/.local/share/virtualenvs/snowflake_dbt-aabVNRJU/lib/python3.10/site-packages/dbt/parser/base.py", line 336, in update_parsed_node_config
    get_rendered(hook.sql, context, parsed_node, capture_macros=True)
  File "/Users/stephen.herron/.local/share/virtualenvs/snowflake_dbt-aabVNRJU/lib/python3.10/site-packages/dbt/clients/jinja.py", line 590, in get_rendered
    return render_template(template, ctx, node)
  File "/Users/stephen.herron/.local/share/virtualenvs/snowflake_dbt-aabVNRJU/lib/python3.10/site-packages/dbt/clients/jinja.py", line 545, in render_template
    return template.render(ctx)
  File "/Users/stephen.herron/.local/share/virtualenvs/snowflake_dbt-aabVNRJU/lib/python3.10/site-packages/jinja2/environment.py", line 1301, in render
    self.environment.handle_exception()
  File "/Users/stephen.herron/.local/share/virtualenvs/snowflake_dbt-aabVNRJU/lib/python3.10/site-packages/jinja2/environment.py", line 936, in handle_exception
    raise rewrite_traceback_stack(source=source)
  File "<template>", line 1, in top-level template code
  File "/Users/stephen.herron/.local/share/virtualenvs/snowflake_dbt-aabVNRJU/lib/python3.10/site-packages/jinja2/sandbox.py", line 393, in call
    return __context.call(__obj, *args, **kwargs)
  File "/Users/stephen.herron/.local/share/virtualenvs/snowflake_dbt-aabVNRJU/lib/python3.10/site-packages/dbt/clients/jinja.py", line 328, in __call__
    with self.track_call():
  File "/Users/stephen.herron/.pyenv/versions/3.10.7/lib/python3.10/contextlib.py", line 135, in __enter__
    return next(self.gen)
  File "/Users/stephen.herron/.local/share/virtualenvs/snowflake_dbt-aabVNRJU/lib/python3.10/site-packages/dbt/clients/jinja.py", line 319, in track_call
    self.node.depends_on.add_macro(unique_id)
AttributeError: 'SeedNode' object has no attribute 'depends_on'
```

### Environment
- OS: MacOS
- Python: 3.10.7
- dbt-core (working version): 1.3.2
- dbt-core (regression version): 1.4.1


### Which database adapter are you using with dbt?

snowflake

### Additional Context

This must be something weird with my project. Out of interest I created a new project in the same virtual env  and that worked.

I will start trying to copy over things to see what is causing the breakage, starting with seeds as the error mentioned `SeedNode`


Discussion:
Think I narrowed it down..

There are seeds in my project that have a post-hook that calls a macro.

e.g.

```
seeds:
  quote_columns: false
   my_project:
     reporting:
       +schema: reporting
       +post-hook:
         - "{{ apply_masking_policy('models') }}"
```

Works ok on previous? Need to try a simpler macro..

I tried changing the post-hook to some generic sql, i.e. "select 1 as test" and the project compiled.

edit: does not work with a `my_silly_macro()` either.


Thanks for opening @rudeb0y !

Agreed that you narrowed it down, and I was able to reproduce the same error using dbt-core==1.4.1 and dbt-snowflake==1.40

Here's what I did to reproduce:

`dbt_project.yaml`
```yaml
name: "my_dbt_project"
version: "1.0.0"
config-version: 2
profile: "my_profile"

seeds:
  +post-hook:
    - "{{ my_silly_macro() }}"
```

`seeds/dummy_seed.csv`
```csv
id
1
```

`macros/my_silly_macro.sql`
```sql
{% macro my_silly_macro() %}
select 1 as test
{% endmacro %}
```

Then run:
```shell
dbt seed -s dummy_seed
```

Then I got the same error as you:
```
AttributeError: 'SeedNode' object has no attribute 'depends_on'
```
Does this remove the `depends_on` attr from seeds? 

https://github.com/dbt-labs/dbt-core/blob/52be5ffaa6688d28234e1871f9707f1bb2b2b311/core/dbt/contracts/util.py#L253
In v1.4, we removed dependency-related attributes from seeds, since they can't actually use `ref`/`source`/`metric`, or call macros directly, in their definition (CSV). That included `depends_on`, `refs`, `sources`, and `metrics`.

_However_, it's true that seeds _can_ have pre- and post-hooks. Those hooks can call macros! It's important we track those dependencies for `dbt seed --select state:modified.macros` ([docs](https://docs.getdbt.com/reference/node-selection/methods#the-state-method)). That's what we're trying to do here, and it's where the error is surfacing:

https://github.com/dbt-labs/dbt-core/blob/b2ea2b8b256e5db1da0b712dfedd7973e1e50a37/core/dbt/clients/jinja.py#L308-L324

---

_Can those macros also call `ref`/`source`/`metric`???_

At first glance, yes. In practice, I sure hope not.

If I try this in `1.3.latest`, adapting @dbeatty10's reproduction case above, it's true that it ""works"":
```sql
{% macro my_silly_macro() %}
select * from {{ ref('my_model') }}
{% endmacro %}
```
I run `dbt seed`, it succeeds:
```
[0m11:36:42.191361 [debug] [Thread-1  ]: On seed.test.seed: 
          insert into "jerco"."dbt_jcohen"."seed" ("id") values
          (%s)
      ...
[0m11:36:42.192621 [debug] [Thread-1  ]: SQL status: INSERT 0 1 in 0.0 seconds
[0m11:36:42.197663 [debug] [Thread-1  ]: Writing runtime SQL for node "seed.test.seed"
[0m11:36:42.212489 [debug] [Thread-1  ]: Using postgres connection "seed.test.seed"
[0m11:36:42.212629 [debug] [Thread-1  ]: On seed.test.seed: /* {"app": "dbt", "dbt_version": "1.3.2", "profile_name": "garage-postgres", "target_name": "dev", "node_id": "seed.test.seed"} */

        select * from "jerco"."dbt_jcohen"."my_model"
```
In `manifest.json`:
```json
...
    "seed.test.seed": {
      "resource_type": "seed",
      "depends_on": {
        "macros": [
          "macro.test.my_silly_macro"
        ],
        "nodes": [
          "model.test.my_model"
        ]
      },
...
```

<img width="562" alt="Screenshot 2023-02-03 at 12 38 41" src="https://user-images.githubusercontent.com/13897643/216594304-d6470cd8-48bd-4ec3-ae8c-1a8e3a5fbbc6.png">

**I hate this.** I'm willing to call it an unsupported & undocumented anti-pattern.

### Proposed resolution

- Add `depends_on` back to seeds, but _only_ `MacroDependsOn` (the same that macros have)
- Raise an explicit error if a seed tries to take a dependency on another resource, via its pre/post hook

I can open a PR for this, with the (bad) code I wrote while trying to understand the codepaths in play—but I definitely want to get @gshank's blessing before moving forward with this approach.

Resolving patch:
diff --git a/core/dbt/contracts/graph/nodes.py b/core/dbt/contracts/graph/nodes.py
--- a/core/dbt/contracts/graph/nodes.py
+++ b/core/dbt/contracts/graph/nodes.py
@@ -37,6 +37,7 @@
 from dbt.contracts.util import Replaceable, AdditionalPropertiesMixin
 from dbt.events.proto_types import NodeInfo
 from dbt.events.functions import warn_or_error
+from dbt.exceptions import ParsingError
 from dbt.events.types import (
     SeedIncreased,
     SeedExceedsLimitSamePath,
@@ -482,6 +483,7 @@ class SeedNode(ParsedNode):  # No SQLDefaults!
     # seeds need the root_path because the contents are not loaded initially
     # and we need the root_path to load the seed later
     root_path: Optional[str] = None
+    depends_on: MacroDependsOn = field(default_factory=MacroDependsOn)
 
     def same_seeds(self, other: "SeedNode") -> bool:
         # for seeds, we check the hashes. If the hashes are different types,
@@ -523,6 +525,39 @@ def empty(self):
         """Seeds are never empty"""
         return False
 
+    def _disallow_implicit_dependencies(self):
+        """Disallow seeds to take implicit upstream dependencies via pre/post hooks"""
+        # Seeds are root nodes in the DAG. They cannot depend on other nodes.
+        # However, it's possible to define pre- and post-hooks on seeds, and for those
+        # hooks to include {{ ref(...) }}. This worked in previous versions, but it
+        # was never officially documented or supported behavior. Let's raise an explicit error,
+        # which will surface during parsing if the user has written code such that we attempt
+        # to capture & record a ref/source/metric call on the SeedNode.
+        # For more details: https://github.com/dbt-labs/dbt-core/issues/6806
+        hooks = [f'- pre_hook: "{hook.sql}"' for hook in self.config.pre_hook] + [
+            f'- post_hook: "{hook.sql}"' for hook in self.config.post_hook
+        ]
+        hook_list = "\n".join(hooks)
+        message = f"""
+Seeds cannot depend on other nodes. dbt detected a seed with a pre- or post-hook
+that calls 'ref', 'source', or 'metric', either directly or indirectly via other macros.
+
+Error raised for '{self.unique_id}', which has these hooks defined: \n{hook_list}
+        """
+        raise ParsingError(message)
+
+    @property
+    def refs(self):
+        self._disallow_implicit_dependencies()
+
+    @property
+    def sources(self):
+        self._disallow_implicit_dependencies()
+
+    @property
+    def metrics(self):
+        self._disallow_implicit_dependencies()
+
     def same_body(self, other) -> bool:
         return self.same_seeds(other)
 
@@ -531,8 +566,8 @@ def depends_on_nodes(self):
         return []
 
     @property
-    def depends_on_macros(self):
-        return []
+    def depends_on_macros(self) -> List[str]:
+        return self.depends_on.macros
 
     @property
     def extra_ctes(self):
diff --git a/core/dbt/contracts/util.py b/core/dbt/contracts/util.py
--- a/core/dbt/contracts/util.py
+++ b/core/dbt/contracts/util.py
@@ -250,7 +250,6 @@ def upgrade_seed_content(node_content):
         "refs",
         "sources",
         "metrics",
-        "depends_on",
         "compiled_path",
         "compiled",
         "compiled_code",
@@ -260,6 +259,8 @@ def upgrade_seed_content(node_content):
     ):
         if attr_name in node_content:
             del node_content[attr_name]
+        # In v1.4, we switched SeedNode.depends_on from DependsOn to MacroDependsOn
+        node_content.get("depends_on", {}).pop("nodes", None)
 
 
 def upgrade_manifest_json(manifest: dict) -> dict:


Test patch:
diff --git a/test/unit/test_contracts_graph_parsed.py b/test/unit/test_contracts_graph_parsed.py
--- a/test/unit/test_contracts_graph_parsed.py
+++ b/test/unit/test_contracts_graph_parsed.py
@@ -432,6 +432,7 @@ def basic_parsed_seed_dict():
         'unique_id': 'seed.test.foo',
         'fqn': ['test', 'seeds', 'foo'],
         'database': 'test_db',
+        'depends_on': {'macros': []},
         'description': '',
         'schema': 'test_schema',
         'tags': [],
@@ -472,6 +473,7 @@ def basic_parsed_seed_object():
         raw_code='',
         unique_id='seed.test.foo',
         fqn=['test', 'seeds', 'foo'],
+        depends_on=MacroDependsOn(),
         database='test_db',
         description='',
         schema='test_schema',
@@ -520,6 +522,7 @@ def complex_parsed_seed_dict():
         'unique_id': 'seed.test.foo',
         'fqn': ['test', 'seeds', 'foo'],
         'database': 'test_db',
+        'depends_on': {'macros': []},
         'description': 'a description',
         'schema': 'test_schema',
         'tags': ['mytag'],
@@ -564,6 +567,7 @@ def complex_parsed_seed_object():
         unique_id='seed.test.foo',
         fqn=['test', 'seeds', 'foo'],
         database='test_db',
+        depends_on=MacroDependsOn(),
         description='a description',
         schema='test_schema',
         tags=['mytag'],
diff --git a/tests/functional/artifacts/expected_manifest.py b/tests/functional/artifacts/expected_manifest.py
--- a/tests/functional/artifacts/expected_manifest.py
+++ b/tests/functional/artifacts/expected_manifest.py
@@ -414,6 +414,7 @@ def expected_seeded_manifest(project, model_database=None, quote_model=False):
                 "fqn": ["test", "seed"],
                 "tags": [],
                 "meta": {},
+                "depends_on": {"macros": []},
                 "schema": my_schema_name,
                 "database": project.database,
                 "alias": "seed",
@@ -1043,6 +1044,7 @@ def expected_references_manifest(project):
                 },
                 "config": get_rendered_seed_config(),
                 "deferred": False,
+                "depends_on": {"macros": []},
                 "description": "The test seed",
                 "docs": {"node_color": None, "show": True},
                 "fqn": ["test", "seed"],
diff --git a/tests/functional/artifacts/test_artifacts.py b/tests/functional/artifacts/test_artifacts.py
--- a/tests/functional/artifacts/test_artifacts.py
+++ b/tests/functional/artifacts/test_artifacts.py
@@ -503,7 +503,6 @@ def test_run_and_generate(self, project, manifest_schema_path, run_results_schem
         start_time = datetime.utcnow()
         results = run_dbt(["compile"])
         assert len(results) == 7
-
         verify_manifest(
             project,
             expected_seeded_manifest(project, quote_model=False),
diff --git a/tests/functional/hooks/test_model_hooks.py b/tests/functional/hooks/test_model_hooks.py
--- a/tests/functional/hooks/test_model_hooks.py
+++ b/tests/functional/hooks/test_model_hooks.py
@@ -2,7 +2,7 @@
 
 from pathlib import Path
 
-from dbt.exceptions import CompilationError
+from dbt.exceptions import CompilationError, ParsingError
 
 from dbt.tests.util import (
     run_dbt,
@@ -251,6 +251,8 @@ def project_config_update(self):
                 "post-hook": [
                     "alter table {{ this }} add column new_col int",
                     "update {{ this }} set new_col = 1",
+                    # call any macro to track dependency: https://github.com/dbt-labs/dbt-core/issues/6806
+                    "select null::{{ dbt.type_int() }} as id",
                 ],
                 "quote_columns": False,
             },
@@ -263,6 +265,36 @@ def test_hooks_on_seeds(self, project):
         assert len(res) == 1, "Expected exactly one item"
 
 
+class TestHooksRefsOnSeeds:
+    """
+    This should not succeed, and raise an explicit error
+    https://github.com/dbt-labs/dbt-core/issues/6806
+    """
+
+    @pytest.fixture(scope="class")
+    def seeds(self):
+        return {"example_seed.csv": seeds__example_seed_csv}
+
+    @pytest.fixture(scope="class")
+    def models(self):
+        return {"schema.yml": properties__seed_models, "post.sql": models__post}
+
+    @pytest.fixture(scope="class")
+    def project_config_update(self):
+        return {
+            "seeds": {
+                "post-hook": [
+                    "select * from {{ ref('post') }}",
+                ],
+            },
+        }
+
+    def test_hook_with_ref_on_seeds(self, project):
+        with pytest.raises(ParsingError) as excinfo:
+            run_dbt(["parse"])
+        assert "Seeds cannot depend on other nodes" in str(excinfo.value)
+
+
 class TestPrePostModelHooksOnSeedsPlusPrefixed(TestPrePostModelHooksOnSeeds):
     @pytest.fixture(scope="class")
     def project_config_update(self):
diff --git a/tests/functional/list/test_list.py b/tests/functional/list/test_list.py
--- a/tests/functional/list/test_list.py
+++ b/tests/functional/list/test_list.py
@@ -380,6 +380,7 @@ def expect_seed_output(self):
                     "incremental_strategy": None,
                     "docs": {"node_color": None, "show": True},
                 },
+                "depends_on": {"macros": []},
                 "unique_id": "seed.test.seed",
                 "original_file_path": normalize("seeds/seed.csv"),
                 "alias": "seed",