Repository: dbt-labs/dbt-core

Issue:
[v0.21] Add get_where_subquery to generic test rendering context
This is a bug resulting from the switch in https://github.com/dbt-labs/dbt-core/pull/3859.

Create any generic test containing code like:

```sql
{%- set relation = api.Relation.create(schema=model.schema, identifier=model.table) -%}
{%- set columns = adapter.get_columns_in_relation(relation) -%}
```
Idea being, we want to get the columns present in the model being tested. This worked in previous versions of dbt-core, but starting in v0.21, it leads to an error like:
```
Field "path" of type dbt.contracts.relation.Path in dbt.adapters.postgres.relation.PostgresRelation has invalid value {'database': Undefined, 'schema': Undefined, 'identifier': Undefined}
```

The simplest fix here, from the user's perspective, is to switch the logic above to:
```sql
{%- set relation = api.Relation.create(schema=model.schema, identifier=model.table) if execute -%}
{%- set columns = adapter.get_columns_in_relation(relation) -%}
```

That has no functional effect, since `get_columns_in_relation` doesn't run at parse time anyway.

From our perspective, I _think_ the move is to add `get_where_subquery` to the rendering context for generic tests. Right now, the macro is (I believe) missing from the rendering context, so it's returning `None` instead of a relation, thus leading to the validation error above when we try to create a relation from the properties of `model`. If the macro is in the rendering context, it should return the actual result of `get_where_subquery` instead.

If this is a simple fix, we should try to get it in for v0.21.1 (which has a first release candidate going out today).

Note that this type of generic test will _not_ work if the `where` config is defined, as `get_where_subquery(ref(...))` will return a string (`select * from ref(...) where ...`), not a relation.


Discussion:

Resolving patch:
diff --git a/core/dbt/context/providers.py b/core/dbt/context/providers.py
--- a/core/dbt/context/providers.py
+++ b/core/dbt/context/providers.py
@@ -1414,8 +1414,13 @@ def _build_namespace(self):
     # 'depends_on.macros' by using the TestMacroNamespace
     def _build_test_namespace(self):
         depends_on_macros = []
+        # all generic tests use a macro named 'get_where_subquery' to wrap 'model' arg
+        # see generic_test_builders.build_model_str
+        get_where_subquery = self.macro_resolver.macros_by_name.get('get_where_subquery')
+        if get_where_subquery:
+            depends_on_macros.append(get_where_subquery.unique_id)
         if self.model.depends_on and self.model.depends_on.macros:
-            depends_on_macros = self.model.depends_on.macros
+            depends_on_macros.extend(self.model.depends_on.macros)
         lookup_macros = depends_on_macros.copy()
         for macro_unique_id in lookup_macros:
             lookup_macro = self.macro_resolver.macros.get(macro_unique_id)


Test patch:
diff --git a/test/integration/008_schema_tests_test/test-context-where-subq-macros/custom_generic_test.sql b/test/integration/008_schema_tests_test/test-context-where-subq-macros/custom_generic_test.sql
new file mode 100644
--- /dev/null
+++ b/test/integration/008_schema_tests_test/test-context-where-subq-macros/custom_generic_test.sql
@@ -0,0 +1,12 @@
+/*{# This test will fail if get_where_subquery() is missing from TestContext + TestMacroNamespace #}*/
+
+{% test self_referential(model) %}
+
+    {%- set relation = api.Relation.create(schema=model.schema, identifier=model.table) -%}
+    {%- set columns = adapter.get_columns_in_relation(relation) -%}
+    {%- set columns_csv = columns | map(attribute='name') | list | join(', ') -%}
+
+    select {{ columns_csv }} from {{ model }}
+    limit 0
+
+{% endtest %}
diff --git a/test/integration/008_schema_tests_test/test-context-where-subq-models/model_a.sql b/test/integration/008_schema_tests_test/test-context-where-subq-models/model_a.sql
new file mode 100644
--- /dev/null
+++ b/test/integration/008_schema_tests_test/test-context-where-subq-models/model_a.sql
@@ -0,0 +1 @@
+select 1 as fun
diff --git a/test/integration/008_schema_tests_test/test-context-where-subq-models/schema.yml b/test/integration/008_schema_tests_test/test-context-where-subq-models/schema.yml
new file mode 100644
--- /dev/null
+++ b/test/integration/008_schema_tests_test/test-context-where-subq-models/schema.yml
@@ -0,0 +1,6 @@
+version: 2
+
+models:
+  - name: model_a
+    tests:
+      - self_referential
diff --git a/test/integration/008_schema_tests_test/test_schema_v2_tests.py b/test/integration/008_schema_tests_test/test_schema_v2_tests.py
--- a/test/integration/008_schema_tests_test/test_schema_v2_tests.py
+++ b/test/integration/008_schema_tests_test/test_schema_v2_tests.py
@@ -793,3 +793,30 @@ def test_postgres_wrong_specification_block(self):
 
         assert len(results) == 1
         assert results[0] == '{"name": "some_seed", "description": ""}'
+
+
+class TestSchemaTestContextWhereSubq(DBTIntegrationTest):
+    @property
+    def schema(self):
+        return "schema_tests_008"
+
+    @property
+    def models(self):
+        return "test-context-where-subq-models"
+
+    @property
+    def project_config(self):
+        return {
+            'config-version': 2,
+            "macro-paths": ["test-context-where-subq-macros"],
+        }
+
+    @use_profile('postgres')
+    def test_postgres_test_context_tests(self):
+        # This test tests that get_where_subquery() is included in TestContext + TestMacroNamespace,
+        # otherwise api.Relation.create() will return an error
+        results = self.run_dbt()
+        self.assertEqual(len(results), 1)
+
+        results = self.run_dbt(['test'])
+        self.assertEqual(len(results), 1)