Repository: dbt-labs/dbt-core

Issue:
model test compilation context does not have the var method in scope
### Describe the bug
The `model` test compilation context does not have the `var` method in scope, but this does work correctly in sources.
In the following example, I have 2 yml files under `model/staging/user`.
1. `source_models.yml`
2. `models.yml`

```
model/staging/user:
|
|-  source_models.yml:
|     - schema: my_source_schema
|     - name: user_properties
|       columns:
|         - name: user_id
|           tests:
|             - relationships_where:
|                 to: source('user', 'all_users')
|                 field: user_id
|                 from_condition: created_at::date <= '{{ var('created_at__date') }}'
|
|
|-  models.yml:
|     - name: staging_user_properties
|       columns:
|         - name: user_id
|           tests:
|             - relationships_where:
|                 to: ref('stg_users')
|                 field: user_id
|                 from_condition: created_at::date <= '{{ var('created_at__date') }}'
|
|
|- stg_users.sql
```
If I compile the project with `dbt compile --vars '{"created_at__date": "2020-01-01"}'` I get:
```
Compilation Error in test relationships_where_staging_user_properties_user_id__user_id__created_at__date_var_created_at__date___ref_stg_users_ (models/staging/user/user_models.yml)
    'var' is undefined`
```
`var` is not available in `models.yml` but it's available in `source_models.yml`

`var` should have same scope in both.

### Steps To Reproduce
1. create a `source` schema test with a dynamic variable as parameter (in the above example, it's `created_at__date`)
2. create a model test with the same dynamic variable

compile the project with `--vars {"key": "value"}`

Compilation will fail for model test with the error:
`'var' is undefined`

### Expected behavior
It should compile successfully for both `source` and `model` tests with the given parameters

sources and models (and other resource types) should all have access to the var method 

### Screenshots and log output
If applicable, add screenshots or log output to help explain your problem.

### System information
**Which database are you using dbt with?**
- [ ] postgres
- [x] redshift
- [ ] bigquery
- [ ] snowflake
- [ ] other (specify: ____________)


**The output of `dbt --version`:**
```
installed version: 0.13.1
   latest version: 0.15.2
```

**The operating system you're using:**
macOS Mojave
Version: 10.14.6

**The output of `python --version`:**
Python 3.7.0
### Additional context

https://getdbt.slack.com/archives/C2JRRQDTL/p1581354620304600



Discussion:
this *might* be fixed in 0.16 -- @beckjake to check
Not fixed in 0.16!

I'm not sure of the exact cause yet, but there is some very fishy rendering code in `dbt.compilation.compile_node` that generates what I'd consider a bad-looking context. I'm also not sure why, but source test args appear to be eagerly evaluated while model test args are lazily evaluated - this probably comes down to parser things and which `compile_node` we call?

@drewbanin - what's the desired behavior here? I kind of think args should be rendered at compile time for both sources and models, in the same context that will render the test itself (otherwise, how would `ref` work?).

Alternatively, we could fix how tests work, removing the whole idea of the `wrapped_sql` field. I would be pretty interested in that, but it's not a small project.
Phew, so this one's fun!
What happens on schemas is that they run through `ConfigRenderer.render_schema_source`, which models don't do. Ultimately this renders the test block of a source declaration using the target context, but not the test block of a model.

I think for parity and sanity, we should make the following changes:
 - config rendering should skip `test` blocks when it's rendering `sources`
 - arguments should not be rendered until runtime
 - at runtime, arguments should be rendered in the same context as the test itself

That will allow `ref` to work, which seems... mostly right. I think it'll be complicated.


This problem is a bit more complicated, and has some design/behavior implications.

The reason this works for sources is because dbt renders the field during source parsing (it renders everything except descriptions), and `var` is in scope.

Models don't do that, so this doesn't work.

However, on model tests (and source tests!) we have an explicit `is_function` check where we look at a test kwarg and decide if it's probably a function. If it is, we render the kwarg as `{key}={value}`. If it isn't, we render the kwarg as `{key}={repr(value)}`.

Because the strings here don't match the regex (and even have curly braces), the test parsing logic passes `'created_at::date <= '{{ var('created_at__date') }}'` along as a function argument:

```
test_relationships_where(
  to=source('user', 'all_users'),
  field="user_id",
  from_condition="created_at::date <= '{{ var('created_at__date') }}'"
)
```

and the `from_condition` kwarg is never rendered.

I still think the thing that's actually wrong here is sources being rendered. Unfortunately, with the way dbt is currently designed, you aren't really supposed to be able to do this - the `from_condition` would have to be only `var('created_at_date')` and you'd have to write a new test macro to do the `created_at::date <= ` interpolation, or something. I'm not sure that's how things ought to be, we can change that now if we like.

So the important questions that need to be answered before we proceed are:
 - should test args be rendered?
 - If args should be rendered, in what context?
 - if args should be rendered, what happens to the existing `source(...)` or `var(...)` syntax? Is it still supported?


Resolving patch:
diff --git a/core/dbt/compilation.py b/core/dbt/compilation.py
--- a/core/dbt/compilation.py
+++ b/core/dbt/compilation.py
@@ -248,7 +248,6 @@ def _is_writable(node):
 def compile_node(adapter, config, node, manifest, extra_context, write=True):
     compiler = Compiler(config)
     node = compiler.compile_node(node, manifest, extra_context)
-    node = _inject_runtime_config(adapter, node, extra_context)
 
     if write and _is_writable(node):
         logger.debug('Writing injected SQL for node "{}"'.format(
@@ -261,21 +260,3 @@ def compile_node(adapter, config, node, manifest, extra_context, write=True):
         )
 
     return node
-
-
-def _inject_runtime_config(adapter, node, extra_context):
-    wrapped_sql = node.wrapped_sql
-    context = _node_context(adapter, node)
-    context.update(extra_context)
-    sql = dbt.clients.jinja.get_rendered(wrapped_sql, context)
-    node.wrapped_sql = sql
-    return node
-
-
-def _node_context(adapter, node):
-    if dbt.tracking.active_user is not None:
-        return {
-            "run_started_at": dbt.tracking.active_user.run_started_at,
-            "invocation_id": dbt.tracking.active_user.invocation_id,
-        }
-    return {}   # this never happens, but make mypy happy
diff --git a/core/dbt/config/renderer.py b/core/dbt/config/renderer.py
--- a/core/dbt/config/renderer.py
+++ b/core/dbt/config/renderer.py
@@ -72,11 +72,25 @@ def _render_profile_data(self, value, keypath):
                 pass
         return result
 
+    @staticmethod
+    def _is_schema_test(keypath) -> bool:
+        # we got passed an UnparsedSourceDefinition
+        if len(keypath) > 2 and keypath[0] == 'tables':
+            if keypath[2] == 'tests':
+                return True
+            elif keypath[2] == 'columns':
+                if len(keypath) > 4 and keypath[4] == 'tests':
+                    return True
+        return False
+
     def _render_schema_source_data(self, value, keypath):
         # things to not render:
         # - descriptions
+        # - test arguments
         if len(keypath) > 0 and keypath[-1] == 'description':
             return value
+        elif self._is_schema_test(keypath):
+            return value
 
         return self.render_value(value)
 


Test patch:
diff --git a/test/integration/029_docs_generate_tests/test_docs_generate.py b/test/integration/029_docs_generate_tests/test_docs_generate.py
--- a/test/integration/029_docs_generate_tests/test_docs_generate.py
+++ b/test/integration/029_docs_generate_tests/test_docs_generate.py
@@ -962,7 +962,7 @@ def expected_seeded_manifest(self, model_database=None):
                     'extra_ctes_injected': True,
                     'extra_ctes': [],
                     'injected_sql': ANY,
-                    'wrapped_sql': 'None',
+                    'wrapped_sql': None,
                 },
                 'seed.test.seed': {
                     'build_path': None,
@@ -1044,7 +1044,7 @@ def expected_seeded_manifest(self, model_database=None):
                     'extra_ctes_injected': True,
                     'extra_ctes': [],
                     'injected_sql': '',
-                    'wrapped_sql': 'None',
+                    'wrapped_sql': None,
                 },
                 'test.test.not_null_model_id': {
                     'alias': 'not_null_model_id',
@@ -1373,7 +1373,7 @@ def expected_postgres_references_manifest(self, model_database=None):
                     'extra_ctes_injected': True,
                     'extra_ctes': [],
                     'injected_sql': ANY,
-                    'wrapped_sql': 'None',
+                    'wrapped_sql': None,
                 },
                 'model.test.ephemeral_summary': {
                     'alias': 'ephemeral_summary',
@@ -1437,7 +1437,7 @@ def expected_postgres_references_manifest(self, model_database=None):
                     'extra_ctes_injected': True,
                     'extra_ctes': [ANY],
                     'injected_sql': ANY,
-                    'wrapped_sql': 'None',
+                    'wrapped_sql': None,
                 },
                 'model.test.view_summary': {
                     'alias': 'view_summary',
@@ -1500,7 +1500,7 @@ def expected_postgres_references_manifest(self, model_database=None):
                     'extra_ctes_injected': True,
                     'extra_ctes': [],
                     'injected_sql': ANY,
-                    'wrapped_sql': 'None',
+                    'wrapped_sql': None,
                 },
                 'seed.test.seed': {
                     'alias': 'seed',
@@ -1579,7 +1579,7 @@ def expected_postgres_references_manifest(self, model_database=None):
                     'extra_ctes_injected': True,
                     'extra_ctes': [],
                     'injected_sql': '',
-                    'wrapped_sql': 'None',
+                    'wrapped_sql': None,
                 },
                 'source.test.my_source.my_table': {
                     'columns': {
@@ -1952,7 +1952,7 @@ def expected_bigquery_complex_manifest(self):
                     'extra_ctes_injected': True,
                     'extra_ctes': [],
                     'injected_sql': ANY,
-                    'wrapped_sql': 'None',
+                    'wrapped_sql': None,
                 },
                 'model.test.multi_clustered': {
                     'alias': 'multi_clustered',
@@ -2031,7 +2031,7 @@ def expected_bigquery_complex_manifest(self):
                     'extra_ctes_injected': True,
                     'extra_ctes': [],
                     'injected_sql': ANY,
-                    'wrapped_sql': 'None',
+                    'wrapped_sql': None,
                 },
                 'model.test.nested_view': {
                     'alias': 'nested_view',
@@ -2111,7 +2111,7 @@ def expected_bigquery_complex_manifest(self):
                     'extra_ctes_injected': True,
                     'extra_ctes': [],
                     'injected_sql': ANY,
-                    'wrapped_sql': 'None',
+                    'wrapped_sql': None,
                 },
                 'model.test.nested_table': {
                     'alias': 'nested_table',
@@ -2155,7 +2155,7 @@ def expected_bigquery_complex_manifest(self):
                     'extra_ctes_injected': True,
                     'extra_ctes': [],
                     'injected_sql': ANY,
-                    'wrapped_sql': 'None',
+                    'wrapped_sql': None,
                 },
                 'seed.test.seed': {
                     'build_path': None,
@@ -2237,7 +2237,7 @@ def expected_bigquery_complex_manifest(self):
                     'extra_ctes_injected': True,
                     'extra_ctes': [],
                     'injected_sql': '',
-                    'wrapped_sql': 'None',
+                    'wrapped_sql': None,
                 },
             },
             'child_map': {
@@ -2462,7 +2462,7 @@ def expected_redshift_incremental_view_manifest(self):
                     'extra_ctes_injected': True,
                     'extra_ctes': [],
                     'injected_sql': ANY,
-                    'wrapped_sql': 'None',
+                    'wrapped_sql': None,
                 },
                 'seed.test.seed': {
                     'build_path': None,
@@ -2544,7 +2544,7 @@ def expected_redshift_incremental_view_manifest(self):
                     'extra_ctes_injected': True,
                     'extra_ctes': [],
                     'injected_sql': ANY,
-                    'wrapped_sql': 'None',
+                    'wrapped_sql': None,
                 },
             },
             'parent_map': {
@@ -2784,7 +2784,7 @@ def expected_run_results(self, quote_schema=True, quote_model=False,
                     'database': model_database,
                     'tags': [],
                     'unique_id': 'model.test.model',
-                    'wrapped_sql': 'None'
+                    'wrapped_sql': None
                 },
                 'thread_id': ANY,
                 'timing': [ANY, ANY],
@@ -2873,7 +2873,7 @@ def expected_run_results(self, quote_schema=True, quote_model=False,
                     'database': self.default_database,
                     'tags': [],
                     'unique_id': 'seed.test.seed',
-                    'wrapped_sql': 'None',
+                    'wrapped_sql': None,
                 },
                 'thread_id': ANY,
                 'timing': [ANY, ANY],
@@ -3163,7 +3163,7 @@ def expected_postgres_references_run_results(self):
                     'database': self.default_database,
                     'tags': [],
                     'unique_id': 'model.test.ephemeral_summary',
-                    'wrapped_sql': 'None',
+                    'wrapped_sql': None,
                 },
                 'thread_id': ANY,
                 'timing': [ANY, ANY],
@@ -3242,7 +3242,7 @@ def expected_postgres_references_run_results(self):
                     'database': self.default_database,
                     'tags': [],
                     'unique_id': 'model.test.view_summary',
-                    'wrapped_sql': 'None',
+                    'wrapped_sql': None,
                 },
                 'thread_id': ANY,
                 'timing': [ANY, ANY],
@@ -3331,7 +3331,7 @@ def expected_postgres_references_run_results(self):
                     'database': self.default_database,
                     'tags': [],
                     'unique_id': 'seed.test.seed',
-                    'wrapped_sql': 'None'
+                    'wrapped_sql': None
                 },
                 'thread_id': ANY,
                 'timing': [ANY, ANY],
diff --git a/test/integration/042_sources_test/malformed_models/schema.yml b/test/integration/042_sources_test/malformed_models/schema.yml
--- a/test/integration/042_sources_test/malformed_models/schema.yml
+++ b/test/integration/042_sources_test/malformed_models/schema.yml
@@ -8,7 +8,7 @@ sources:
         identifier: source
         tests:
           - relationships:
-            # this is invalid
+            # this is invalid (list of 3 1-key dicts instead of a single 3-key dict)
               - column_name: favorite_color
               - to: ref('descendant_model')
               - field: favorite_color
diff --git a/test/integration/042_sources_test/malformed_schema_tests/model.sql b/test/integration/042_sources_test/malformed_schema_tests/model.sql
new file mode 100644
--- /dev/null
+++ b/test/integration/042_sources_test/malformed_schema_tests/model.sql
@@ -0,0 +1 @@
+select * from {{ source('test_source', 'test_table') }}
diff --git a/test/integration/042_sources_test/malformed_schema_tests/schema.yml b/test/integration/042_sources_test/malformed_schema_tests/schema.yml
new file mode 100644
--- /dev/null
+++ b/test/integration/042_sources_test/malformed_schema_tests/schema.yml
@@ -0,0 +1,14 @@
+version: 2
+sources:
+  - name: test_source
+    schema: "{{ var('test_run_schema') }}"
+    tables:
+      - name: test_table
+        identifier: source
+        columns:
+          - name: favorite_color
+            tests:
+              - relationships:
+                  to: ref('model')
+                  # this will get rendered as its literal
+                  field: "{{ 'favorite' ~ 'color' }}"
diff --git a/test/integration/042_sources_test/test_sources.py b/test/integration/042_sources_test/test_sources.py
--- a/test/integration/042_sources_test/test_sources.py
+++ b/test/integration/042_sources_test/test_sources.py
@@ -377,6 +377,19 @@ def test_postgres_malformed_schema_strict_will_break_run(self):
             self.run_dbt_with_vars(['seed'], strict=True)
 
 
+class TestRenderingInSourceTests(BaseSourcesTest):
+    @property
+    def models(self):
+        return "malformed_schema_tests"
+
+    @use_profile('postgres')
+    def test_postgres_render_in_source_tests(self):
+        self.run_dbt_with_vars(['seed'])
+        self.run_dbt_with_vars(['run'])
+        # syntax error at or near "{", because the test isn't rendered
+        self.run_dbt_with_vars(['test'], expect_pass=False)
+
+
 class TestUnquotedSources(SuccessfulSourcesTest):
     @property
     def project_config(self):
diff --git a/test/unit/test_contracts_graph_compiled.py b/test/unit/test_contracts_graph_compiled.py
--- a/test/unit/test_contracts_graph_compiled.py
+++ b/test/unit/test_contracts_graph_compiled.py
@@ -139,7 +139,6 @@ def test_basic_compiled(self):
             'extra_ctes': [{'id': 'whatever', 'sql': 'select * from other'}],
             'extra_ctes_injected': True,
             'injected_sql': 'with whatever as (select * from other) select * from whatever',
-            'wrapped_sql': 'None',
         }
         node = self.ContractType(
             package_name='test',
@@ -166,7 +165,6 @@ def test_basic_compiled(self):
             extra_ctes=[InjectedCTE('whatever', 'select * from other')],
             extra_ctes_injected=True,
             injected_sql='with whatever as (select * from other) select * from whatever',
-            wrapped_sql='None',
         )
         self.assert_symmetric(node, node_dict)
         self.assertFalse(node.empty)