Repository: dbt-labs/dbt-core

Issue:
handle exceptions that occur inside of jinja
## Issue

### Issue description
Given sql with invalid jinja like:
```
{{ config(
    materialized = "table",
    schema = 'custom_schema' -- render this in a custom schema
) }}
```

dbt will throw a very unhelpful error that looks like:
```
  File "/usr/local/Cellar/dbt/0.10.2/libexec/lib/python3.7/site-packages/dbt/parser/base_sql.py", line 95, in parse_sql_nodes
    macros=macros)
  File "/usr/local/Cellar/dbt/0.10.2/libexec/lib/python3.7/site-packages/dbt/parser/base.py", line 105, in parse_node
    capture_macros=True)
  File "/usr/local/Cellar/dbt/0.10.2/libexec/lib/python3.7/site-packages/dbt/clients/jinja.py", line 198, in get_rendered
    return render_template(template, ctx, node)
  File "/usr/local/Cellar/dbt/0.10.2/libexec/lib/python3.7/site-packages/dbt/clients/jinja.py", line 185, in render_template
    return template.render(ctx)
  File "/usr/local/Cellar/dbt/0.10.2/libexec/lib/python3.7/site-packages/jinja2/asyncsupport.py", line 76, in render
    return original_render(self, *args, **kwargs)
  File "/usr/local/Cellar/dbt/0.10.2/libexec/lib/python3.7/site-packages/jinja2/environment.py", line 1008, in render
    return self.environment.handle_exception(exc_info, True)
  File "/usr/local/Cellar/dbt/0.10.2/libexec/lib/python3.7/site-packages/jinja2/environment.py", line 780, in handle_exception
    reraise(exc_type, exc_value, tb)
  File "/usr/local/Cellar/dbt/0.10.2/libexec/lib/python3.7/site-packages/jinja2/_compat.py", line 37, in reraise
    raise value.with_traceback(tb)
  File "<template>", line 1, in top-level template code
TypeError: exceptions must derive from BaseException
```

It will likely be difficult to pinpoint the exact error, but it should ideally be able to throw some sort of human-readable parsing error, indicating the model where the syntax error occurred.


Discussion:
I can't reproduce this on any dbt versions, going back as far as 0.10.2. I consistently get this message:

```
Encountered an error:
Compilation Error in model bad (models/bad.sql)
  expected token ',', got 'this'
    line 3
      schema = 'custom_schema' -- render this in a custom schema
```

Which definitely isn't perfect, but seems pretty helpful to me. Is it possible this was fixed by a jinja update, or something like that?

I do recognize that stack trace as something I've seen in the past, and it's definitely unhelpful.
Are you #wontfix-ing me???

Agreed, I just gave this a spin and couldn't reproduce it either. I didn't see anything in the Jinja changelog that seemed germane to this issue either. Going to close this, but will keep an eye out for this behavior in the wild.
Found a way to trigger this error! In a model that does not define `thing`: `select * from {{ ref(thing + 'a') }}`.
Python version 3.6 for this, I think 2.7 won't have the same issue.

Resolving patch:
diff --git a/dbt/clients/jinja.py b/dbt/clients/jinja.py
--- a/dbt/clients/jinja.py
+++ b/dbt/clients/jinja.py
@@ -195,9 +195,8 @@ class ParserMacroCapture(jinja2.Undefined):
         """
         This class sets up the parser to capture macros.
         """
-        def __init__(self, hint=None, obj=None, name=None,
-                     exc=None):
-            super(jinja2.Undefined, self).__init__()
+        def __init__(self, hint=None, obj=None, name=None, exc=None):
+            super(ParserMacroCapture, self).__init__(hint=hint, name=name)
             self.node = node
             self.name = name
             self.package_name = node.get('package_name')
@@ -211,18 +210,22 @@ def __deepcopy__(self, memo):
                                 self.node.get('original_file_path'))
 
             logger.debug(
-                'A ParserMacroCapture has been deecopy()d, invalid reference '
-                'to "{}" in node {}.{} (source path: {})'
+                'dbt encountered an undefined variable, "{}" in node {}.{} '
+                '(source path: {})'
                 .format(self.name, self.node.get('package_name'),
-                        self.node.get('name'),
-                        path))
+                        self.node.get('name'), path))
 
+            # match jinja's message
             dbt.exceptions.raise_compiler_error(
-                'dbt has detected at least one invalid reference in {}.{}. '
-                'Check logs for more information'
-                .format(self.node.get('package_name'), self.node.get('name'))
+                "{!r} is undefined".format(self.name),
+                node=self.node
             )
 
+        def __getitem__(self, name):
+            # Propagate the undefined value if a caller accesses this as if it
+            # were a dictionary
+            return self
+
         def __getattr__(self, name):
             if name == 'name' or _is_dunder_name(name):
                 raise AttributeError(


Test patch: