Repository: dbt-labs/dbt-core
Pull request: improve error message for parsing args
resolves #5887

<!---
  Include the number of the issue addressed by this PR above if applicable.
  PRs for code changes without an associated issue *will not be merged*.
  See CONTRIBUTING.md for more information.
-->

### Description
Adding more clear error message for parsing error in dbt python model, this is what the new error looks like
```
22:57:25  Running with dbt=1.3.0-b2
22:57:26  Encountered an error:
Parsing Error in model python_model (models/python_model.py)
  Run into the following error when trying to literal_eval an arg 
  malformed node or string: <ast.Name object at 0x14d102970>
  https://docs.python.org/3/library/ast.html#ast.literal_eval
  In dbt python model, `dbt.ref`, `dbt.source`, `dbt.config` function args only support Python literal structures
```
<!---
  Describe the Pull Request here. Add any references and info to help reviewers
  understand your changes. Include any tradeoffs you considered.
-->

### Checklist

- [x] I have read [the contributing guide](https://github.com/dbt-labs/dbt-core/blob/main/CONTRIBUTING.md) and understand what's expected of me
- [x] I have signed the [CLA](https://docs.getdbt.com/docs/contributor-license-agreements)
- [x] I have run this code in development and it appears to resolve the stated issue
- [x] This PR includes tests, or tests are not required/relevant for this PR
- [x] I have [opened an issue to add/update docs](https://github.com/dbt-labs/docs.getdbt.com/issues/new/choose), or docs changes are not required/relevant for this PR
- [x] I have run `changie new` to [create a changelog entry](https://github.com/dbt-labs/dbt-core/blob/main/CONTRIBUTING.md#adding-a-changelog-entry)


Problem statement:
[CT-1215] [Bug] support indirect strings in `ref` and `source` for Python models
### Is this a new bug in dbt-core?

- [X] I believe this is a new bug in dbt-core
- [X] I have searched the existing issues, and I could not find an existing issue for this bug

### Current Behavior

**EDIT**: see comment below. this does not fail due to iterables/lists/dictionaries, but rather us statically analyzing the code and being unable to indirectly use strings via variables. thus an even simpler reproduction:

```python
def model(dbt, session):

    model_name = "orders"

    return dbt.ref(model_name)
```

Take `debug1.py` and `debug2.py`, built off of jaffle_shop, as simple examples:

```python
def model(dbt, session):

    model_names = ["orders", "customers"]
    models = []

    for model_name in model_names:
        models.extend(dbt.ref(model_name))

    return models[0]
```

```python
def model(dbt, session):

    models = {
        "orders": None,
        "customers": None
    } 

    for model_name in models:
        models[model_name] = dbt.ref(model_name)

    return models["orders"]
```

I would expect both of these examples to work, but they fail with parsing error below 

### Expected Behavior

above works

### Steps To Reproduce

```bash
gh repo clone dbt-labs/jaffle_shop
cd jaffle_shop
dbt seed
dbt run
code debug1.py debug2.py
```

copy code above into those files

```bash
dbt run -s debug1
dbt run -s debug2
```

### Relevant log output

```shell
@lostmygithubaccount ➜ .../learn-dbt-py/demos/jaffle_shops/snowpark (main ✗) $ dbt run -s debug1
16:32:45  Running with dbt=1.3.0-b2
16:32:45  Unable to do partial parsing because profile has changed
16:32:46  Encountered an error:
Parsing Error in model debug1 (models/debug1.py)
  malformed node or string on line 8: <ast.Name object at 0x7f262340ba60>
16:32:46  Traceback (most recent call last):
  File "/workspaces/codyspace/dbt-core/core/dbt/parser/models.py", line 88, in _safe_eval
    return ast.literal_eval(node)
  File "/usr/local/lib/python3.10/ast.py", line 108, in literal_eval
    return _convert(node_or_string)
  File "/usr/local/lib/python3.10/ast.py", line 107, in _convert
    return _convert_signed_num(node)
  File "/usr/local/lib/python3.10/ast.py", line 81, in _convert_signed_num
    return _convert_num(node)
  File "/usr/local/lib/python3.10/ast.py", line 72, in _convert_num
    _raise_malformed_node(node)
  File "/usr/local/lib/python3.10/ast.py", line 69, in _raise_malformed_node
    raise ValueError(msg + f': {node!r}')
ValueError: malformed node or string on line 8: <ast.Name object at 0x7f262340ba60>

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/workspaces/codyspace/dbt-core/core/dbt/main.py", line 129, in main
    results, succeeded = handle_and_check(args)
  File "/workspaces/codyspace/dbt-core/core/dbt/main.py", line 191, in handle_and_check
    task, res = run_from_args(parsed)
  File "/workspaces/codyspace/dbt-core/core/dbt/main.py", line 238, in run_from_args
    results = task.run()
  File "/workspaces/codyspace/dbt-core/core/dbt/task/runnable.py", line 453, in run
    self._runtime_initialize()
  File "/workspaces/codyspace/dbt-core/core/dbt/task/runnable.py", line 161, in _runtime_initialize
    super()._runtime_initialize()
  File "/workspaces/codyspace/dbt-core/core/dbt/task/runnable.py", line 94, in _runtime_initialize
    self.load_manifest()
  File "/workspaces/codyspace/dbt-core/core/dbt/task/runnable.py", line 81, in load_manifest
    self.manifest = ManifestLoader.get_full_manifest(self.config)
  File "/workspaces/codyspace/dbt-core/core/dbt/parser/manifest.py", line 220, in get_full_manifest
    manifest = loader.load()
  File "/workspaces/codyspace/dbt-core/core/dbt/parser/manifest.py", line 350, in load
    self.parse_project(
  File "/workspaces/codyspace/dbt-core/core/dbt/parser/manifest.py", line 475, in parse_project
    parser.parse_file(block)
  File "/workspaces/codyspace/dbt-core/core/dbt/parser/base.py", line 414, in parse_file
    self.parse_node(file_block)
  File "/workspaces/codyspace/dbt-core/core/dbt/parser/base.py", line 388, in parse_node
    self.render_update(node, config)
  File "/workspaces/codyspace/dbt-core/core/dbt/parser/models.py", line 226, in render_update
    self.parse_python_model(node, config, context)
  File "/workspaces/codyspace/dbt-core/core/dbt/parser/models.py", line 207, in parse_python_model
    dbtParser.visit(tree)
  File "/usr/local/lib/python3.10/ast.py", line 410, in visit
    return visitor(node)
  File "/usr/local/lib/python3.10/ast.py", line 418, in generic_visit
    self.visit(item)
  File "/usr/local/lib/python3.10/ast.py", line 410, in visit
    return visitor(node)
  File "/usr/local/lib/python3.10/ast.py", line 418, in generic_visit
    self.visit(item)
  File "/usr/local/lib/python3.10/ast.py", line 410, in visit
    return visitor(node)
  File "/usr/local/lib/python3.10/ast.py", line 418, in generic_visit
    self.visit(item)
  File "/usr/local/lib/python3.10/ast.py", line 410, in visit
    return visitor(node)
  File "/usr/local/lib/python3.10/ast.py", line 420, in generic_visit
    self.visit(value)
  File "/usr/local/lib/python3.10/ast.py", line 410, in visit
    return visitor(node)
  File "/workspaces/codyspace/dbt-core/core/dbt/parser/models.py", line 129, in visit_Call
    self.visit_Call(obj)
  File "/workspaces/codyspace/dbt-core/core/dbt/parser/models.py", line 122, in visit_Call
    args, kwargs = self._get_call_literals(node)
  File "/workspaces/codyspace/dbt-core/core/dbt/parser/models.py", line 104, in _get_call_literals
    rendered = self._safe_eval(arg)
  File "/workspaces/codyspace/dbt-core/core/dbt/parser/models.py", line 91, in _safe_eval
    raise ParsingException(msg, node=self.dbt_node) from exc
dbt.exceptions.ParsingException: Parsing Error in model debug1 (models/debug1.py)
  malformed node or string on line 8: <ast.Name object at 0x7f262340ba60>
```

```shell
@lostmygithubaccount ➜ .../learn-dbt-py/demos/jaffle_shops/snowpark (main ✗) $ dbt run -s debug2
16:35:55  Running with dbt=1.3.0-b2
16:35:55  Unable to do partial parsing because profile has changed
16:35:56  Encountered an error:
Parsing Error in model debug2 (models/debug2.py)
  malformed node or string on line 9: <ast.Name object at 0x7f2f0ce29780>
16:35:56  Traceback (most recent call last):
  File "/workspaces/codyspace/dbt-core/core/dbt/parser/models.py", line 88, in _safe_eval
    return ast.literal_eval(node)
  File "/usr/local/lib/python3.10/ast.py", line 108, in literal_eval
    return _convert(node_or_string)
  File "/usr/local/lib/python3.10/ast.py", line 107, in _convert
    return _convert_signed_num(node)
  File "/usr/local/lib/python3.10/ast.py", line 81, in _convert_signed_num
    return _convert_num(node)
  File "/usr/local/lib/python3.10/ast.py", line 72, in _convert_num
    _raise_malformed_node(node)
  File "/usr/local/lib/python3.10/ast.py", line 69, in _raise_malformed_node
    raise ValueError(msg + f': {node!r}')
ValueError: malformed node or string on line 9: <ast.Name object at 0x7f2f0ce29780>

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/workspaces/codyspace/dbt-core/core/dbt/main.py", line 129, in main
    results, succeeded = handle_and_check(args)
  File "/workspaces/codyspace/dbt-core/core/dbt/main.py", line 191, in handle_and_check
    task, res = run_from_args(parsed)
  File "/workspaces/codyspace/dbt-core/core/dbt/main.py", line 238, in run_from_args
    results = task.run()
  File "/workspaces/codyspace/dbt-core/core/dbt/task/runnable.py", line 453, in run
    self._runtime_initialize()
  File "/workspaces/codyspace/dbt-core/core/dbt/task/runnable.py", line 161, in _runtime_initialize
    super()._runtime_initialize()
  File "/workspaces/codyspace/dbt-core/core/dbt/task/runnable.py", line 94, in _runtime_initialize
    self.load_manifest()
  File "/workspaces/codyspace/dbt-core/core/dbt/task/runnable.py", line 81, in load_manifest
    self.manifest = ManifestLoader.get_full_manifest(self.config)
  File "/workspaces/codyspace/dbt-core/core/dbt/parser/manifest.py", line 220, in get_full_manifest
    manifest = loader.load()
  File "/workspaces/codyspace/dbt-core/core/dbt/parser/manifest.py", line 350, in load
    self.parse_project(
  File "/workspaces/codyspace/dbt-core/core/dbt/parser/manifest.py", line 475, in parse_project
    parser.parse_file(block)
  File "/workspaces/codyspace/dbt-core/core/dbt/parser/base.py", line 414, in parse_file
    self.parse_node(file_block)
  File "/workspaces/codyspace/dbt-core/core/dbt/parser/base.py", line 388, in parse_node
    self.render_update(node, config)
  File "/workspaces/codyspace/dbt-core/core/dbt/parser/models.py", line 226, in render_update
    self.parse_python_model(node, config, context)
  File "/workspaces/codyspace/dbt-core/core/dbt/parser/models.py", line 207, in parse_python_model
    dbtParser.visit(tree)
  File "/usr/local/lib/python3.10/ast.py", line 410, in visit
    return visitor(node)
  File "/usr/local/lib/python3.10/ast.py", line 418, in generic_visit
    self.visit(item)
  File "/usr/local/lib/python3.10/ast.py", line 410, in visit
    return visitor(node)
  File "/usr/local/lib/python3.10/ast.py", line 418, in generic_visit
    self.visit(item)
  File "/usr/local/lib/python3.10/ast.py", line 410, in visit
    return visitor(node)
  File "/usr/local/lib/python3.10/ast.py", line 418, in generic_visit
    self.visit(item)
  File "/usr/local/lib/python3.10/ast.py", line 410, in visit
    return visitor(node)
  File "/usr/local/lib/python3.10/ast.py", line 420, in generic_visit
    self.visit(value)
  File "/usr/local/lib/python3.10/ast.py", line 410, in visit
    return visitor(node)
  File "/workspaces/codyspace/dbt-core/core/dbt/parser/models.py", line 122, in visit_Call
    args, kwargs = self._get_call_literals(node)
  File "/workspaces/codyspace/dbt-core/core/dbt/parser/models.py", line 104, in _get_call_literals
    rendered = self._safe_eval(arg)
  File "/workspaces/codyspace/dbt-core/core/dbt/parser/models.py", line 91, in _safe_eval
    raise ParsingException(msg, node=self.dbt_node) from exc
dbt.exceptions.ParsingException: Parsing Error in model debug2 (models/debug2.py)
  malformed node or string on line 9: <ast.Name object at 0x7f2f0ce29780>
```


### Environment

```markdown
- OS: linux
- Python: 3.10
- dbt: off of main branch (1.3)
```


### Which database adapter are you using with dbt?

snowflake

### Additional Context

_No response_

Hints from issue discussion:
this is a bit complicated -- even this would fail:

```python
def model(dbt, session):

    model_name = "orders"

    return dbt.ref(model_name)
```

intermediate step could be more clear error message

Commit message: update error message

Patch to review:
diff --git a/core/dbt/main.py b/core/dbt/main.py
index 0770a41ca8b..d708e5bd050 100644
--- a/core/dbt/main.py
+++ b/core/dbt/main.py
@@ -1,4 +1,5 @@
 from typing import List
+
 from dbt.logger import log_cache_events, log_manager
 
 import argparse
@@ -43,7 +44,12 @@
 
 from dbt.utils import ExitCodes, args_to_dict
 from dbt.config.profile import read_user_config
-from dbt.exceptions import InternalException, NotImplementedException, FailedToConnectException
+from dbt.exceptions import (
+    InternalException,
+    NotImplementedException,
+    FailedToConnectException,
+    ParsingException,
+)
 
 
 class DBTVersion(argparse.Action):
@@ -143,7 +149,8 @@ def main(args=None):
 
         except BaseException as e:
             fire_event(MainEncounteredError(exc=str(e)))
-            fire_event(MainStackTrace(stack_trace=traceback.format_exc()))
+            if not isinstance(e, ParsingException):
+                fire_event(MainStackTrace(stack_trace=traceback.format_exc()))
             exit_code = ExitCodes.UnhandledError.value
 
     sys.exit(exit_code)
diff --git a/core/dbt/parser/models.py b/core/dbt/parser/models.py
index 7f4fe2f26ef..8a2cc814f4b 100644
--- a/core/dbt/parser/models.py
+++ b/core/dbt/parser/models.py
@@ -86,11 +86,12 @@ def _flatten_attr(cls, node):
     def _safe_eval(self, node):
         try:
             return ast.literal_eval(node)
-        except (SyntaxError, ValueError, TypeError) as exc:
-            msg = validator_error_message(exc)
-            raise ParsingException(msg, node=self.dbt_node) from exc
-        except (MemoryError, RecursionError) as exc:
-            msg = validator_error_message(exc)
+        except (SyntaxError, ValueError, TypeError, MemoryError, RecursionError) as exc:
+            msg = validator_error_message(
+                f"Run into the following error when trying to literal_eval an arg \n{exc}\n"
+                "https://docs.python.org/3/library/ast.html#ast.literal_eval\n"
+                "In dbt python model, `dbt.ref`, `dbt.source`, `dbt.config` function args only support Python literal structures"
+            )
             raise ParsingException(msg, node=self.dbt_node) from exc
 
     def _get_call_literals(self, node):
diff --git a/test/unit/test_parser.py b/test/unit/test_parser.py
index 5cc9d1af418..e88bfbc845b 100644
--- a/test/unit/test_parser.py
+++ b/test/unit/test_parser.py
@@ -713,6 +713,24 @@ def test_parse_error(self):
         with self.assertRaises(CompilationException):
             self.parser.parse_file(block)
 
+    def test_parse_ref_with_non_string(self):
+        py_code = """
+def model(dbt, session):
+
+    model_names = ["orders", "customers"]
+    models = []
+
+    for model_name in model_names:
+        models.extend(dbt.ref(model_name))
+
+    return models[0]
+        """
+        block = self.file_block_for(py_code, 'nested/py_model.py')
+        self.parser.manifest.files[block.file.file_id] = block.file
+        with self.assertRaises(ParsingException):
+            self.parser.parse_file(block)    
+    
+
 
 class StaticModelParserTest(BaseParserTest):
     def setUp(self):


Code review comments:
diff_hunk: @@ -143,7 +149,8 @@ def main(args=None):
 
         except BaseException as e:
             fire_event(MainEncounteredError(exc=str(e)))
-            fire_event(MainStackTrace(stack_trace=traceback.format_exc()))
+            if not isinstance(e, ParsingException):
path: core/dbt/main.py
text: @author:
This would avoid generate the whole stack trace after the dbt command finish for Parsing Exception.

diff_hunk: @@ -86,11 +86,12 @@ def _flatten_attr(cls, node):
     def _safe_eval(self, node):
         try:
             return ast.literal_eval(node)
-        except (SyntaxError, ValueError, TypeError) as exc:
-            msg = validator_error_message(exc)
-            raise ParsingException(msg, node=self.dbt_node) from exc
-        except (MemoryError, RecursionError) as exc:
-            msg = validator_error_message(exc)
+        except (SyntaxError, ValueError, TypeError, MemoryError, RecursionError) as exc:
+            msg = validator_error_message(
+                f"Run into the following error when trying to literal_eval an arg \n{exc}\n"
path: core/dbt/parser/models.py
text: @user1:
perhaps just "Error when trying to literal_eval an arg to ref() or source()" here? 

@author:
Updated!