Repository: dbt-labs/dbt-core

Issue:
[CT-722] `DuplicateYamlKeyException` needs to be warning, not (broken) exception
Net-new bug in v1.2. See also https://github.com/dbt-labs/dbt-core/issues/5268, which I believe is caused by the same root change (https://github.com/dbt-labs/dbt-core/pull/5146).

https://github.com/dbt-labs/dbt-core/blob/eea872c31994c90e995405c74496c1ce20e07025/core/dbt/clients/yaml_helper.py#L75-L88

If `path=None`, it has no `searched_path` / `relative_path`. If I change that to 

### Reproduction case

```yml
# dbt_project.yml
vars:
  one_var: foo
  one_var: bar
```
```
$ dbt parse
17:20:39  Encountered an error:
'NoneType' object has no attribute 'searched_path'
```

If I change the code linked above to:
```python
     except dbt.exceptions.DuplicateYamlKeyException as e: 
         # TODO: We may want to raise an exception instead of a warning in the future. 
         if path:
             e.msg = f"{e} {path.searched_path}/{path.relative_path}." 
         dbt.exceptions.warn_or_raise(e, log_fmt=warning_tag("{}")) 
```

```
$ dbt parse
17:21:36  [WARNING]: Compilation Error
  Duplicate 'one_var' key found in yaml file
17:21:36  Encountered an error while reading the project:
17:21:36    ERROR: Runtime Error
  dbt_project.yml does not parse to a dictionary
17:21:36  Encountered an error:
Runtime Error
  Could not run dbt
```

This is still a problem. This parses successfully in v1.1. We MUST make this a warning, not an error.

We can do that by using `SafeLoader` instead of `UniqueKeyLoader`, and still returning the loaded result (if not `--warn-error`):
```python
    except dbt.exceptions.DuplicateYamlKeyException as e:
        if path:
            # TODO: We may want to raise an exception instead of a warning in the future.
            e.msg = f"{e} {path.searched_path}/{path.relative_path}."
        dbt.exceptions.warn_or_raise(e, log_fmt=warning_tag("{}"))
        return yaml.load(contents, Loader=SafeLoader)
```


Discussion:

Resolving patch:
diff --git a/core/dbt/clients/yaml_helper.py b/core/dbt/clients/yaml_helper.py
--- a/core/dbt/clients/yaml_helper.py
+++ b/core/dbt/clients/yaml_helper.py
@@ -8,7 +8,6 @@
 except ImportError:
     from yaml import Loader, SafeLoader, Dumper  # type: ignore  # noqa: F401
 
-from dbt.ui import warning_tag
 
 YAML_ERROR_MESSAGE = """
 Syntax error near line {line_number}
@@ -21,26 +20,6 @@
 """.strip()
 
 
-class UniqueKeyLoader(SafeLoader):
-    """A subclass that checks for unique yaml mapping nodes.
-
-    This class extends `SafeLoader` from the `yaml` library to check for
-    unique top level keys (mapping nodes). See issue (https://github.com/yaml/pyyaml/issues/165)
-    and solution (https://gist.github.com/pypt/94d747fe5180851196eb?permalink_comment_id=4015118).
-    """
-
-    def construct_mapping(self, node, deep=False):
-        mapping = set()
-        for key_node, value_node in node.value:
-            key = self.construct_object(key_node, deep=deep)
-            if key in mapping:
-                raise dbt.exceptions.DuplicateYamlKeyException(
-                    f"Duplicate {key!r} key found in yaml file"
-                )
-            mapping.add(key)
-        return super().construct_mapping(node, deep)
-
-
 def line_no(i, line, width=3):
     line_number = str(i).ljust(width)
     return "{}| {}".format(line_number, line)
@@ -69,7 +48,7 @@ def contextualized_yaml_error(raw_contents, error):
 
 
 def safe_load(contents) -> Optional[Dict[str, Any]]:
-    return yaml.load(contents, Loader=UniqueKeyLoader)
+    return yaml.load(contents, Loader=SafeLoader)
 
 
 def load_yaml_text(contents, path=None):
@@ -82,7 +61,3 @@ def load_yaml_text(contents, path=None):
             error = str(e)
 
         raise dbt.exceptions.ValidationException(error)
-    except dbt.exceptions.DuplicateYamlKeyException as e:
-        # TODO: We may want to raise an exception instead of a warning in the future.
-        e.msg = f"{e} {path.searched_path}/{path.relative_path}."
-        dbt.exceptions.warn_or_raise(e, log_fmt=warning_tag("{}"))


Test patch:
diff --git a/tests/functional/duplications/test_basic_duplications.py b/tests/functional/duplications/test_basic_duplications.py
deleted file mode 100644
--- a/tests/functional/duplications/test_basic_duplications.py
+++ /dev/null
@@ -1,33 +0,0 @@
-import pytest
-
-from dbt.tests.util import run_dbt_and_capture, run_dbt
-from dbt.exceptions import DuplicateYamlKeyException
-
-duplicate_key_schema__schema_yml = """
-version: 2
-models:
-  - name: my_model
-models:
-  - name: my_model
-"""
-
-my_model_sql = """
-  select 1 as fun
-"""
-
-
-class TestBasicDuplications:
-    @pytest.fixture(scope="class")
-    def models(self):
-        return {
-            "my_model.sql": my_model_sql,
-            "schema.yml": duplicate_key_schema__schema_yml,
-        }
-
-    def test_warning_in_stdout(self, project):
-        results, stdout = run_dbt_and_capture(["run"])
-        assert "Duplicate 'models' key found in yaml file models/schema.yml" in stdout
-
-    def test_exception_is_raised_with_warn_error_flag(self, project):
-        with pytest.raises(DuplicateYamlKeyException):
-            run_dbt(["--warn-error", "run"])