Repository: dbt-labs/dbt-core

Issue:
new and old sytle archives clobber each other
## Issue
Archives have no duplicate detection in the manifest like other refables

dbt should error on duplicates, especially duplicate archive names, at parse time.

### Issue description

In 0.14.0, we have a new archive definition feature where you declare archives in `{% archive NAME %}...{% endarchive %}` blocks, and we made archives refable. But I didn't update the graph loader code to detect duplicates, so there are two annoying consequences:

 - If you have an archive defined in dbt_project.yml and an archive with the same name as its `target_table` defined in an archives/*.sql file, the former will always "win", silently
 - Duplicates where the archive name matches a model name will be detected, but only at `ref` time instead of parse time

### System information
dbt 0.14.0a1
any OS
any Python

### Steps to reproduce

in `dbt_project.yml`:

```
archive:
  - source_schema: dbt_jbeck
    target_schema: dbt_jbeck
    tables:
      - source_table: x
        target_table: x_archive
        updated_at: date
        unique_key: id
```

in `archives/x_archive.sql`:

```
{% archive x_archive %}
    {{
        config({'target_database': 'DBT_TEST', 'target_schema': 'dbt_jbeck', 'updated_at': 'date', 'strategy': 'timestamp', 'unique_key': 'id'})
    }}
    select * from DBT_TEST.dbt_jbeck.x
{% endarchive %}
```

Run dbt archive:
```
$ dbt archive
Running with dbt=0.14.0-a1
Found 1 models, 2 tests, 1 archives, 0 analyses, 107 macros, 0 operations, 2 seed files, 1 sources
14:12:24 | Concurrency: 4 threads (target='snowflake')
14:12:24 |
14:12:24 | 1 of 1 START archive x_archive --> DBT_TEST.dbt_jbeck.x_archive...... [RUN]
14:13:29 | 1 of 1 OK archived x_archive --> DBT_TEST.dbt_jbeck.x_archive........ [SUCCESS 100 in 64.21s]
14:13:29 |
14:13:29 | Finished running 1 archives in 81.02s.
```



Discussion:
Good catch @beckjake! Yes, let's add dupe-detection into the loader.

This is probably a less common case, but we also check for duplicate database/schema/identifier identifiers. This check ensures that two models aren't aliases to the same destination relation.

Do we need to do something special to include archives in this check? I imagine that would be especially appropriate since you can specify the target database/schema/table for archives explicitly 
@drewbanin I thought that we didn't handle this, but I think that might already work already:
https://github.com/fishtown-analytics/dbt/blob/73607b85b7825b7fca01a340c52469317eaeae7a/core/dbt/loader.py#L184 - this gets called (eventually) by `GraphLoader.load_all`.

That means the only case we miss is when unique IDs generated by the two archive parsers collide, which should be a pretty easy fix!

Resolving patch:
diff --git a/core/dbt/loader.py b/core/dbt/loader.py
--- a/core/dbt/loader.py
+++ b/core/dbt/loader.py
@@ -63,6 +63,18 @@ def _load_macros(self, internal_manifest=None):
                 resource_type=NodeType.Macro,
             ))
 
+    def _load_archives_from_project(self):
+        archive_parser = ArchiveParser(self.root_project, self.all_projects,
+                                       self.macro_manifest)
+        for key, node in archive_parser.load_and_parse().items():
+            # we have another archive parser, so we have to check for
+            # collisions
+            existing = self.nodes.get(key)
+            if existing:
+                dbt.exceptions.raise_duplicate_resource_name(existing, node)
+            else:
+                self.nodes[key] = node
+
     def _load_seeds(self):
         parser = SeedParser(self.root_project, self.all_projects,
                             self.macro_manifest)
@@ -86,10 +98,7 @@ def _load_nodes(self):
                                  self.macro_manifest)
         self.nodes.update(hook_parser.load_and_parse())
 
-        archive_parser = ArchiveParser(self.root_project, self.all_projects,
-                                       self.macro_manifest)
-        self.nodes.update(archive_parser.load_and_parse())
-
+        self._load_archives_from_project()
         self._load_seeds()
 
     def _load_docs(self):


Test patch:
diff --git a/test/integration/004_simple_archive_test/models-collision/archive_actual.sql b/test/integration/004_simple_archive_test/models-collision/archive_actual.sql
new file mode 100644
--- /dev/null
+++ b/test/integration/004_simple_archive_test/models-collision/archive_actual.sql
@@ -0,0 +1 @@
+select 1 as id
diff --git a/test/integration/004_simple_archive_test/test_simple_archive.py b/test/integration/004_simple_archive_test/test_simple_archive.py
--- a/test/integration/004_simple_archive_test/test_simple_archive.py
+++ b/test/integration/004_simple_archive_test/test_simple_archive.py
@@ -405,6 +405,76 @@ def test__postgres__invalid(self):
         self.assertIn('target_database', str(exc.exception))
 
 
+class TestConflictArchive(DBTIntegrationTest):
+    @property
+    def schema(self):
+        return "simple_archive_004"
+
+    @property
+    def models(self):
+        return "test/integration/004_simple_archive_test/models"
+
+    @property
+    def project_config(self):
+        return {}
+
+
+    @use_profile('postgres')
+    def test__postgres_archive_block_archive_collision(self):
+        self.use_default_project({
+            "archive-paths": ['test/integration/004_simple_archive_test/test-archives-pg'],
+            "archive": [
+                {
+                    "source_schema": self.unique_schema(),
+                    "target_schema": self.unique_schema(),
+                    "tables": [
+                        {
+                            "source_table": "seed",
+                            "target_table": "archive_actual",
+                            "updated_at": 'updated_at',
+                            "unique_key": '''id || '-' || first_name'''
+                        },
+                    ],
+                },
+            ],
+        })
+
+        with self.assertRaises(dbt.exceptions.CompilationException) as exc:
+            self.run_dbt(['compile'], expect_pass=False)
+
+    def test__postgres_archive_block_model_collision(self):
+        self.use_default_project({
+            "source-paths": ['test/integration/004_simple_archive_test/models-collision'],
+            "archive-paths": ['test/integration/004_simple_archive_test/test-archives-pg'],
+            "archive": [],
+        })
+
+        with self.assertRaises(dbt.exceptions.CompilationException) as exc:
+            self.run_dbt(['compile'], expect_pass=False)
+
+    def test__postgres_archive_model_collision(self):
+        self.use_default_project({
+            "source-paths": ['test/integration/004_simple_archive_test/models-collision'],
+            "archive": [
+                {
+                    "source_schema": self.unique_schema(),
+                    "target_schema": self.unique_schema(),
+                    "tables": [
+                        {
+                            "source_table": "seed",
+                            "target_table": "archive_actual",
+                            "updated_at": 'updated_at',
+                            "unique_key": '''id || '-' || first_name'''
+                        },
+                    ],
+                },
+            ],
+        })
+
+        with self.assertRaises(dbt.exceptions.CompilationException) as exc:
+            self.run_dbt(['compile'], expect_pass=False)
+
+
 class TestCheckCols(TestSimpleArchiveFiles):
     NUM_ARCHIVE_MODELS = 2
     def _assertTablesEqualSql(self, relation_a, relation_b, columns=None):