Repository: dbt-labs/dbt-core

Issue:
[CORE-20] [Bug] Jobs running on dbt 1.0.0 are failing when setup to defer to a pre 1.0.0 manifest
### Is there an existing issue for this?

- [X] I have searched the existing issues

### Current Behavior

When a job with `defer` set is updated from a pre 1.0.0 version of dbt-core to 1.0.0, the job fails with the following error:

`Field "metrics" of type Mapping[str, ParsedMetric] is missing in WritableManifest instance`-

### Expected Behavior

Jobs with `defer` set should be able to be updated to dbt-core 1.0.0 without error.

### Steps To Reproduce

1. Have a job set to 0.21.1 (last pre-release)
2. Enable the defer option to defer to previous manifests of the same job
3. Run the job
4. Upgrade the job to 1.0.0
5. Run the job again

### Relevant log output

```shell
`Field "metrics" of type Mapping[str, ParsedMetric] is missing in WritableManifest instance`-
```


### Environment

```markdown
- OS:
- Python:
- dbt:
```


### What database are you using dbt with?

_No response_

### Additional Context

_No response_


Discussion:
Having the same issue in dbt cloud, works fine in the CLI.

<img width="2342" alt="ScreenShot 2021-12-10 at 15 52 46@2x" src="https://user-images.githubusercontent.com/55678713/145655081-12ae38b5-1447-43c1-82a5-85e899017d41.png">

This is a tricky chicken-and-egg problem. Artifacts are versioned, such that we should be able to compare artifact version and raise a helpful error message:

https://github.com/dbt-labs/dbt-core/blob/3fc715f06673f24718c761e5d0cf173766954816/core/dbt/exceptions.py#L271-L296

That error message is supposed to be raised in the `try`/`except` where we raise this error:
https://github.com/dbt-labs/dbt-core/blob/3fc715f06673f24718c761e5d0cf173766954816/core/dbt/contracts/state.py#L16-L20

Unfortunately, we raise a different error here:

```python
ipdb> self.manifest = WritableManifest.read(str(manifest_path))
*** mashumaro.exceptions.MissingField: Field "metrics" of type Mapping[str, ParsedMetric] is missing in WritableManifest instance
```

Is there any way we can trip the `IncompatibleSchemaException` instead?

---

It would definitely be preferable to raise a different error message, one that more clearly indicates the cause of the problem is a version mismatch. The job would still fail, though.

In this particular case, there are things that dbt Cloud could do here to help, such as including dbt version in the deferral candidate logic: the run must be successful _and_ using the same dbt version as the current job definition. That's even more feasible, post-v1, since dbt Cloud will consider all versions `1.0.x` to be `1.0 (latest)`.

There's a desirable future world in which such logic would be _too_ strict, because artifact schema versions aren't changing with each new minor version of `dbt-core`. That's something we need to build toward.

Resolving patch:
diff --git a/core/dbt/contracts/state.py b/core/dbt/contracts/state.py
--- a/core/dbt/contracts/state.py
+++ b/core/dbt/contracts/state.py
@@ -14,7 +14,8 @@ def __init__(self, path: Path):
         manifest_path = self.path / 'manifest.json'
         if manifest_path.exists() and manifest_path.is_file():
             try:
-                self.manifest = WritableManifest.read(str(manifest_path))
+                # we want to bail with an error if schema versions don't match
+                self.manifest = WritableManifest.read_and_check_versions(str(manifest_path))
             except IncompatibleSchemaException as exc:
                 exc.add_filename(str(manifest_path))
                 raise
@@ -22,7 +23,8 @@ def __init__(self, path: Path):
         results_path = self.path / 'run_results.json'
         if results_path.exists() and results_path.is_file():
             try:
-                self.results = RunResultsArtifact.read(str(results_path))
+                # we want to bail with an error if schema versions don't match
+                self.results = RunResultsArtifact.read_and_check_versions(str(results_path))
             except IncompatibleSchemaException as exc:
                 exc.add_filename(str(results_path))
                 raise
diff --git a/core/dbt/contracts/util.py b/core/dbt/contracts/util.py
--- a/core/dbt/contracts/util.py
+++ b/core/dbt/contracts/util.py
@@ -9,6 +9,7 @@
 from dbt.exceptions import (
     InternalException,
     RuntimeException,
+    IncompatibleSchemaException
 )
 from dbt.version import __version__
 from dbt.events.functions import get_invocation_id
@@ -158,6 +159,8 @@ def get_metadata_env() -> Dict[str, str]:
     }
 
 
+# This is used in the ManifestMetadata, RunResultsMetadata, RunOperationResultMetadata,
+# FreshnessMetadata, and CatalogMetadata classes
 @dataclasses.dataclass
 class BaseArtifactMetadata(dbtClassMixin):
     dbt_schema_version: str
@@ -177,6 +180,17 @@ def __post_serialize__(self, dct):
         return dct
 
 
+# This is used as a class decorator to set the schema_version in the
+# 'dbt_schema_version' class attribute. (It's copied into the metadata objects.)
+# Name attributes of SchemaVersion in classes with the 'schema_version' decorator:
+#   manifest
+#   run-results
+#   run-operation-result
+#   sources
+#   catalog
+#   remote-compile-result
+#   remote-execution-result
+#   remote-run-result
 def schema_version(name: str, version: int):
     def inner(cls: Type[VersionedSchema]):
         cls.dbt_schema_version = SchemaVersion(
@@ -187,6 +201,7 @@ def inner(cls: Type[VersionedSchema]):
     return inner
 
 
+# This is used in the ArtifactMixin and RemoteResult classes
 @dataclasses.dataclass
 class VersionedSchema(dbtClassMixin):
     dbt_schema_version: ClassVar[SchemaVersion]
@@ -198,6 +213,30 @@ def json_schema(cls, embeddable: bool = False) -> Dict[str, Any]:
             result['$id'] = str(cls.dbt_schema_version)
         return result
 
+    @classmethod
+    def read_and_check_versions(cls, path: str):
+        try:
+            data = read_json(path)
+        except (EnvironmentError, ValueError) as exc:
+            raise RuntimeException(
+                f'Could not read {cls.__name__} at "{path}" as JSON: {exc}'
+            ) from exc
+
+        # Check metadata version. There is a class variable 'dbt_schema_version', but
+        # that doesn't show up in artifacts, where it only exists in the 'metadata'
+        # dictionary.
+        if hasattr(cls, 'dbt_schema_version'):
+            if 'metadata' in data and 'dbt_schema_version' in data['metadata']:
+                previous_schema_version = data['metadata']['dbt_schema_version']
+                # cls.dbt_schema_version is a SchemaVersion object
+                if str(cls.dbt_schema_version) != previous_schema_version:
+                    raise IncompatibleSchemaException(
+                        expected=str(cls.dbt_schema_version),
+                        found=previous_schema_version
+                    )
+
+        return cls.from_dict(data)  # type: ignore
+
 
 T = TypeVar('T', bound='ArtifactMixin')
 
@@ -205,6 +244,8 @@ def json_schema(cls, embeddable: bool = False) -> Dict[str, Any]:
 # metadata should really be a Generic[T_M] where T_M is a TypeVar bound to
 # BaseArtifactMetadata. Unfortunately this isn't possible due to a mypy issue:
 # https://github.com/python/mypy/issues/7520
+# This is used in the WritableManifest, RunResultsArtifact, RunOperationResultsArtifact,
+# and CatalogArtifact
 @dataclasses.dataclass(init=False)
 class ArtifactMixin(VersionedSchema, Writable, Readable):
     metadata: BaseArtifactMetadata


Test patch:
diff --git a/test/integration/062_defer_state_test/previous_state/manifest.json b/test/integration/062_defer_state_test/previous_state/manifest.json
new file mode 100644
--- /dev/null
+++ b/test/integration/062_defer_state_test/previous_state/manifest.json
@@ -0,0 +1,6 @@
+{
+    "metadata": {
+        "dbt_schema_version": "https://schemas.getdbt.com/dbt/manifest/v3.json",
+        "dbt_version": "0.21.1"
+    }
+}
diff --git a/test/integration/062_defer_state_test/test_modified_state.py b/test/integration/062_defer_state_test/test_modified_state.py
--- a/test/integration/062_defer_state_test/test_modified_state.py
+++ b/test/integration/062_defer_state_test/test_modified_state.py
@@ -6,7 +6,7 @@
 
 import pytest
 
-from dbt.exceptions import CompilationException
+from dbt.exceptions import CompilationException, IncompatibleSchemaException
 
 
 class TestModifiedState(DBTIntegrationTest):
@@ -36,7 +36,7 @@ def _symlink_test_folders(self):
         for entry in os.listdir(self.test_original_source_path):
             src = os.path.join(self.test_original_source_path, entry)
             tst = os.path.join(self.test_root_dir, entry)
-            if entry in {'models', 'seeds', 'macros'}:
+            if entry in {'models', 'seeds', 'macros', 'previous_state'}:
                 shutil.copytree(src, tst)
             elif os.path.isdir(entry) or entry.endswith('.sql'):
                 os.symlink(src, tst)
@@ -202,3 +202,10 @@ def test_postgres_changed_exposure(self):
         results, stdout = self.run_dbt_and_capture(['run', '--models', '+state:modified', '--state', './state'])
         assert len(results) == 1
         assert results[0].node.name == 'view_model'
+
+    @use_profile('postgres')
+    def test_postgres_previous_version_manifest(self):
+        # This tests that a different schema version in the file throws an error
+        with self.assertRaises(IncompatibleSchemaException) as exc:
+            results = self.run_dbt(['ls', '-s',  'state:modified',  '--state',  './previous_state'])
+            self.assertEqual(exc.CODE, 10014)