Repository: dbt-labs/dbt-core

Issue:
Performance issues with a nontrivial number of sources
### Describe the bug
In dbt v0.17.0-rc1, it appears that performance degrades greatly when a nontrivial number of sources are added to a project. This is a regression: I could not reproduce this performance failure mode in dbt v0.16.x.

I tested this out by repeatedly running `dbt ls` and recording runtimes. The data looks like:

|sources|runtime (s)|time per node (s)|
|-------|-----------|-----------------|
|25     |4.049509287|0.1619803715     |
|50     |7.128911018|0.1425782204     |
|75     |12.29483604|0.1639311473     |
|100    |18.71185374|0.1871185374     |
|125    |29.37486005|0.2349988804     |
|150    |38.06821609|0.2537881072     |
|175    |50.92429304|0.2909959602     |
|200    |67.39295316|0.3369647658     |
|225    |79.97533703|0.3554459423     |


<img width="604" alt="Screen Shot 2020-05-20 at 9 37 11 AM" src="https://user-images.githubusercontent.com/1541139/82452626-81f30b80-9a7d-11ea-874c-0ea848ebd386.png">

This data indicates that a `dbt ls` command with 225 sources takes 1m20s to run. A corresponding `dbt ls` on 0.16.1 runs in 4s! 

There may be some not-ideal algorithmic complexity issues to look into here. Additionally, the fixed cost for parsing a single source is _super_ high. Most of this latency appears to come from the serialization and deserialization of data in the source patching part of the codebase.

The `patch_source` method accounts for the majority of the runtime of this `dbt ls` command, but notably, there are no sources to patch in my example project!

<img width="1047" alt="Screen Shot 2020-05-20 at 9 43 41 AM" src="https://user-images.githubusercontent.com/1541139/82453430-82d86d00-9a7e-11ea-9484-ac4126b716d0.png">

The relevant part of the codebase is around here:

https://github.com/fishtown-analytics/dbt/blob/75dbb0bc19376b2905d5bbb66284b9be3bf3c93c/core/dbt/parser/sources.py#L44-L68


### Possible resolutions
- Can we skip the source patching code if the source is not patched?
- The slowest parts of this execution are around serialization and deserialization in hologram (I think). Is there an easy way to make this serialization/deserialization significantly faster?

**The output of `dbt --version`:**
```
dbt v0.17.0-rc1
```

**The operating system you're using:** macOS

**The output of `python --version`:**  3.7.7


Discussion:
I assure you, there's no way dbt 0.17.0rc1 is running with python 2.7.7. 😄
Maybe we should have people run `dbt debug` instead, so we can capture homebrew/virtualenv installs?
oops - i meant 3.7.7 - that was a typo

Resolving patch:
diff --git a/core/dbt/contracts/graph/model_config.py b/core/dbt/contracts/graph/model_config.py
--- a/core/dbt/contracts/graph/model_config.py
+++ b/core/dbt/contracts/graph/model_config.py
@@ -267,8 +267,9 @@ def update_from(
         return self.from_dict(dct, validate=validate)
 
     def finalize_and_validate(self: T) -> T:
-        self.to_dict(validate=True)
-        return self.replace()
+        # from_dict will validate for us
+        dct = self.to_dict(omit_none=False, validate=False)
+        return self.from_dict(dct)
 
 
 @dataclass
diff --git a/core/dbt/parser/sources.py b/core/dbt/parser/sources.py
--- a/core/dbt/parser/sources.py
+++ b/core/dbt/parser/sources.py
@@ -46,6 +46,8 @@ def patch_source(
         unpatched: UnpatchedSourceDefinition,
         patch: Optional[SourcePatch],
     ) -> UnpatchedSourceDefinition:
+        if patch is None:
+            return unpatched
 
         source_dct = unpatched.source.to_dict()
         table_dct = unpatched.table.to_dict()


Test patch: