Repository: dbt-labs/dbt-core

Issue:
Large projects see significant slowdown in resolve_graph using v0.21.0 
### Describe the bug
Since upgrading to 0.21.0 I have seen a significant slowdown in the speed with which DBT runs this includes both `dbt compile` and `dbt run`. I am running on a redshift data warehouse, though I don't know if the issue is redshift specific.

### Steps To Reproduce
Simply run `dbt run` or `dbt compile` with partial parse enabled or disabled the issue occurs either way.

### Expected behavior
I saw a 3-5x slowdown so i expected it to run significantly faster

### Screenshots and log output
![image](https://user-images.githubusercontent.com/52387295/136232247-5bb6da9a-c4a1-4e84-96ba-ffa145dd1861.png)

![image](https://user-images.githubusercontent.com/52387295/136232382-995cb4d3-7b00-4113-8a2c-32efb1dd636a.png)


### System information
**Which database are you using dbt with?**
- [ ] postgres
- [√] redshift
- [ ] bigquery
- [ ] snowflake
- [ ] other (specify: ____________)


**The output of `dbt --version`:**
```
installed version: 0.21.0
   latest version: 0.21.0

Up to date!

Plugins:
  - bigquery: 0.21.0
  - snowflake: 0.21.0
  - redshift: 0.21.0
  - postgres: 0.21.0
```

**The operating system you're using:**
Mac OS Big Sur 11.3.1
**The output of `python --version`:**
Python 3.7.5

### Additional context
Add any other context about the problem here.



Discussion:
@isaacsantelli Weird! Thanks for the bug report.

It looks like, once dbt starts running nodes (`Concurrency:` stdout log line), everything is proceeding quite quickly. So it's got to be something with:
- project parsing (though I see you've got partial parsing enabled)
- metadata queries to build the adapter cache

Could you check the debug-level logs (`logs/dbt.log`), and see where the greatest gaps in timestamps are? Based on the messages that dbt is logging along the way, we should be able to figure out which steps have caused the significant slowdown.
Attached is the log from today : [dbt.log](https://github.com/dbt-labs/dbt/files/7295233/dbt.log)

@isaac-taylor Thanks for the quick response. It looks like all metadata queries are running quite quickly. The weird thing I'm noticing is significant slowdown associated with firing some anonymous usage events:
```
2021-10-06 15:00:22.947280 (MainThread): Running with dbt=0.21.0
2021-10-06 15:00:23.204965 (MainThread): running dbt with arguments Namespace(cls=<class 'dbt.task.compile.CompileTask'>, debug=False, defer=None, exclude=None, full_refresh=False, log_cache_events=False, log_format='default', parse_only=False, partial_parse=None, profile=None, profiles_dir='/Users/isaacsantelli/.dbt', project_dir=None, record_timing_info=None, rpc_method='compile', select=None, selector_name=None, single_threaded=False, state=None, strict=False, target=None, test_new_parser=False, threads=None, use_cache=True, use_colors=None, use_experimental_parser=False, vars='{}', version_check=True, warn_error=False, which='compile', write_json=True)
2021-10-06 15:00:23.205623 (MainThread): Tracking: tracking
2021-10-06 15:00:23.224004 (MainThread): Sending event: {'category': 'dbt', 'action': 'invocation', 'label': 'start', 'context': [<snowplow_tracker.self_describing_json.SelfDescribingJson object at 0x106899150>, <snowplow_tracker.self_describing_json.SelfDescribingJson object at 0x106f65490>, <snowplow_tracker.self_describing_json.SelfDescribingJson object at 0x106f653d0>]}
--- NO DELAY HERE ---
2021-10-06 15:00:24.006624 (MainThread): Partial parsing enabled: 0 files deleted, 0 files added, 0 files changed.
2021-10-06 15:00:24.007105 (MainThread): Partial parsing enabled, no changes found, skipping parsing
2021-10-06 15:00:24.173568 (MainThread): Sending event: {'category': 'dbt', 'action': 'load_project', 'label': '9d8ddbe5-286c-4be5-898e-4b23a11ac320', 'context': [<snowplow_tracker.self_describing_json.SelfDescribingJson object at 0x10835ed10>]}
--- 60 SECOND DELAY ---
2021-10-06 15:01:35.086751 (MainThread): Sending event: {'category': 'dbt', 'action': 'resource_counts', 'label': '9d8ddbe5-286c-4be5-898e-4b23a11ac320', 'context': [<snowplow_tracker.self_describing_json.SelfDescribingJson object at 0x1086192d0>]}
2021-10-06 15:01:35.087205 (MainThread): Found 313 models, 2798 tests, 5 snapshots, 0 analyses, 821 macros, 1 operation, 33 seed files, 236 sources, 0 exposures
--- 40 SECOND DELAY ---
2021-10-06 15:02:13.265683 (MainThread): 
2021-10-06 15:02:13.267495 (MainThread): Acquiring new redshift connection "master".
```
I don't know the details of your development/deployment environment, and whether those requests might be passing through some sort of proxy, or if the issue is with event collection on our end. Could you try [disabling anonymous usage tracking](https://docs.getdbt.com/reference/profiles.yml#send_anonymous_usage_stats), and see if that speeds up your invocations?
I'm experiencing the same issue and disabling user tracking doesn't help.

Here's the log entry before disabling usage tracking:
```
2021-10-06 16:38:21.524362 (MainThread): Running with dbt=0.21.0
2021-10-06 16:38:21.828803 (MainThread): running dbt with arguments Namespace(cls=<class 'dbt.task.compile.CompileTask'>, debug=False, defer=None, exclude=None, full_refresh=False, log_cache_events=False, log_format='default', parse_only=False, partial_parse=True, profile=None, profiles_dir='/Users/ilmari.aalto/.dbt', project_dir=None, record_timing_info=None, rpc_method='compile', select=None, selector_name=None, single_threaded=False, state=None, strict=False, target=None, test_new_parser=False, threads=None, use_cache=True, use_colors=None, use_experimental_parser=False, vars='{}', version_check=True, warn_error=False, which='compile', write_json=True)
2021-10-06 16:38:21.829559 (MainThread): Tracking: tracking
2021-10-06 16:38:21.841164 (MainThread): Sending event: {'category': 'dbt', 'action': 'invocation', 'label': 'start', 'context': [<snowplow_tracker.self_describing_json.SelfDescribingJson object at 0x10ae64a60>, <snowplow_tracker.self_describing_json.SelfDescribingJson object at 0x10ae86610>, <snowplow_tracker.self_describing_json.SelfDescribingJson object at 0x10ae90130>]}
2021-10-06 16:38:22.400905 (MainThread): Partial parsing enabled: 0 files deleted, 0 files added, 0 files changed.
2021-10-06 16:38:22.401256 (MainThread): Partial parsing enabled, no changes found, skipping parsing
2021-10-06 16:38:22.588468 (MainThread): Sending event: {'category': 'dbt', 'action': 'load_project', 'label': '5043baeb-8ae1-40c4-aabb-1a273ced6e46', 'context': [<snowplow_tracker.self_describing_json.SelfDescribingJson object at 0x10cfef100>]}
--- 40 SECOND DELAY ---
2021-10-06 16:39:01.165051 (MainThread): write_gpickle is deprecated and will be removed in 3.0.Use ``pickle.dump(G, path, protocol)``
2021-10-06 16:39:01.225095 (MainThread): Sending event: {'category': 'dbt', 'action': 'resource_counts', 'label': '5043baeb-8ae1-40c4-aabb-1a273ced6e46', 'context': [<snowplow_tracker.self_describing_json.SelfDescribingJson object at 0x10cfefc70>]}
2021-10-06 16:39:01.225543 (MainThread): Found 335 models, 2713 tests, 3 snapshots, 5 analyses, 397 macros, 3 operations, 10 seed files, 135 sources, 0 exposures
--- 20 SECOND DELAY ---
2021-10-06 16:39:19.281222 (MainThread): 
2021-10-06 16:39:19.282803 (MainThread): Acquiring new redshift connection "master".
```

Here's after:
```
2021-10-06 16:45:46.255823 (MainThread): Running with dbt=0.21.0
2021-10-06 16:45:46.872313 (MainThread): running dbt with arguments Namespace(cls=<class 'dbt.task.compile.CompileTask'>, debug=False, defer=None, exclude=None, full_refresh=False, log_cache_events=False, log_format='default', parse_only=False, partial_parse=True, profile=None, profiles_dir='/Users/ilmari.aalto/.dbt', project_dir=None, record_timing_info=None, rpc_method='compile', select=None, selector_name=None, single_threaded=False, state=None, strict=False, target=None, test_new_parser=False, threads=None, use_cache=True, use_colors=None, use_experimental_parser=False, vars='{}', version_check=True, warn_error=False, which='compile', write_json=True)
2021-10-06 16:45:46.872989 (MainThread): Tracking: do not track
2021-10-06 16:45:47.574988 (MainThread): Partial parsing enabled: 0 files deleted, 0 files added, 0 files changed.
2021-10-06 16:45:47.575376 (MainThread): Partial parsing enabled, no changes found, skipping parsing
--- 40 SECOND DELAY ---
2021-10-06 16:46:25.705940 (MainThread): write_gpickle is deprecated and will be removed in 3.0.Use ``pickle.dump(G, path, protocol)``
2021-10-06 16:46:25.765966 (MainThread): Found 335 models, 2713 tests, 3 snapshots, 5 analyses, 397 macros, 3 operations, 10 seed files, 135 sources, 0 exposures
--- 20 SECOND DELAY ---
2021-10-06 16:46:43.499142 (MainThread): 
2021-10-06 16:46:43.500753 (MainThread): Acquiring new redshift connection "master".
```

The output of `dbt --version`:
```
installed version: 0.21.0
   latest version: 0.21.0

Up to date!

Plugins:
  - bigquery: 0.21.0
  - snowflake: 0.21.0
  - redshift: 0.21.0
  - postgres: 0.21.0
```

The operating system you're using:
Mac OS Big Sur 11.4

The output of `python3 --version`:
Python 3.8.12

I set `send_anonymous_usage_stats` to `False` and the issue is persisting attached is an updated copy of my dbt log
[dbt.log](https://github.com/dbt-labs/dbt/files/7295952/dbt.log)

Logs show a gap of about 2 minutes with nothing in between 
```
2021-10-06 17:10:36.620771 (MainThread): Tracking: do not track
2021-10-06 17:10:37.441933 (MainThread): Partial parsing enabled: 0 files deleted, 0 files added, 0 files changed.
2021-10-06 17:10:37.442459 (MainThread): Partial parsing enabled, no changes found, skipping parsing
2021-10-06 17:12:06.817106 (MainThread): Found 313 models, 2798 tests, 5 snapshots, 0 analyses, 821 macros, 1 operation, 33 seed files, 236 sources, 0 exposures
```
Also if it would be helpful info we have a DC2 redshift cluster
@ilmari-aalto @isaacsantelli Thanks for that additional info. I've been able to reproduce the issue by compiling a project with a _lot_ of tests:
```
2021-10-07 10:57:02.979374 (MainThread): Running with dbt=0.21.0
2021-10-07 10:57:03.073803 (MainThread): running dbt with arguments Namespace(cls=<class 'dbt.task.compile.CompileTask'>, debug=True, defer=None, exclude=None, full_refresh=False, log_cache_events=False, log_format='default', parse_only=False, partial_parse=None, profile='garage-postgres', profiles_dir='/Users/jerco/.dbt', project_dir=None, record_timing_info=None, rpc_method='compile', select=None, selector_name=None, single_threaded=False, state=None, strict=False, target=None, test_new_parser=False, threads=None, use_cache=True, use_colors=None, use_experimental_parser=False, vars='{}', version_check=True, warn_error=False, which='compile', write_json=True)
2021-10-07 10:57:03.074672 (MainThread): Tracking: tracking
2021-10-07 10:57:03.098361 (MainThread): Sending event: {'category': 'dbt', 'action': 'invocation', 'label': 'start', 'context': [<snowplow_tracker.self_describing_json.SelfDescribingJson object at 0x110b2b670>, <snowplow_tracker.self_describing_json.SelfDescribingJson object at 0x111051550>, <snowplow_tracker.self_describing_json.SelfDescribingJson object at 0x111051760>]}
2021-10-07 10:57:04.412107 (MainThread): Partial parsing enabled: 0 files deleted, 0 files added, 0 files changed.
2021-10-07 10:57:04.412464 (MainThread): Partial parsing enabled, no changes found, skipping parsing
2021-10-07 10:57:04.699027 (MainThread): Sending event: {'category': 'dbt', 'action': 'load_project', 'label': 'ef646720-f445-4177-afd1-9e835e0e389e', 'context': [<snowplow_tracker.self_describing_json.SelfDescribingJson object at 0x1102d7dc0>]}
2021-10-07 10:58:59.490064 (MainThread): write_gpickle is deprecated and will be removed in 3.0.Use ``pickle.dump(G, path, protocol)``
2021-10-07 10:58:59.579928 (MainThread): Sending event: {'category': 'dbt', 'action': 'resource_counts', 'label': 'ef646720-f445-4177-afd1-9e835e0e389e', 'context': [<snowplow_tracker.self_describing_json.SelfDescribingJson object at 0x11100b280>]}
2021-10-07 10:58:59.580330 (MainThread): Found 2000 models, 6000 tests, 0 snapshots, 0 analyses, 162 macros, 0 operations, 0 seed files, 0 sources, 0 exposures
2021-10-07 10:59:06.360928 (MainThread): 
2021-10-07 10:59:06.366090 (MainThread): Acquiring new postgres connection "master".
```

Based on the logging, I believe the primary delay is cropping up after parsing (manifest construction), during [graph construction](https://github.com/dbt-labs/dbt/blob/b501f4317c8180ab259f456ab254b34f74d0d288/core/dbt/task/runnable.py#L84). (I'm not sure about the second delay, between `Found ...` and the first metadata query.)

@leahwicz I'm pretty sure this is related to the new dependency linking, related to the test blocking feature for `dbt build`. It looks like we acknowledged that performance might be an issue several weeks ago: https://github.com/dbt-labs/dbt/pull/3792#issuecomment-918385504.

To confirm this, I added timestamped lines to the beginning + end of [`resolve_graph`](https://github.com/dbt-labs/dbt/blob/b501f4317c8180ab259f456ab254b34f74d0d288/core/dbt/compilation.py#L435):

```
Running with dbt=0.21.0
Partial parsing enabled: 0 files deleted, 0 files added, 0 files changed.
Partial parsing enabled, no changes found, skipping parsing
13:05:58 | Start graph resolution
13:07:48 | End graph resolution
Found 2000 models, 6000 tests, 0 snapshots, 0 analyses, 162 macros, 0 operations, 0 seed files, 0 sources, 0 exposures
```

This issue isn't unique to Redshift, and it's especially painful because partial parsing can't help here—users are going to feel this pain during every invocation of a runnable task.

I think our options are:
- (long-term ideal) Refactor the approach to graph linking with performance front of mind
- (workable, temporary) Turn off the new dependency linking for non-`build` tasks, to undo the regression in performance
Thanks for looking at it quickly @jtcohen6 ! I wonder what's behind the second delay, but perhaps it always was there I just didn't pay attention to it 😛 . To be clear, the slowness doesn't affect our Production runs as the delay there is not relevant, but slows down interactive, local development.

I always thought our project was small to mid-size, but now I'm seeing it can be considered large 😄 
Our project is fair sized: 
```Found 218 models, 1409 tests, 0 snapshots, 0 analyses, 405 macros, 0 operations, 5 seed files, 79 sources, 0 exposures```

0.21.0 is very slow to start after that line, even with partial parsing as well. Snowflake here.

Resolving patch:
diff --git a/core/dbt/exceptions.py b/core/dbt/exceptions.py
--- a/core/dbt/exceptions.py
+++ b/core/dbt/exceptions.py
@@ -838,31 +838,47 @@ def raise_duplicate_macro_name(node_1, node_2, namespace) -> NoReturn:
 
 def raise_duplicate_resource_name(node_1, node_2):
     duped_name = node_1.name
+    node_type = NodeType(node_1.resource_type)
+    pluralized = (
+        node_type.pluralize()
+        if node_1.resource_type == node_2.resource_type
+        else "resources"  # still raise if ref() collision, e.g. model + seed
+    )
 
-    if node_1.resource_type in NodeType.refable():
-        get_func = 'ref("{}")'.format(duped_name)
-    elif node_1.resource_type == NodeType.Source:
+    action = "looking for"
+    # duplicate 'ref' targets
+    if node_type in NodeType.refable():
+        formatted_name = f'ref("{duped_name}")'
+    # duplicate sources
+    elif node_type == NodeType.Source:
         duped_name = node_1.get_full_source_name()
-        get_func = node_1.get_source_representation()
-    elif node_1.resource_type == NodeType.Documentation:
-        get_func = 'doc("{}")'.format(duped_name)
-    elif node_1.resource_type == NodeType.Test and "schema" in node_1.tags:
-        return
+        formatted_name = node_1.get_source_representation()
+    # duplicate docs blocks
+    elif node_type == NodeType.Documentation:
+        formatted_name = f'doc("{duped_name}")'
+    # duplicate generic tests
+    elif node_type == NodeType.Test and hasattr(node_1, "test_metadata"):
+        column_name = f'column "{node_1.column_name}" in ' if node_1.column_name else ""
+        model_name = node_1.file_key_name
+        duped_name = f'{node_1.name}" defined on {column_name}"{model_name}'
+        action = "running"
+        formatted_name = "tests"
+    # all other resource types
     else:
-        get_func = '"{}"'.format(duped_name)
+        formatted_name = duped_name
 
+    # should this be raise_parsing_error instead?
     raise_compiler_error(
-        'dbt found two resources with the name "{}". Since these resources '
-        "have the same name,\ndbt will be unable to find the correct resource "
-        "when {} is used. To fix this,\nchange the name of one of "
-        "these resources:\n- {} ({})\n- {} ({})".format(
-            duped_name,
-            get_func,
-            node_1.unique_id,
-            node_1.original_file_path,
-            node_2.unique_id,
-            node_2.original_file_path,
-        )
+        f"""
+dbt found two {pluralized} with the name "{duped_name}".
+
+Since these resources have the same name, dbt will be unable to find the correct resource
+when {action} {formatted_name}.
+
+To fix this, change the name of one of these resources:
+- {node_1.unique_id} ({node_1.original_file_path})
+- {node_2.unique_id} ({node_2.original_file_path})
+    """.strip()
     )
 
 


Test patch:
diff --git a/core/dbt/parser/generic_test_builders.py b/core/dbt/parser/generic_test_builders.py
--- a/core/dbt/parser/generic_test_builders.py
+++ b/core/dbt/parser/generic_test_builders.py
@@ -25,9 +25,13 @@
 from dbt.parser.search import FileBlock
 
 
-def get_nice_generic_test_name(
+def synthesize_generic_test_names(
     test_type: str, test_name: str, args: Dict[str, Any]
 ) -> Tuple[str, str]:
+    # Using the type, name, and arguments to this generic test, synthesize a (hopefully) unique name
+    # Will not be unique if multiple tests have same name + arguments, and only configs differ
+    # Returns a shorter version (hashed/truncated, for the compiled file)
+    # as well as the full name (for the unique_id + FQN)
     flat_args = []
     for arg_name in sorted(args):
         # the model is already embedded in the name, so skip it
@@ -263,13 +267,25 @@ def __init__(
         if self.namespace is not None:
             self.package_name = self.namespace
 
-        compiled_name, fqn_name = self.get_test_name()
-        self.compiled_name: str = compiled_name
-        self.fqn_name: str = fqn_name
-
-        # use hashed name as alias if too long
-        if compiled_name != fqn_name and "alias" not in self.config:
-            self.config["alias"] = compiled_name
+        # If the user has provided a custom name for this generic test, use it
+        # Then delete the "name" argument to avoid passing it into the test macro
+        # Otherwise, use an auto-generated name synthesized from test inputs
+        self.compiled_name: str = ""
+        self.fqn_name: str = ""
+
+        if "name" in self.args:
+            # Assign the user-defined name here, which will be checked for uniqueness later
+            # we will raise an error if two tests have same name for same model + column combo
+            self.compiled_name = self.args["name"]
+            self.fqn_name = self.args["name"]
+            del self.args["name"]
+        else:
+            short_name, full_name = self.get_synthetic_test_names()
+            self.compiled_name = short_name
+            self.fqn_name = full_name
+            # use hashed name as alias if full name is too long
+            if short_name != full_name and "alias" not in self.config:
+                self.config["alias"] = short_name
 
     def _bad_type(self) -> TypeError:
         return TypeError('invalid target type "{}"'.format(type(self.target)))
@@ -281,13 +297,23 @@ def extract_test_args(test, name=None) -> Tuple[str, Dict[str, Any]]:
                 "test must be dict or str, got {} (value {})".format(type(test), test)
             )
 
-        test = list(test.items())
-        if len(test) != 1:
-            raise_parsing_error(
-                "test definition dictionary must have exactly one key, got"
-                " {} instead ({} keys)".format(test, len(test))
-            )
-        test_name, test_args = test[0]
+        # If the test is a dictionary with top-level keys, the test name is "test_name"
+        # and the rest are arguments
+        # {'name': 'my_favorite_test', 'test_name': 'unique', 'config': {'where': '1=1'}}
+        if "test_name" in test.keys():
+            test_name = test.pop("test_name")
+            test_args = test
+        # If the test is a nested dictionary with one top-level key, the test name
+        # is the dict name, and nested keys are arguments
+        # {'unique': {'name': 'my_favorite_test', 'config': {'where': '1=1'}}}
+        else:
+            test = list(test.items())
+            if len(test) != 1:
+                raise_parsing_error(
+                    "test definition dictionary must have exactly one key, got"
+                    " {} instead ({} keys)".format(test, len(test))
+                )
+            test_name, test_args = test[0]
 
         if not isinstance(test_args, dict):
             raise_parsing_error(
@@ -401,7 +427,8 @@ def macro_name(self) -> str:
             macro_name = "{}.{}".format(self.namespace, macro_name)
         return macro_name
 
-    def get_test_name(self) -> Tuple[str, str]:
+    def get_synthetic_test_names(self) -> Tuple[str, str]:
+        # Returns two names: shorter (for the compiled file), full (for the unique_id + FQN)
         if isinstance(self.target, UnparsedNodeUpdate):
             name = self.name
         elif isinstance(self.target, UnpatchedSourceDefinition):
@@ -410,7 +437,7 @@ def get_test_name(self) -> Tuple[str, str]:
             raise self._bad_type()
         if self.namespace is not None:
             name = "{}_{}".format(self.namespace, name)
-        return get_nice_generic_test_name(name, self.target.name, self.args)
+        return synthesize_generic_test_names(name, self.target.name, self.args)
 
     def construct_config(self) -> str:
         configs = ",".join(
diff --git a/test/integration/025_duplicate_model_tests/test_duplicate_exposure.py b/test/integration/025_duplicate_model_tests/test_duplicate_exposure.py
--- a/test/integration/025_duplicate_model_tests/test_duplicate_exposure.py
+++ b/test/integration/025_duplicate_model_tests/test_duplicate_exposure.py
@@ -14,7 +14,7 @@ def models(self):
 
     @use_profile("postgres")
     def test_postgres_duplicate_exposure(self):
-        message = "dbt found two resources with the name"
+        message = "dbt found two exposures with the name"
         try:
             self.run_dbt(["compile"])
             self.assertTrue(False, "dbt did not throw for duplicate exposures")
diff --git a/test/integration/025_duplicate_model_tests/test_duplicate_model.py b/test/integration/025_duplicate_model_tests/test_duplicate_model.py
--- a/test/integration/025_duplicate_model_tests/test_duplicate_model.py
+++ b/test/integration/025_duplicate_model_tests/test_duplicate_model.py
@@ -14,7 +14,7 @@ def models(self):
 
     @use_profile("postgres")
     def test_postgres_duplicate_model_enabled(self):
-        message = "dbt found two resources with the name"
+        message = "dbt found two models with the name"
         try:
             self.run_dbt(["run"])
             self.assertTrue(False, "dbt did not throw for duplicate models")
@@ -80,7 +80,7 @@ def packages_config(self):
     @use_profile("postgres")
     def test_postgres_duplicate_model_enabled_across_packages(self):
         self.run_dbt(["deps"])
-        message = "dbt found two resources with the name"
+        message = "dbt found two models with the name"
         try:
             self.run_dbt(["run"])
             self.assertTrue(False, "dbt did not throw for duplicate models")
diff --git a/test/integration/025_duplicate_model_tests/test_duplicate_source.py b/test/integration/025_duplicate_model_tests/test_duplicate_source.py
--- a/test/integration/025_duplicate_model_tests/test_duplicate_source.py
+++ b/test/integration/025_duplicate_model_tests/test_duplicate_source.py
@@ -14,7 +14,7 @@ def models(self):
 
     @use_profile("postgres")
     def test_postgres_duplicate_source_enabled(self):
-        message = "dbt found two resources with the name"
+        message = "dbt found two sources with the name"
         try:
             self.run_dbt(["compile"])
             self.assertTrue(False, "dbt did not throw for duplicate sources")
diff --git a/tests/functional/schema_tests/fixtures.py b/tests/functional/schema_tests/fixtures.py
--- a/tests/functional/schema_tests/fixtures.py
+++ b/tests/functional/schema_tests/fixtures.py
@@ -420,6 +420,73 @@
 SELECT 'NOT_NULL' AS id
 """
 
+
+dupe_generic_tests_collide__schema_yml = """
+version: 2
+models:
+- name: model_a
+  columns:
+  - name: id
+    tests:
+    - not_null:
+        config:
+          where: "1=1"
+    - not_null:
+        config:
+          where: "1=2"
+
+"""
+
+dupe_generic_tests_collide__model_a = """
+SELECT 'NOT_NULL' AS id
+"""
+
+
+custom_generic_test_names__schema_yml = """
+version: 2
+models:
+- name: model_a
+  columns:
+  - name: id
+    tests:
+    - not_null:
+        name: not_null_where_1_equals_1
+        config:
+          where: "1=1"
+    - not_null:
+        name: not_null_where_1_equals_2
+        config:
+          where: "1=2"
+
+"""
+
+custom_generic_test_names__model_a = """
+SELECT 'NOT_NULL' AS id
+"""
+
+custom_generic_test_names_alt_format__schema_yml = """
+version: 2
+models:
+- name: model_a
+  columns:
+  - name: id
+    tests:
+    - name: not_null_where_1_equals_1
+      test_name: not_null
+      config:
+        where: "1=1"
+    - name: not_null_where_1_equals_2
+      test_name: not_null
+      config:
+        where: "1=2"
+
+"""
+
+custom_generic_test_names_alt_format__model_a = """
+SELECT 'NOT_NULL' AS id
+"""
+
+
 test_context_where_subq_macros__custom_generic_test_sql = """
 /*{# This test will fail if get_where_subquery() is missing from TestContext + TestMacroNamespace #}*/
 
@@ -1266,6 +1333,30 @@ def name_collision():
     }
 
 
+@pytest.fixture(scope="class")
+def dupe_tests_collide():
+    return {
+        "schema.yml": dupe_generic_tests_collide__schema_yml,
+        "model_a.sql": dupe_generic_tests_collide__model_a,
+    }
+
+
+@pytest.fixture(scope="class")
+def custom_generic_test_names():
+    return {
+        "schema.yml": custom_generic_test_names__schema_yml,
+        "model_a.sql": custom_generic_test_names__model_a,
+    }
+
+
+@pytest.fixture(scope="class")
+def custom_generic_test_names_alt_format():
+    return {
+        "schema.yml": custom_generic_test_names_alt_format__schema_yml,
+        "model_a.sql": custom_generic_test_names_alt_format__model_a,
+    }
+
+
 @pytest.fixture(scope="class")
 def test_context_where_subq_macros():
     return {"custom_generic_test.sql": test_context_where_subq_macros__custom_generic_test_sql}
diff --git a/tests/functional/schema_tests/test_schema_v2_tests.py b/tests/functional/schema_tests/test_schema_v2_tests.py
--- a/tests/functional/schema_tests/test_schema_v2_tests.py
+++ b/tests/functional/schema_tests/test_schema_v2_tests.py
@@ -16,6 +16,9 @@
     seeds,
     test_context_models,
     name_collision,
+    dupe_tests_collide,
+    custom_generic_test_names,
+    custom_generic_test_names_alt_format,
     test_context_where_subq_macros,
     invalid_schema_models,
     all_models,
@@ -25,7 +28,7 @@
     project_files,
     case_sensitive_models__uppercase_SQL,
 )
-from dbt.exceptions import ParsingException
+from dbt.exceptions import ParsingException, CompilationException
 from dbt.contracts.results import TestStatus
 
 
@@ -658,6 +661,63 @@ def test_collision_test_names_get_hash(
         assert test_results[1].node.unique_id in expected_unique_ids
 
 
+class TestGenericTestsCollide:
+    @pytest.fixture(scope="class")
+    def models(self, dupe_tests_collide):  # noqa: F811
+        return dupe_tests_collide
+
+    def test_generic_test_collision(
+        self,
+        project,
+    ):
+        """These tests collide, since only the configs differ"""
+        with pytest.raises(CompilationException) as exc:
+            run_dbt()
+        assert "dbt found two tests with the name" in str(exc)
+
+
+class TestGenericTestsCustomNames:
+    @pytest.fixture(scope="class")
+    def models(self, custom_generic_test_names):  # noqa: F811
+        return custom_generic_test_names
+
+    # users can define custom names for specific instances of generic tests
+    def test_generic_tests_with_custom_names(
+        self,
+        project,
+    ):
+        """These tests don't collide, since they have user-provided custom names"""
+        results = run_dbt()
+        test_results = run_dbt(["test"])
+
+        # model + both tests run
+        assert len(results) == 1
+        assert len(test_results) == 2
+
+        # custom names propagate to the unique_id
+        expected_unique_ids = [
+            "test.test.not_null_where_1_equals_1.7b96089006",
+            "test.test.not_null_where_1_equals_2.8ae586e17f",
+        ]
+        assert test_results[0].node.unique_id in expected_unique_ids
+        assert test_results[1].node.unique_id in expected_unique_ids
+
+
+class TestGenericTestsCustomNamesAltFormat(TestGenericTestsCustomNames):
+    @pytest.fixture(scope="class")
+    def models(self, custom_generic_test_names_alt_format):  # noqa: F811
+        return custom_generic_test_names_alt_format
+
+    # exactly as above, just alternative format for yaml definition
+    def test_collision_test_names_get_hash(
+        self,
+        project,
+    ):
+        """These tests don't collide, since they have user-provided custom names,
+        defined using an alternative format"""
+        super().test_generic_tests_with_custom_names(project)
+
+
 class TestInvalidSchema:
     @pytest.fixture(scope="class")
     def models(self, invalid_schema_models):  # noqa: F811