Repository: dbt-labs/dbt-core

Issue:
Use all adapter names for adapter.dispatch
### Describe the feature

The `dbt-redshift` adapter plugin inherits from the `dbt-postgres` adapter plugin. This reflects the fact that they share a lot of code: connection mechanism (`psycopg2`), basic syntax, types, materializations, etc. (This can be obfuscated by the fact that their shared implementations of macros are also often the `default__` dbt implementations.)

Today, when dbt dispatches `package_name.macro_name` macro on Redshift, it looks for:

1. `my_project.redshift__macro_name`
2. `my_project.redshift__macro_name`
3. `package_name.postgres__concat`
4. `package_name.postgres__concat`

I think it should instead look for:

1. `my_project.redshift__macro_name`
2. `my_project.postgres__macro_name`
3. `my_project.default__macro_name`
4. `package_name.redshift__macro_name`
5. `package_name.postgres__macro_name`
6. `package_name.default__macro_name`

This is a possibility we weighed in #2679, to the point that we make it possible as a one-line change:
https://github.com/fishtown-analytics/dbt/blob/5ba5271da99bc1ef7fbad2f7d0b45634087300fa/core/dbt/context/providers.py#L104-L109

### What's the risk?

There are cases today where Redshift uses the `default__` implementation and Postgres uses a custom `postgres__` implementation, e.g. [`dbt_utils.dateadd`](https://github.com/fishtown-analytics/dbt-utils/blob/master/macros/cross_db_utils/dateadd.sql).

I don't think this is a breaking change per se, because Redshift largely supports PostgreSQL syntax, but we may want to add implementations that explicitly preserve existing behavior:

```sql
{% macro redshift__dateadd(datepart, interval, from_date_or_timestamp) %}
    {% do return(default__dateadd(datepart, interval, from_date_or_timestamp)) %}
{% endmacro %}
```

### Who will this benefit?
This will enable maintainers of similar-yet-slightly-different plugins, e.g. `dbt-sqlserver` and `dbt-synapse`, who can then "share" macro implementations between parent and child adapters. See https://github.com/dbt-msft/dbt-synapse/issues/18.

It's possible to imagine many more adapter inheritances in the future:
- a dedicated `dbt-databricks` or `dbt-delta` that inherits from `dbt-spark`
- a `dbt-hive` that inherits from `dbt-spark` or `dbt-presto`
- a `dbt-materialize` that inherits from `dbt-postgres`


Discussion:
friendly, curious bump. is this still something in consideration for Margaret Mead. Relatedly, I just peeped [MaterializeInc/materialize-dbt-utils](https://github.com/MaterializeInc/materialize-dbt-utils). Pretty cool to see folks catching dispatch fever!
Yes, this is still something I'd like to do for Margaret Mead! The change itself is _very_ straightforward. We'll want to add some tests for it, and it will likely require a few changes in packages, especially dbt-utils.

Resolving patch:
diff --git a/core/dbt/context/providers.py b/core/dbt/context/providers.py
--- a/core/dbt/context/providers.py
+++ b/core/dbt/context/providers.py
@@ -8,7 +8,9 @@
 
 from dbt import deprecations
 from dbt.adapters.base.column import Column
-from dbt.adapters.factory import get_adapter, get_adapter_package_names
+from dbt.adapters.factory import (
+    get_adapter, get_adapter_package_names, get_adapter_type_names
+)
 from dbt.clients import agate_helper
 from dbt.clients.jinja import get_rendered, MacroGenerator, MacroStack
 from dbt.config import RuntimeConfig, Project
@@ -107,10 +109,11 @@ def commit(self):
         return self._adapter.commit_if_has_connection()
 
     def _get_adapter_macro_prefixes(self) -> List[str]:
-        # a future version of this could have plugins automatically call fall
-        # back to their dependencies' dependencies by using
-        # `get_adapter_type_names` instead of `[self.config.credentials.type]`
-        search_prefixes = [self._adapter.type(), 'default']
+        # order matters for dispatch:
+        #  1. current adapter
+        #  2. any parent adapters (dependencies)
+        #  3. 'default'
+        search_prefixes = get_adapter_type_names(self._adapter.type()) + ['default']
         return search_prefixes
 
     def dispatch(


Test patch:
diff --git a/test/integration/012_deprecation_tests/test_deprecations.py b/test/integration/012_deprecation_tests/test_deprecations.py
--- a/test/integration/012_deprecation_tests/test_deprecations.py
+++ b/test/integration/012_deprecation_tests/test_deprecations.py
@@ -110,6 +110,14 @@ def test_postgres_adapter_macro_fail(self):
 
     @use_profile('redshift')
     def test_redshift_adapter_macro(self):
+        self.assertEqual(deprecations.active_deprecations, set())
+        # pick up the postgres macro
+        self.run_dbt(strict=False)
+        expected = {'adapter-macro'}
+        self.assertEqual(expected, deprecations.active_deprecations)
+        
+    @use_profile('bigquery')
+    def test_bigquery_adapter_macro(self):
         self.assertEqual(deprecations.active_deprecations, set())
         # picked up the default -> error
         with self.assertRaises(dbt.exceptions.CompilationException) as exc:
@@ -147,6 +155,15 @@ def test_postgres_adapter_macro_pkg_fail(self):
 
     @use_profile('redshift')
     def test_redshift_adapter_macro_pkg(self):
+        self.assertEqual(deprecations.active_deprecations, set())
+        # pick up the postgres macro
+        self.assertEqual(deprecations.active_deprecations, set())
+        self.run_dbt(strict=False)
+        expected = {'adapter-macro'}
+        self.assertEqual(expected, deprecations.active_deprecations)
+
+    @use_profile('bigquery')
+    def test_bigquery_adapter_macro_pkg(self):
         self.assertEqual(deprecations.active_deprecations, set())
         # picked up the default -> error
         with self.assertRaises(dbt.exceptions.CompilationException) as exc:
diff --git a/test/integration/016_macro_tests/dispatch-inheritance-models/model.sql b/test/integration/016_macro_tests/dispatch-inheritance-models/model.sql
new file mode 100644
--- /dev/null
+++ b/test/integration/016_macro_tests/dispatch-inheritance-models/model.sql
@@ -0,0 +1,2 @@
+{{ dispatch_to_parent() }}
+select 1 as id
diff --git a/test/integration/016_macro_tests/macros/my_macros.sql b/test/integration/016_macro_tests/macros/my_macros.sql
--- a/test/integration/016_macro_tests/macros/my_macros.sql
+++ b/test/integration/016_macro_tests/macros/my_macros.sql
@@ -15,8 +15,23 @@
 {% endmacro %}
 
 
-{# there is no no default__dispatch_to_nowhere! #}
+{# there is no default__dispatch_to_nowhere! #}
 {% macro dispatch_to_nowhere() %}
 	{% set macro = adapter.dispatch('dispatch_to_nowhere') %}
 	{{ macro() }}
 {% endmacro %}
+
+
+{% macro dispatch_to_parent() %}
+	{% set macro = adapter.dispatch('dispatch_to_parent') %}
+	{{ macro() }}
+{% endmacro %}
+
+{% macro default__dispatch_to_parent() %}
+	{% set msg = 'No default implementation of dispatch_to_parent' %}
+    {{ exceptions.raise_compiler_error(msg) }}
+{% endmacro %}
+
+{% macro postgres__dispatch_to_parent() %}
+	{{ return('') }}
+{% endmacro %}
diff --git a/test/integration/016_macro_tests/test_macros.py b/test/integration/016_macro_tests/test_macros.py
--- a/test/integration/016_macro_tests/test_macros.py
+++ b/test/integration/016_macro_tests/test_macros.py
@@ -95,6 +95,20 @@ def test_postgres_invalid_macro(self):
         assert "In dispatch: No macro named 'dispatch_to_nowhere' found" in str(exc.value)
 
 
+class TestDispatchMacroUseParent(DBTIntegrationTest):
+    @property
+    def schema(self):
+        return "test_macros_016"
+
+    @property
+    def models(self):
+        return "dispatch-inheritance-models"
+
+    @use_profile('redshift')
+    def test_redshift_inherited_macro(self):
+        self.run_dbt(['run'])
+
+
 class TestMacroOverrideBuiltin(DBTIntegrationTest):
     @property
     def schema(self):
diff --git a/test/integration/029_docs_generate_tests/test_docs_generate.py b/test/integration/029_docs_generate_tests/test_docs_generate.py
--- a/test/integration/029_docs_generate_tests/test_docs_generate.py
+++ b/test/integration/029_docs_generate_tests/test_docs_generate.py
@@ -204,7 +204,7 @@ def _redshift_stats(self):
             "encoded": {
                 "id": "encoded",
                 "label": "Encoded",
-                "value": "Y",
+                "value": AnyStringWith('AUTO'),
                 "description": "Indicates whether any column in the table has compression encoding defined.",
                 "include": True
             },
diff --git a/test/unit/test_context.py b/test/unit/test_context.py
--- a/test/unit/test_context.py
+++ b/test/unit/test_context.py
@@ -242,6 +242,22 @@ def assert_has_keys(
     },
 }
 
+POSTGRES_PROFILE_DATA = {
+    'target': 'test',
+    'quoting': {},
+    'outputs': {
+        'test': {
+            'type': 'postgres',
+            'host': 'localhost',
+            'schema': 'analytics',
+            'user': 'test',
+            'pass': 'test',
+            'dbname': 'test',
+            'port': 1,
+        }
+    },
+}
+
 PROJECT_DATA = {
     'name': 'root',
     'version': '0.1',
@@ -368,6 +384,9 @@ def get_include_paths():
 def config():
     return config_from_parts_or_dicts(PROJECT_DATA, PROFILE_DATA)
 
+@pytest.fixture
+def config_postgres():
+    return config_from_parts_or_dicts(PROJECT_DATA, POSTGRES_PROFILE_DATA)
 
 @pytest.fixture
 def manifest_fx(config):
@@ -399,8 +418,8 @@ def redshift_adapter(config, get_adapter):
 
 
 @pytest.fixture
-def postgres_adapter(config, get_adapter):
-    adapter = postgres.PostgresAdapter(config)
+def postgres_adapter(config_postgres, get_adapter):
+    adapter = postgres.PostgresAdapter(config_postgres)
     inject_adapter(adapter, postgres.Plugin)
     get_adapter.return_value = adapter
     yield adapter
@@ -521,13 +540,13 @@ def test_resolve_specific(config, manifest_extended, redshift_adapter, get_inclu
                                    'dbt', 'root']).macro is rs_macro
 
 
-def test_resolve_default(config, manifest_extended, postgres_adapter, get_include_paths):
+def test_resolve_default(config_postgres, manifest_extended, postgres_adapter, get_include_paths):
     dbt_macro = manifest_extended.macros['macro.dbt.default__some_macro']
     package_macro = manifest_extended.macros['macro.root.default__some_macro']
 
     ctx = providers.generate_runtime_model(
         model=mock_model(),
-        config=config,
+        config=config_postgres,
         manifest=manifest_extended,
     )