Repository: dbt-labs/dbt-core

Issue:
[CT-895] [Spike] Consolidate current_timestamp & associates
Split out from https://github.com/dbt-labs/dbt-utils/pull/577 + https://github.com/dbt-labs/dbt-utils/pull/597, resolves https://github.com/dbt-labs/dbt-utils/issues/339

### Describe the feature
Rewrite `current_timestamp` + `current_timestamp_utc` to be built out of core/plugin functionality (e.g. `date_function`) and a `convert_timezone` macro.

Currently in `dbt-core`, there are _three_ methods/macros all purporting to do the same thing. I've linked the examples from Postgres.
- [`date_function`](https://github.com/dbt-labs/dbt-core/blob/75f3e8cb749c7b52d226844394ecb34c1e0070a4/plugins/postgres/dbt/adapters/postgres/impl.py#L66-L68) (Python classmethod)
- [`snapshot_get_time`](https://github.com/dbt-labs/dbt-core/blob/75f3e8cb749c7b52d226844394ecb34c1e0070a4/plugins/postgres/dbt/include/postgres/macros/adapters.sql#L131-L133) (macro used for snapshots)
- [`current_timestamp`](https://github.com/dbt-labs/dbt-core/blob/75f3e8cb749c7b52d226844394ecb34c1e0070a4/plugins/postgres/dbt/include/postgres/macros/adapters.sql#L121-L123) (macro used nowhere, except as [_the default implementation_ of `snapshot_get_time`](https://github.com/dbt-labs/dbt-core/blob/75f3e8cb749c7b52d226844394ecb34c1e0070a4/core/dbt/include/global_project/macros/materializations/snapshots/strategies.sql#L50-L59) — but often the two are different, mostly for data type reasons)

Then, there are the two versions currently living in `dbt-utils`:
- [`dbt_utils.current_timestamp`](https://github.com/dbt-labs/dbt-utils/blob/44eeb1978c1c720baca2e2cfe07671e94db02c7c/macros/cross_db_utils/current_timestamp.sql#L5)
- [`dbt_utils.current_timestamp_in_utc`](https://github.com/dbt-labs/dbt-utils/blob/44eeb1978c1c720baca2e2cfe07671e94db02c7c/macros/cross_db_utils/current_timestamp.sql#L31)

### Things we need to figure out

- Are the different implementations here functionally identical? (Doug's initial finding was "yes": https://github.com/dbt-labs/dbt-utils/pull/597#issuecomment-1142239494)
- If the data types are different, why?
- In what cases would their functional result differ? I'm imagining a case where Snowflake parameters [`TIMEZONE`](https://docs.snowflake.com/en/sql-reference/parameters.html#timezone) and [`TIMESTAMP_TYPE_MAPPING`](https://docs.snowflake.com/en/sql-reference/parameters.html#timestamp-type-mapping) are both set

### Ideal outcome

When I close my eyes and imagine how this could be really good:
1. We have _one_ method/macro, defined in `dbt-core` and implemented by each adapter plugin, that provides the SQL for getting the current timestamp. Whenever we need the current timestamp in a different data type, we use that `current_timestamp` macro plus a data type conversion function (see dbt-labs/dbt-utils#586).
2. We have a `convert_timezone` macro, defined in `dbt-core` and implemented by each adapter plugin. @clausherther [has already written one](https://github.com/calogica/dbt-date/blob/main/macros/calendar_date/convert_timezone.sql) for the `dbt-date` package—Claus, would you have any interest in kicking it upstream? :)
3. It's trivial to write a `current_timestamp_in_utc` macro that builds on top of 1+2. Whether that lives in `dbt-core`, `dbt-utils`, or somewhere else doesn't matter so much to me. Wherever it lives, it should look a lot like [`dbt_date.now(tz=None)`](https://github.com/calogica/dbt-date/blob/main/macros/calendar_date/now.sql). We'd leave the `dbt_utils` one in place for backwards compatibility.

### Questions

### Describe alternatives you've considered
Leaving all of it where it is :)

### Additional context

Timezones are no fun

This stuff is pretty tricky to test. [We're not _really_ testing it currently](https://github.com/dbt-labs/dbt-utils/blob/44eeb1978c1c720baca2e2cfe07671e94db02c7c/tests/functional/cross_db_utils/fixture_current_timestamp.py#L4). I had an idea in https://github.com/dbt-labs/dbt-utils/pull/577#discussion_r869149492, but it's not a great one. We are indirectly testing our use of `snapshot_get_time` in `dbt-core` functional tests (specifically [`test_hard_delete_snapshot`](https://github.com/dbt-labs/dbt-core/blob/main/tests/functional/simple_snapshot/test_hard_delete_snapshot.py).

### Who will this benefit?

- Users who want to call well-named macros, and expect them to do the thing they're meant to
- Claus, as maintainer of the `dbt-date` package (I hope!)


Discussion:
I have no idea what this means in terms of actual work, but I'm in. 👍 
Couple of unordered thoughts on `convert_timezone` from `dbt_date`:
- Timezone conversion at a minimum needs a target timezone. In `dbt_date`, we currently require a default project timezone via a global `var`, e.g. ` "dbt_date:time_zone": "America/Los_Angeles"`. That way something like `dbt_date.now()` by default [always returns](https://github.com/calogica/dbt-date/blob/20308b3f0225e905a290c85228b931652c4e3eb0/macros/calendar_date/now.sql#L2) a "local" time.
- Some platforms also allow you to specify a source timezone. So, for example, you can convert from something other than UTC to some other non-UTC timezone. So, `convert_timezone` has both `target_tz` and `source_tz` parameters, both optional. 
- In `dbt-date`, we effectively strip off the `tz` bundle on Snowflake by casting to a `timestamp_ntz`. I've found that to be the most sensible, since it IMO it better preserves the converted time.
Converting to UTC and then casting to `timestampntz` (`datetime` in BQ) generally seems like the Way To Go. As analytical practice, it makes sense to pull out the timezone and store as a separate column (`local_timezone`). Relevant: https://github.com/dbt-labs/docs.getdbt.com/discussions/1504
I'm going to transfer this to `dbt-core`. I think the real blocker there is around understanding + consolidating the different options there. I also want to keep the follow-on work from the cross-db migration organized in one place.
Just learned a couple of things on Snowflake:
- The 3 parameter version of convert_timezone that allows you to specify a source timezone does not accept a `timestamp_tz` column (any more?). So, one has to convert to `timestamp_ntz` before passing that in.
- If your SF instance/account/session/... is to set to timezone X and you're trying to convert what you think is a timestamp in timezone Z to timezone X, nothing will actually get converted, b/c SF assumes source and target are the same (and you just stripped off the tz 😞 )

Resolving patch:
diff --git a/core/dbt/clients/jinja_static.py b/core/dbt/clients/jinja_static.py
--- a/core/dbt/clients/jinja_static.py
+++ b/core/dbt/clients/jinja_static.py
@@ -15,7 +15,7 @@ def statically_extract_macro_calls(string, ctx, db_wrapper=None):
         if hasattr(func_call, "node") and hasattr(func_call.node, "name"):
             func_name = func_call.node.name
         else:
-            # func_call for dbt_utils.current_timestamp macro
+            # func_call for dbt.current_timestamp macro
             # Call(
             #   node=Getattr(
             #     node=Name(


Test patch:
diff --git a/core/dbt/tests/util.py b/core/dbt/tests/util.py
--- a/core/dbt/tests/util.py
+++ b/core/dbt/tests/util.py
@@ -4,7 +4,7 @@
 import json
 import warnings
 from datetime import datetime
-from typing import List
+from typing import Dict, List
 from contextlib import contextmanager
 from dbt.adapters.factory import Adapter
 
@@ -35,6 +35,7 @@
 #   relation_from_name
 #   check_relation_types (table/view)
 #   check_relations_equal
+#   check_relation_has_expected_schema
 #   check_relations_equal_with_relations
 #   check_table_does_exist
 #   check_table_does_not_exist
@@ -321,6 +322,17 @@ def check_relations_equal(adapter, relation_names: List, compare_snapshot_cols=F
     )
 
 
+# Used to check that a particular relation has an expected schema
+# expected_schema should look like {"column_name": "expected datatype"}
+def check_relation_has_expected_schema(adapter, relation_name, expected_schema: Dict):
+    relation = relation_from_name(adapter, relation_name)
+    with get_connection(adapter):
+        actual_columns = {c.name: c.data_type for c in adapter.get_columns_in_relation(relation)}
+    assert (
+        actual_columns == expected_schema
+    ), f"Actual schema did not match expected, actual: {json.dumps(actual_columns)}"
+
+
 # This can be used when checking relations in different schemas, by supplying
 # a list of relations. Called by 'check_relations_equal'.
 # Uses:
diff --git a/test/unit/test_macro_calls.py b/test/unit/test_macro_calls.py
--- a/test/unit/test_macro_calls.py
+++ b/test/unit/test_macro_calls.py
@@ -23,7 +23,7 @@ def setUp(self):
                     *
                     from {{ model }} )
                 {% endmacro %}""",
-            "{% macro test_my_test(model) %} select {{ dbt_utils.current_timestamp() }} {% endmacro %}",
+            "{% macro test_my_test(model) %} select {{ current_timestamp_backcompat() }} {% endmacro %}",
             "{% macro some_test(model) -%} {{ return(adapter.dispatch('test_some_kind4', 'foo_utils4')) }} {%- endmacro %}",
             "{% macro some_test(model) -%} {{ return(adapter.dispatch('test_some_kind5', macro_namespace = 'foo_utils5')) }} {%- endmacro %}",
         ]
@@ -34,7 +34,7 @@ def setUp(self):
             ['get_snapshot_unique_id'],
             ['get_columns_in_query'],
             ['get_snapshot_unique_id'],
-            ['dbt_utils.current_timestamp'],
+            ['current_timestamp_backcompat'],
             ['test_some_kind4', 'foo_utils4.test_some_kind4'],
             ['test_some_kind5', 'foo_utils5.test_some_kind5'],
         ]
diff --git a/tests/adapter/dbt/tests/adapter/utils/test_timestamps.py b/tests/adapter/dbt/tests/adapter/utils/test_timestamps.py
new file mode 100644
--- /dev/null
+++ b/tests/adapter/dbt/tests/adapter/utils/test_timestamps.py
@@ -0,0 +1,55 @@
+import pytest
+import re
+from dbt.tests.util import check_relation_has_expected_schema, run_dbt
+
+_MODEL_CURRENT_TIMESTAMP = """
+select {{ current_timestamp() }} as current_timestamp,
+       {{ current_timestamp_in_utc_backcompat() }} as current_timestamp_in_utc_backcompat,
+       {{ current_timestamp_backcompat() }} as current_timestamp_backcompat
+"""
+
+_MODEL_EXPECTED_SQL = """
+select now() as current_timestamp,
+       (current_timestamp at time zone 'utc')::TIMESTAMP as current_timestamp_in_utc_backcompat,
+       current_timestamp::TIMESTAMP as current_timestamp_backcompat
+"""
+
+
+class BaseCurrentTimestamps:
+    @pytest.fixture(scope="class")
+    def models(self):
+        return {"get_current_timestamp.sql": _MODEL_CURRENT_TIMESTAMP}
+
+    # any adapters that don't want to check can set expected schema to None
+    @pytest.fixture(scope="class")
+    def expected_sql(self):
+        return _MODEL_EXPECTED_SQL
+
+    @pytest.fixture(scope="class")
+    def expected_schema(self):
+        return {
+            "current_timestamp": "timestamp with time zone",
+            "current_timestamp_in_utc_backcompat": "timestamp without time zone",
+            "current_timestamp_backcompat": "timestamp without time zone",
+        }
+
+    def test_current_timestamps(self, project, models, expected_schema, expected_sql):
+        results = run_dbt(["run"])
+        assert len(results) == 1
+        check_relation_has_expected_schema(
+            project.adapter,
+            relation_name="get_current_timestamp",
+            expected_schema=expected_schema,
+        )
+
+        if expected_sql:
+            generated_sql = results.results[0].node.compiled_code
+            generated_sql_check = re.sub(r"\s+", "", generated_sql).lower()
+            expected_sql_check = re.sub(r"\s+", "", expected_sql).lower()
+            assert (
+                expected_sql_check == generated_sql_check
+            ), f"generated sql did not match expected: {generated_sql}"
+
+
+class TestCurrentTimestamps(BaseCurrentTimestamps):
+    pass