Repository: dbt-labs/dbt-core

Issue:
[CT-3325] [Tech Debt][Spike+] Test `dbt-core` with Python 3.12
### Housekeeping

- [X] I am a maintainer of dbt-core

### Short description

Python 3.12 was released last month, let's test that `dbt-core` works with this

https://www.python.org/downloads/release/python-3120/

### Acceptance criteria

* ~Bundles can be created with Python 3.12~
  * Removed since the current bundle integration tests are failing ([link](https://github.com/dbt-labs/dbt-core-bundles/actions/workflows/integration.yml)), will test with `dbt-core` 1.7.9 and 1.8.0b1
* dbt-common also supports python 3.12
* Update testing matrix to use python 3.12

### Impact to Other Teams

* upstream dependencies that will also need to be made compatible with 3.12: dbt-adapters dbt-semantic-interfaces
* 1p adapters


### Will backports be required?

No

### Context

https://docs.getdbt.com/faqs/Core/install-python-compatibility

“New dbt minor versions will add support for new Python3 minor versions as soon as all dependencies can support it. In turn, dbt minor versions will drop support for old Python3 minor versions right before they reach end of life.”


Discussion:
Deprecation notices appear when you run `dbt init` with `dbt-postgres` with Python 3.12:

```
dbt init -h
/Users/sam/.pyenv/versions/3.12.0/envs/myproject-dbt/lib/python3.12/site-packages/dateutil/tz/tz.py:37: DeprecationWarning: datetime.datetime.utcfromtimestamp() is deprecated and scheduled for removal in a future version. Use timezone-aware objects to represent datetimes in UTC: datetime.datetime.fromtimestamp(timestamp, datetime.UTC).
<frozen importlib._bootstrap>:488: DeprecationWarning: Type google._upb._message.MessageMapContainer uses PyType_Spec with a metaclass that has custom tp_new. This is deprecated and will no longer be allowed in Python 3.14.
<frozen importlib._bootstrap>:488: DeprecationWarning: Type google._upb._message.ScalarMapContainer uses PyType_Spec with a metaclass that has custom tp_new. This is deprecated and will no longer be allowed in Python 3.14.
Usage: dbt init [OPTIONS] [PROJECT_NAME]
...
```
I get the warning when I run dbt debug - also using dbt-postgres and Python 3.12

```
dbt debug
DeprecationWarning: datetime.datetime.utcfromtimestamp() is deprecated and scheduled for removal in a future version. Use timezone-aware objects to represent datetimes in UTC: datetime.datetime.fromtimestamp(timestamp, datetime.UTC).
<frozen importlib._bootstrap>:488: DeprecationWarning: Type google._upb._message.MessageMapContainer uses PyType_Spec with a metaclass that has custom tp_new. This is deprecated and will no longer be allowed in Python 3.14.
<frozen importlib._bootstrap>:488: DeprecationWarning: Type google._upb._message.ScalarMapContainer uses PyType_Spec with a metaclass that has custom tp_new. This is deprecated and will no longer be allowed in Python 3.14.
```
> I get the warning when I run dbt debug - also using dbt-postgres and Python 3.12
> 
> ```
> dbt debug
> DeprecationWarning: datetime.datetime.utcfromtimestamp() is deprecated and scheduled for removal in a future version. Use timezone-aware objects to represent datetimes in UTC: datetime.datetime.fromtimestamp(timestamp, datetime.UTC).
> <frozen importlib._bootstrap>:488: DeprecationWarning: Type google._upb._message.MessageMapContainer uses PyType_Spec with a metaclass that has custom tp_new. This is deprecated and will no longer be allowed in Python 3.14.
> <frozen importlib._bootstrap>:488: DeprecationWarning: Type google._upb._message.ScalarMapContainer uses PyType_Spec with a metaclass that has custom tp_new. This is deprecated and will no longer be allowed in Python 3.14.
> ```

FWIW those seem to come from dependencies rather than dbt-core itself.
Hi folks - I identified a new problem with dbt-core and Python 3.12.1.

3.12 formally removed `distutils`, which was deprecated in 3.10. This removal wasn't fully implemented until 3.12.1, which released this week. The `distutils` import `strtobool` is used in dbt-core in two places - [here](https://github.com/dbt-labs/dbt-core/blob/main/core/dbt/task/test.py) and [here](https://github.com/dbt-labs/dbt-core/blob/main/.github/actions/latest-wrangler/main.py). While some other functions of `distutils` were moved to other standard Python packages/functions/modules, `strtobool` was not and [the PEP 632 migration guide](https://peps.python.org/pep-0632/#migration-advice) suggests that project maintainers reimplement its functionality.
Hello - is there an update on this issue? It doesn't look to be a major reimplementation in the dbt-core logic - isolated to this file I believe: dbt-core/blob/main/core/dbt/task/test.py. I believe this logic would just have to be implemented directly instead of using strtobool: https://docs.python.org/3.9/distutils/apiref.html#distutils.util.strtobool

We'd like to stay on the current version of Python within a reasonable timeframe to reduce exposure to vulnerabilities. Thanks so much!! 



Resolving patch:
diff --git a/core/dbt/utils.py b/core/dbt/utils.py
--- a/core/dbt/utils.py
+++ b/core/dbt/utils.py
@@ -369,3 +369,21 @@ def args_to_dict(args):
 
         dict_args[key] = var_args[key]
     return dict_args
+
+
+# Taken from https://github.com/python/cpython/blob/3.11/Lib/distutils/util.py
+# This is a copy of the function from distutils.util, which was removed in Python 3.12.
+def strtobool(val: str) -> bool:
+    """Convert a string representation of truth to True or False.
+
+    True values are 'y', 'yes', 't', 'true', 'on', and '1'; false values
+    are 'n', 'no', 'f', 'false', 'off', and '0'.  Raises ValueError if
+    'val' is anything else.
+    """
+    val = val.lower()
+    if val in ("y", "yes", "t", "true", "on", "1"):
+        return True
+    elif val in ("n", "no", "f", "false", "off", "0"):
+        return False
+    else:
+        raise ValueError("invalid truth value %r" % (val,))
diff --git a/core/setup.py b/core/setup.py
--- a/core/setup.py
+++ b/core/setup.py
@@ -95,6 +95,7 @@
         "Programming Language :: Python :: 3.9",
         "Programming Language :: Python :: 3.10",
         "Programming Language :: Python :: 3.11",
+        "Programming Language :: Python :: 3.12",
     ],
     python_requires=">=3.8",
 )


Test patch:
diff --git a/core/dbt/task/test.py b/core/dbt/task/test.py
--- a/core/dbt/task/test.py
+++ b/core/dbt/task/test.py
@@ -1,12 +1,10 @@
-from distutils.util import strtobool
-
 import agate
 import daff
 import io
 import json
 import re
 from dataclasses import dataclass
-from dbt.utils import _coerce_decimal
+from dbt.utils import _coerce_decimal, strtobool
 from dbt_common.events.format import pluralize
 from dbt_common.dataclass_schema import dbtClassMixin
 import threading
diff --git a/tests/unit/test_plugin_manager.py b/tests/unit/test_plugin_manager.py
--- a/tests/unit/test_plugin_manager.py
+++ b/tests/unit/test_plugin_manager.py
@@ -101,13 +101,25 @@ def test_get_nodes(self, tracking, get_nodes_plugins):
         nodes = pm.get_nodes()
 
         assert len(nodes.models) == 2
-        assert tracking.track_plugin_get_nodes.called_once_with(
-            {
-                "plugin_name": get_nodes_plugins[0].name,
-                "num_model_nodes": 2,
-                "num_model_packages": 1,
-            }
-        )
+
+        expected_calls = [
+            mock.call(
+                {
+                    "plugin_name": get_nodes_plugins[0].name,
+                    "num_model_nodes": 1,
+                    "num_model_packages": 1,
+                }
+            ),
+            mock.call(
+                {
+                    "plugin_name": get_nodes_plugins[1].name,
+                    "num_model_nodes": 1,
+                    "num_model_packages": 1,
+                }
+            ),
+        ]
+
+        tracking.track_plugin_get_nodes.assert_has_calls(expected_calls)
 
     def test_get_manifest_artifact(self, get_artifacts_plugins):
         pm = PluginManager(plugins=get_artifacts_plugins)