Repository: dbt-labs/dbt-core

Issue:
[CT-1635] Populate adapter_response for tests + sources
### Describe the feature
#2747 requested that dbt log bytes processed for _all_ query types, including tests, source-freshness, and run-operations. This prompted the first part of a solution in #2961: adding an `adapter_response` dict to record database-specific information about the queries dbt has run. In v0.19.0, that will include `bytes_processed` for all materializations (model runs, seeds, snapshots).

We'd like to record bytes processed for tests and source snapshots, too. What does this really mean?
- `dbt test` should populate `adapter_response` in `run_results.json`
- `dbt source snapshot-freshness` should populate `adapter_response` in `sources.json` 

(I'm less clear on how we'd manage this in `run-operation`, since it doesn't write to `run_results`.)

Eventually, this has the potential to dovetail nicely with:
- #2079: add a new log line to the "summary" stdout, with the total bytes processed from the invocation. In an ideal world, users could get a human-friendly rounded number in stdout and the raw numbers in `run_results.json` for analysis/aggregation.
- #2580: the ability to define which result information is populated to `message` in stdout (maybe via Jinja macro??)

### Describe alternatives you've considered
- Not doing this

### Additional context
- Most immediately relevant to BigQuery (bytes processed)
- Useful for any database which sends back significant information in its response

### Who will this benefit?
- Projects with larger datasets, where filtering is important to limit cost of test queries



Discussion:
This issue has been marked as Stale because it has been open for 180 days with no activity. If you would like the issue to remain open, please remove the stale label or comment on the issue, or it will be closed in 7 days.
This issue has been marked as Stale because it has been open for 180 days with no activity. If you would like the issue to remain open, please remove the stale label or comment on the issue, or it will be closed in 7 days.
I think this may be as simple as using `result.adapter_response` to populate the `RunResult` for tests + freshness results, where currently we're setting them to just `{}`:

https://github.com/dbt-labs/dbt-core/blob/main/core/dbt/include/global_project/macros/materializations/models/table/create_table_as.sql

https://github.com/dbt-labs/dbt-core/blob/75f3e8cb749c7b52d226844394ecb34c1e0070a4/core/dbt/task/freshness.py#L123

We can look to the `run` task for the reference implementation:

https://github.com/dbt-labs/dbt-core/blob/75f3e8cb749c7b52d226844394ecb34c1e0070a4/core/dbt/task/run.py#L221-L223
One thing we'd need to change to get this working for `dbt source freshness`: Currently, the `collect_freshness` macro returns _just_ its `table` result. We need it to return the `response` object as well, which we can then serialize into an `adapter_response` dictionary.

To illustrate the changes that I think would be needed: https://github.com/dbt-labs/dbt-core/commit/a066c1e8bbdd54d7b8d658b1c3eb905d756606bf
hey @jtcohen6, this feature would be great.
I noticed the `help_wanted` label was recently removed, not sure (from reading [the docs](https://github.com/dbt-labs/dbt-core/blob/main/CONTRIBUTING.md)) whether this means contributions won't be accepted. But if possible I'd like to contribute.

> To illustrate the changes that I think would be needed: https://github.com/dbt-labs/dbt-core/commit/a066c1e8bbdd54d7b8d658b1c3eb905d756606bf

Regarding testing, would the following code change suffice (specifically for source freshness)? https://github.com/sebas-bdr/dbt-core/commit/572f1439efa17e096d6a53b866c37cd2cd43f213

I would love to have this feature to see how much my tests are costing!

I was able to get it working pretty easily for tests by returning the adapter response alongside the `TestResultData` object in `TestRunner.execute_test`:

https://github.com/dbt-labs/dbt-core/blob/89d111a5f6b0ddf925f3cb62c90164f6a038a1e3/core/dbt/task/test.py#L94
Same way that the ModelRunner does it: https://github.com/dbt-labs/dbt-core/blob/89d111a5f6b0ddf925f3cb62c90164f6a038a1e3/core/dbt/task/run.py#L219 Then the adapter response can get passed up through `TestRunner.execute`.

Maybe not the prettiest solution but wanted to advocate for this being very useful + straightforward! thx dbt devs <3

Resolving patch:
diff --git a/core/dbt/adapters/base/impl.py b/core/dbt/adapters/base/impl.py
--- a/core/dbt/adapters/base/impl.py
+++ b/core/dbt/adapters/base/impl.py
@@ -17,7 +17,6 @@
     Iterator,
     Set,
 )
-
 import agate
 import pytz
 
@@ -54,7 +53,7 @@
     CodeExecutionStatus,
     CatalogGenerationError,
 )
-from dbt.utils import filter_null_values, executor, cast_to_str
+from dbt.utils import filter_null_values, executor, cast_to_str, AttrDict
 
 from dbt.adapters.base.connections import Connection, AdapterResponse
 from dbt.adapters.base.meta import AdapterMeta, available
@@ -943,7 +942,7 @@ def execute_macro(
         context_override: Optional[Dict[str, Any]] = None,
         kwargs: Dict[str, Any] = None,
         text_only_columns: Optional[Iterable[str]] = None,
-    ) -> agate.Table:
+    ) -> AttrDict:
         """Look macro_name up in the manifest and execute its results.
 
         :param macro_name: The name of the macro to execute.
@@ -1028,7 +1027,7 @@ def _get_one_catalog(
             manifest=manifest,
         )
 
-        results = self._catalog_filter_table(table, manifest)
+        results = self._catalog_filter_table(table, manifest)  # type: ignore[arg-type]
         return results
 
     def get_catalog(self, manifest: Manifest) -> Tuple[agate.Table, List[Exception]]:
@@ -1060,7 +1059,7 @@ def calculate_freshness(
         loaded_at_field: str,
         filter: Optional[str],
         manifest: Optional[Manifest] = None,
-    ) -> Dict[str, Any]:
+    ) -> Tuple[AdapterResponse, Dict[str, Any]]:
         """Calculate the freshness of sources in dbt, and return it"""
         kwargs: Dict[str, Any] = {
             "source": source,
@@ -1069,7 +1068,8 @@ def calculate_freshness(
         }
 
         # run the macro
-        table = self.execute_macro(FRESHNESS_MACRO_NAME, kwargs=kwargs, manifest=manifest)
+        result = self.execute_macro(FRESHNESS_MACRO_NAME, kwargs=kwargs, manifest=manifest)
+        adapter_response, table = result.response, result.table  # type: ignore[attr-defined]
         # now we have a 1-row table of the maximum `loaded_at_field` value and
         # the current time according to the db.
         if len(table) != 1 or len(table[0]) != 2:
@@ -1083,11 +1083,12 @@ def calculate_freshness(
 
         snapshotted_at = _utc(table[0][1], source, loaded_at_field)
         age = (snapshotted_at - max_loaded_at).total_seconds()
-        return {
+        freshness = {
             "max_loaded_at": max_loaded_at,
             "snapshotted_at": snapshotted_at,
             "age": age,
         }
+        return adapter_response, freshness
 
     def pre_model_hook(self, config: Mapping[str, Any]) -> Any:
         """A hook for running some operation before the model materialization
diff --git a/core/dbt/task/freshness.py b/core/dbt/task/freshness.py
--- a/core/dbt/task/freshness.py
+++ b/core/dbt/task/freshness.py
@@ -105,10 +105,10 @@ def execute(self, compiled_node, manifest):
             )
 
         relation = self.adapter.Relation.create_from_source(compiled_node)
-        # given a Source, calculate its fresnhess.
+        # given a Source, calculate its freshness.
         with self.adapter.connection_for(compiled_node):
             self.adapter.clear_transaction()
-            freshness = self.adapter.calculate_freshness(
+            adapter_response, freshness = self.adapter.calculate_freshness(
                 relation,
                 compiled_node.loaded_at_field,
                 compiled_node.freshness.filter,
@@ -124,7 +124,7 @@ def execute(self, compiled_node, manifest):
             timing=[],
             execution_time=0,
             message=None,
-            adapter_response={},
+            adapter_response=adapter_response.to_dict(omit_none=True),
             failures=None,
             **freshness,
         )


Test patch:
diff --git a/core/dbt/task/test.py b/core/dbt/task/test.py
--- a/core/dbt/task/test.py
+++ b/core/dbt/task/test.py
@@ -5,6 +5,7 @@
 from dbt.events.format import pluralize
 from dbt.dataclass_schema import dbtClassMixin
 import threading
+from typing import Dict, Any
 
 from .compile import CompileRunner
 from .run import RunTask
@@ -38,6 +39,7 @@ class TestResultData(dbtClassMixin):
     failures: int
     should_warn: bool
     should_error: bool
+    adapter_response: Dict[str, Any]
 
     @classmethod
     def validate(cls, data):
@@ -137,6 +139,7 @@ def execute_test(self, test: TestNode, manifest: Manifest) -> TestResultData:
                 map(_coerce_decimal, table.rows[0]),
             )
         )
+        test_result_dct["adapter_response"] = result["response"].to_dict(omit_none=True)
         TestResultData.validate(test_result_dct)
         return TestResultData.from_dict(test_result_dct)
 
@@ -171,7 +174,7 @@ def execute(self, test: TestNode, manifest: Manifest):
             thread_id=thread_id,
             execution_time=0,
             message=message,
-            adapter_response={},
+            adapter_response=result.adapter_response,
             failures=failures,
         )
 
diff --git a/tests/functional/custom_singular_tests/test_custom_singular_tests.py b/tests/functional/custom_singular_tests/test_custom_singular_tests.py
--- a/tests/functional/custom_singular_tests/test_custom_singular_tests.py
+++ b/tests/functional/custom_singular_tests/test_custom_singular_tests.py
@@ -103,3 +103,8 @@ def test_data_tests(self, project, tests):
             assert result.status == "fail"
             assert not result.skipped
             assert result.failures > 0
+            assert result.adapter_response == {
+                "_message": "SELECT 1",
+                "code": "SELECT",
+                "rows_affected": 1,
+            }
diff --git a/tests/functional/sources/test_source_fresher_state.py b/tests/functional/sources/test_source_fresher_state.py
--- a/tests/functional/sources/test_source_fresher_state.py
+++ b/tests/functional/sources/test_source_fresher_state.py
@@ -112,7 +112,7 @@ def _assert_freshness_results(self, path, state):
                     "warn_after": {"count": 10, "period": "hour"},
                     "error_after": {"count": 18, "period": "hour"},
                 },
-                "adapter_response": {},
+                "adapter_response": {"_message": "SELECT 1", "code": "SELECT", "rows_affected": 1},
                 "thread_id": AnyStringWith("Thread-"),
                 "execution_time": AnyFloat(),
                 "timing": [
diff --git a/tests/functional/sources/test_source_freshness.py b/tests/functional/sources/test_source_freshness.py
--- a/tests/functional/sources/test_source_freshness.py
+++ b/tests/functional/sources/test_source_freshness.py
@@ -103,7 +103,7 @@ def _assert_freshness_results(self, path, state):
                     "warn_after": {"count": 10, "period": "hour"},
                     "error_after": {"count": 18, "period": "hour"},
                 },
-                "adapter_response": {},
+                "adapter_response": {"_message": "SELECT 1", "code": "SELECT", "rows_affected": 1},
                 "thread_id": AnyStringWith("Thread-"),
                 "execution_time": AnyFloat(),
                 "timing": [