Repository: dbt-labs/dbt-core

Issue:
[CT-1003] run_results.json not generated when on-run-end fails
### Is this a new bug in dbt-core?

- [X] I believe this is a new bug in dbt-core
- [X] I have searched the existing issues, and I could not find an existing issue for this bug

### Current Behavior

The dbt artifact `run_results.json` is not generated when a statement in the `on-run-end` fails. This behavior does not occur when the statement fails in a post-hook.

### Expected Behavior

`run_results.json` must be generated even if a statement fails at the `on-run-end` and must contain dbt model execution results to indicate which models ran successfully or failed.

### Steps To Reproduce

1. Create a sample dbt project with `dbt init`
2. Run `dbt build` and confirm `target/run_results.json` exists.
3. Run `dbt clean` to remove the `target` directory.
4. Add `on-run-end: "select col1"` in the `dbt_project.yml` and it causes dbt a `Database error`.
5. Run `dbt build` and confirm `target/run_results.json` does not exist.

### Relevant log output

_No response_

### Environment

```markdown
- OS: MacOS 12.3.1
- Python: 3.8.5
- dbt: 1.1 and 1.2
```


### Which database adapter are you using with dbt?

postgres, redshift

### Additional Context

_No response_


Discussion:
I am able to reproduce this easily enough. This should be fixed. Thanks for reporting @seunghanhong!

Acceptance criteria: runs failing from `on-run-end` hooks should still results in a `run_results.json` entry with details on the failure. 
N.B. Be sure to check with cloud that the existence of this file isn't being used as a indicator of a successful run.
> N.B. Be sure to check with cloud that the existence of this file isn't being used as a indicator of a successful run.

My sense is that:
- dbt Cloud job scheduler uses the exit code produced by dbt-core's CLI to determine whether the job succeeded. In this case, an error was raised, so it should be a nonzero exit code (error) regardless
- If `run_results.json` is not produced, it does prevent the Metadata API from ingesting that artifact and being able to report on that run. Resolution of this ticket will allow the Metadata API to include information about runs with `on-run-end` hooks that raise errors.

Resolving patch:
diff --git a/core/dbt/contracts/results.py b/core/dbt/contracts/results.py
--- a/core/dbt/contracts/results.py
+++ b/core/dbt/contracts/results.py
@@ -220,7 +220,9 @@ def from_execution_results(
         generated_at: datetime,
         args: Dict,
     ):
-        processed_results = [process_run_result(result) for result in results]
+        processed_results = [
+            process_run_result(result) for result in results if isinstance(result, RunResult)
+        ]
         meta = RunResultsMetadata(
             dbt_schema_version=str(cls.dbt_schema_version),
             generated_at=generated_at,
diff --git a/core/dbt/task/run.py b/core/dbt/task/run.py
--- a/core/dbt/task/run.py
+++ b/core/dbt/task/run.py
@@ -20,7 +20,7 @@
 from dbt.contracts.graph.compiled import CompileResultNode
 from dbt.contracts.graph.model_config import Hook
 from dbt.contracts.graph.parsed import ParsedHookNode
-from dbt.contracts.results import NodeStatus, RunResult, RunStatus, RunningStatus
+from dbt.contracts.results import NodeStatus, RunResult, RunStatus, RunningStatus, BaseResult
 from dbt.exceptions import (
     CompilationException,
     InternalException,
@@ -394,12 +394,22 @@ def safe_run_hooks(
     ) -> None:
         try:
             self.run_hooks(adapter, hook_type, extra_context)
-        except RuntimeException:
+        except RuntimeException as exc:
             fire_event(DatabaseErrorRunningHook(hook_type=hook_type.value))
-            raise
+            self.node_results.append(
+                BaseResult(
+                    status=RunStatus.Error,
+                    thread_id="main",
+                    timing=[],
+                    message=f"{hook_type.value} failed, error:\n {exc.msg}",
+                    adapter_response=exc.msg,
+                    execution_time=0,
+                    failures=1,
+                )
+            )
 
     def print_results_line(self, results, execution_time):
-        nodes = [r.node for r in results] + self.ran_hooks
+        nodes = [r.node for r in results if hasattr(r, "node")] + self.ran_hooks
         stat_line = get_counts(nodes)
 
         execution = ""
@@ -444,9 +454,6 @@ def after_run(self, adapter, results):
         with adapter.connection_named("master"):
             self.safe_run_hooks(adapter, RunHookType.End, extras)
 
-    def after_hooks(self, adapter, results, elapsed):
-        self.print_results_line(results, elapsed)
-
     def get_node_selector(self) -> ResourceTypeSelector:
         if self.manifest is None or self.graph is None:
             raise InternalException("manifest and graph must be set to get perform node selection")
diff --git a/core/dbt/task/runnable.py b/core/dbt/task/runnable.py
--- a/core/dbt/task/runnable.py
+++ b/core/dbt/task/runnable.py
@@ -413,9 +413,6 @@ def populate_adapter_cache(self, adapter, required_schemas: Set[BaseRelation] =
                 {"adapter_cache_construction_elapsed": cache_populate_time}
             )
 
-    def before_hooks(self, adapter):
-        pass
-
     def before_run(self, adapter, selected_uids: AbstractSet[str]):
         with adapter.connection_named("master"):
             self.populate_adapter_cache(adapter)
@@ -423,24 +420,24 @@ def before_run(self, adapter, selected_uids: AbstractSet[str]):
     def after_run(self, adapter, results):
         pass
 
-    def after_hooks(self, adapter, results, elapsed):
+    def print_results_line(self, node_results, elapsed):
         pass
 
     def execute_with_hooks(self, selected_uids: AbstractSet[str]):
         adapter = get_adapter(self.config)
+        started = time.time()
         try:
-            self.before_hooks(adapter)
-            started = time.time()
             self.before_run(adapter, selected_uids)
             res = self.execute_nodes()
             self.after_run(adapter, res)
-            elapsed = time.time() - started
-            self.after_hooks(adapter, res, elapsed)
-
         finally:
             adapter.cleanup_connections()
+            elapsed = time.time() - started
+            self.print_results_line(self.node_results, elapsed)
+            result = self.get_result(
+                results=self.node_results, elapsed_time=elapsed, generated_at=datetime.utcnow()
+            )
 
-        result = self.get_result(results=res, elapsed_time=elapsed, generated_at=datetime.utcnow())
         return result
 
     def write_result(self, result):


Test patch: