Repository: dbt-labs/dbt-core

Issue:
[CT-3185] Hundreds of: Skipping due to fail_fast
### Is this a new bug in dbt-core?

- [X] I believe this is a new bug in dbt-core
- [X] I have searched the existing issues, and I could not find an existing issue for this bug

I searched open and closed issues for "Skipping due to fail_fast".

### Current Behavior

I can get as many as hundreds of these. 900 in the run I've just done, because something near the beginning of the dag failed - because I'm setting up clones of databases with different permissions to test something.

This wasn't always the case, I feel like it started fairly recently. Weeks ago, not months ago. We moved from 1.5.2 to 1.6.1 on the 13th of September. I checked the behaviour is the same in 1.6.1 and 1.6.5.

```
00:46:42    Skipping due to fail_fast
00:46:42
00:46:42    Skipping due to fail_fast
00:46:42
00:46:42    Skipping due to fail_fast
00:46:42
00:46:42    Skipping due to fail_fast
00:46:42
00:46:42    Skipping due to fail_fast
00:46:42
00:46:42    Skipping due to fail_fast
00:46:42
00:46:42    Skipping due to fail_fast
00:46:42
00:46:42    Skipping due to fail_fast
00:46:42
00:46:42    Skipping due to fail_fast
00:46:42
00:46:42    Skipping due to fail_fast
00:46:42
00:46:42    Skipping due to fail_fast
00:46:42
00:46:42    Skipping due to fail_fast
00:46:42
```

### Expected Behavior

Since it is not saying what it is skipping, and all of those are the same, it should just say something once. Not sure what. Maybe "Skipping ### models due to fail_fast", ### being the number of models skipped.

### Steps To Reproduce

1. Set up a failure at the beginning of the DAG.
2. Run: `dbt run --fail-fast`

### Relevant log output

_No response_

### Environment

```markdown
- OS: Linux
- Python: 3.10.13
- dbt: 1.6.5
- dbt-snowflake: 1.6.4
```


### Which database adapter are you using with dbt?

snowflake

### Additional Context

_No response_


Discussion:
Thanks for opening @nevdelap 

Oof! Yeah, showing that message 900+ times is not ideal. I was able to reproduce this. 

<details>
19:53:19  Concurrency: 8 threads (target='dev')
19:53:19  
19:53:19  1 of 81 START sql view model DBT_GGOHEEN.stg_tpcds_core__call_center ........... [RUN]
19:53:19  2 of 81 START sql view model DBT_GGOHEEN.stg_tpcds_core__catalog_returns ....... [RUN]
19:53:19  3 of 81 START sql view model DBT_GGOHEEN.stg_tpcds_core__catalog_sales ......... [RUN]
19:53:19  4 of 81 START sql view model DBT_GGOHEEN.stg_tpcds_core__customer .............. [RUN]
19:53:19  5 of 81 START sql view model DBT_GGOHEEN.stg_tpcds_core__customer_address ...... [RUN]
19:53:19  6 of 81 START sql view model DBT_GGOHEEN.stg_tpcds_core__customer_demographics . [RUN]
19:53:19  7 of 81 START sql view model DBT_GGOHEEN.stg_tpcds_core__date_dim .............. [RUN]
19:53:19  8 of 81 START sql view model DBT_GGOHEEN.stg_tpcds_core__household_demographics  [RUN]
19:53:20  4 of 81 OK created sql view model DBT_GGOHEEN.stg_tpcds_core__customer ......... [SUCCESS 1 in 1.40s]
19:53:20  9 of 81 START sql view model DBT_GGOHEEN.stg_tpcds_core__income_band ........... [RUN]
19:53:21  1 of 81 ERROR creating sql view model DBT_GGOHEEN.stg_tpcds_core__call_center .. [ERROR in 1.57s]
19:53:21  10 of 81 START sql view model DBT_GGOHEEN.stg_tpcds_core__item ................. [RUN]
19:53:21  CANCEL query model.mega_corp_big_co_inc.stg_tpcds_core__item ................... [CANCEL]
19:53:21  CANCEL query model.mega_corp_big_co_inc.stg_tpcds_core__catalog_returns ........ [CANCEL]
19:53:21  6 of 81 OK created sql view model DBT_GGOHEEN.stg_tpcds_core__customer_demographics  [SUCCESS 1 in 2.42s]
19:53:21  7 of 81 OK created sql view model DBT_GGOHEEN.stg_tpcds_core__date_dim ......... [SUCCESS 1 in 2.42s]
19:53:21  5 of 81 OK created sql view model DBT_GGOHEEN.stg_tpcds_core__customer_address . [SUCCESS 1 in 2.43s]
19:53:21  3 of 81 OK created sql view model DBT_GGOHEEN.stg_tpcds_core__catalog_sales .... [SUCCESS 1 in 2.39s]
19:53:21  CANCEL query model.mega_corp_big_co_inc.stg_tpcds_core__income_band ............ [CANCEL]
19:53:21  8 of 81 OK created sql view model DBT_GGOHEEN.stg_tpcds_core__household_demographics  [SUCCESS 1 in 2.37s]
19:53:21  2 of 81 OK created sql view model DBT_GGOHEEN.stg_tpcds_core__catalog_returns .. [SUCCESS 1 in 2.46s]
19:53:21  CANCEL query model.mega_corp_big_co_inc.stg_tpcds_core__customer_address ....... [CANCEL]
19:53:21  CANCEL query model.mega_corp_big_co_inc.stg_tpcds_core__customer_demographics .. [CANCEL]
19:53:21  CANCEL query model.mega_corp_big_co_inc.stg_tpcds_core__date_dim ............... [CANCEL]
19:53:21  CANCEL query model.mega_corp_big_co_inc.stg_tpcds_core__catalog_sales .......... [CANCEL]
19:53:21  CANCEL query model.mega_corp_big_co_inc.stg_tpcds_core__household_demographics . [CANCEL]
19:53:22  9 of 81 OK created sql view model DBT_GGOHEEN.stg_tpcds_core__income_band ...... [SUCCESS 1 in 1.15s]
19:53:22  10 of 81 OK created sql view model DBT_GGOHEEN.stg_tpcds_core__item ............ [SUCCESS 1 in 1.23s]
19:53:22  
19:53:22  Database Error in model stg_tpcds_core__call_center (models/staging/tpcds_core/stg_tpcds_core__call_center.sql)
19:53:22    001003 (42000): SQL compilation error:
19:53:22    syntax error line 16 at position 8 unexpected 'cc_call_center_sk'.
19:53:22    syntax error line 17 at position 29 unexpected 'call_center_id'.
19:53:22    syntax error line 18 at position 29 unexpected 'rec_start_date'.
19:53:22    syntax error line 19 at position 27 unexpected 'rec_end_date'.
19:53:22    syntax error line 20 at position 29 unexpected 'closed_date_sk'.
19:53:22    syntax error line 21 at position 27 unexpected 'open_date_sk'.
19:53:22    syntax error line 22 at position 19 unexpected 'name'.
19:53:22    syntax error line 23 at position 20 unexpected 'class'.
19:53:22    syntax error line 24 at position 24 unexpected 'employees'.
19:53:22    syntax error line 25 at position 20 unexpected 'sq_ft'.
19:53:22    syntax error line 26 at position 20 unexpected 'hours'.
19:53:22    syntax error line 27 at position 22 unexpected 'manager'.
19:53:22    syntax error line 28 at position 21 unexpected 'mkt_id'.
19:53:22    syntax error line 29 at position 24 unexpected 'mkt_class'.
19:53:22    syntax error line 30 at position 23 unexpected 'mkt_desc'.
19:53:22    syntax error line 31 at position 29 unexpected 'market_manager'.
19:53:22    syntax error line 32 at position 23 unexpected 'division'.
19:53:22    syntax error line 33 at position 28 unexpected 'division_name'.
19:53:22    syntax error line 34 at position 22 unexpected 'company'.
19:53:22    syntax error line 35 at position 27 unexpected 'company_name'.
19:53:22    syntax error line 36 at position 28 unexpected 'street_number'.
19:53:22    syntax error line 37 at position 26 unexpected 'street_name'.
19:53:22    syntax error line 38 at position 26 unexpected 'street_type'.
19:53:22    syntax error line 39 at position 27 unexpected 'suite_number'.
19:53:22    syntax error line 40 at position 19 unexpected 'city'.
19:53:22    syntax error line 41 at position 21 unexpected 'county'.
19:53:22    syntax error line 42 at position 20 unexpected 'state'.
19:53:22    syntax error line 43 at position 18 unexpected 'zip'.
19:53:22    syntax error line 44 at position 22 unexpected 'country'.
19:53:22    syntax error line 45 at position 25 unexpected 'gmt_offset'.
19:53:22    syntax error line 46 at position 29 unexpected 'tax_percentage'.
19:53:22    syntax error line 52 at position 0 unexpected 'select'.
19:53:22    compiled Code at target/run/mega_corp_big_co_inc/models/staging/tpcds_core/stg_tpcds_core__call_center.sql
19:53:22  
19:53:22  Finished running 19 view models, 54 tests, 8 table models in 0 hours 0 minutes and 14.09 seconds (14.09s).
19:53:22  
19:53:22  Completed with 80 errors and 0 warnings:
19:53:22  
19:53:22  Database Error in model stg_tpcds_core__call_center (models/staging/tpcds_core/stg_tpcds_core__call_center.sql)
19:53:22    001003 (42000): SQL compilation error:
19:53:22    syntax error line 16 at position 8 unexpected 'cc_call_center_sk'.
19:53:22    syntax error line 17 at position 29 unexpected 'call_center_id'.
19:53:22    syntax error line 18 at position 29 unexpected 'rec_start_date'.
19:53:22    syntax error line 19 at position 27 unexpected 'rec_end_date'.
19:53:22    syntax error line 20 at position 29 unexpected 'closed_date_sk'.
19:53:22    syntax error line 21 at position 27 unexpected 'open_date_sk'.
19:53:22    syntax error line 22 at position 19 unexpected 'name'.
19:53:22    syntax error line 23 at position 20 unexpected 'class'.
19:53:22    syntax error line 24 at position 24 unexpected 'employees'.
19:53:22    syntax error line 25 at position 20 unexpected 'sq_ft'.
19:53:22    syntax error line 26 at position 20 unexpected 'hours'.
19:53:22    syntax error line 27 at position 22 unexpected 'manager'.
19:53:22    syntax error line 28 at position 21 unexpected 'mkt_id'.
19:53:22    syntax error line 29 at position 24 unexpected 'mkt_class'.
19:53:22    syntax error line 30 at position 23 unexpected 'mkt_desc'.
19:53:22    syntax error line 31 at position 29 unexpected 'market_manager'.
19:53:22    syntax error line 32 at position 23 unexpected 'division'.
19:53:22    syntax error line 33 at position 28 unexpected 'division_name'.
19:53:22    syntax error line 34 at position 22 unexpected 'company'.
19:53:22    syntax error line 35 at position 27 unexpected 'company_name'.
19:53:22    syntax error line 36 at position 28 unexpected 'street_number'.
19:53:22    syntax error line 37 at position 26 unexpected 'street_name'.
19:53:22    syntax error line 38 at position 26 unexpected 'street_type'.
19:53:22    syntax error line 39 at position 27 unexpected 'suite_number'.
19:53:22    syntax error line 40 at position 19 unexpected 'city'.
19:53:22    syntax error line 41 at position 21 unexpected 'county'.
19:53:22    syntax error line 42 at position 20 unexpected 'state'.
19:53:22    syntax error line 43 at position 18 unexpected 'zip'.
19:53:22    syntax error line 44 at position 22 unexpected 'country'.
19:53:22    syntax error line 45 at position 25 unexpected 'gmt_offset'.
19:53:22    syntax error line 46 at position 29 unexpected 'tax_percentage'.
19:53:22    syntax error line 52 at position 0 unexpected 'select'.
19:53:22    compiled Code at target/run/mega_corp_big_co_inc/models/staging/tpcds_core/stg_tpcds_core__call_center.sql
19:53:22  
19:53:22  Skipping due to fail_fast
19:53:22  
19:53:22  Skipping due to fail_fast
19:53:22  
19:53:22  Skipping due to fail_fast
19:53:22  
19:53:22  Skipping due to fail_fast
19:53:22  
19:53:22  Skipping due to fail_fast
19:53:22  
19:53:22  Skipping due to fail_fast
19:53:22  
19:53:22  Skipping due to fail_fast
19:53:22  
19:53:22  Skipping due to fail_fast
19:53:22  
19:53:22  Skipping due to fail_fast
19:53:22  
19:53:22  Skipping due to fail_fast
19:53:22  
19:53:22  Skipping due to fail_fast
19:53:22  
19:53:22  Skipping due to fail_fast
19:53:22  
19:53:22  Skipping due to fail_fast
19:53:22  
19:53:22  Skipping due to fail_fast
19:53:22  
19:53:22  Skipping due to fail_fast
19:53:22  
19:53:22  Skipping due to fail_fast
19:53:22  
19:53:22  Skipping due to fail_fast
19:53:22  
19:53:22  Skipping due to fail_fast
19:53:22  
19:53:22  Skipping due to fail_fast
19:53:22  
19:53:22  Skipping due to fail_fast
19:53:22  
19:53:22  Skipping due to fail_fast
19:53:22  
19:53:22  Skipping due to fail_fast
19:53:22  
19:53:22  Skipping due to fail_fast
19:53:22  
19:53:22  Skipping due to fail_fast
19:53:22  
19:53:22  Skipping due to fail_fast
19:53:22  
19:53:22  Skipping due to fail_fast
19:53:22  
19:53:22  Skipping due to fail_fast
19:53:22  
19:53:22  Skipping due to fail_fast
19:53:22  
19:53:22  Skipping due to fail_fast
19:53:22  
19:53:22  Skipping due to fail_fast
19:53:22  
19:53:22  Skipping due to fail_fast
19:53:22  
19:53:22  Skipping due to fail_fast
19:53:22  
19:53:22  Skipping due to fail_fast
19:53:22  
19:53:22  Skipping due to fail_fast
19:53:22  
19:53:22  Skipping due to fail_fast
19:53:22  
19:53:22  Skipping due to fail_fast
19:53:22  
19:53:22  Skipping due to fail_fast
19:53:22  
19:53:22  Skipping due to fail_fast
19:53:22  
19:53:22  Skipping due to fail_fast
19:53:22  
19:53:22  Skipping due to fail_fast
19:53:22  
19:53:22  Skipping due to fail_fast
19:53:22  
19:53:22  Skipping due to fail_fast
19:53:22  
19:53:22  Skipping due to fail_fast
19:53:22  
19:53:22  Skipping due to fail_fast
19:53:22  
19:53:22  Skipping due to fail_fast
19:53:22  
19:53:22  Skipping due to fail_fast
19:53:22  
19:53:22  Skipping due to fail_fast
19:53:22  
19:53:22  Skipping due to fail_fast
19:53:22  
19:53:22  Skipping due to fail_fast
19:53:22  
19:53:22  Skipping due to fail_fast
19:53:22  
19:53:22  Skipping due to fail_fast
19:53:22  
19:53:22  Skipping due to fail_fast
19:53:22  
19:53:22  Skipping due to fail_fast
19:53:22  
19:53:22  Skipping due to fail_fast
19:53:22  
19:53:22  Skipping due to fail_fast
19:53:22  
19:53:22  Skipping due to fail_fast
19:53:22  
19:53:22  Skipping due to fail_fast
19:53:22  
19:53:22  Skipping due to fail_fast
19:53:22  
19:53:22  Skipping due to fail_fast
19:53:22  
19:53:22  Skipping due to fail_fast
19:53:22  
19:53:22  Skipping due to fail_fast
19:53:22  
19:53:22  Skipping due to fail_fast
19:53:22  
19:53:22  Skipping due to fail_fast
19:53:22  
19:53:22  Skipping due to fail_fast
19:53:22  
19:53:22  Skipping due to fail_fast
19:53:22  
19:53:22  Skipping due to fail_fast
19:53:22  
19:53:22  Skipping due to fail_fast
19:53:22  
19:53:22  Skipping due to fail_fast
19:53:22  
19:53:22  Skipping due to fail_fast
19:53:22  
19:53:22  Skipping due to fail_fast
19:53:22  
19:53:22  Skipping due to fail_fast
19:53:22  
19:53:22  Skipping due to fail_fast
19:53:22  
19:53:22  Skipping due to fail_fast
19:53:22  
19:53:22  Skipping due to fail_fast
19:53:22  
19:53:22  Skipping due to fail_fast
19:53:22  
19:53:22  Skipping due to fail_fast
19:53:22  
19:53:22  Skipping due to fail_fast
19:53:22  
19:53:22  Skipping due to fail_fast
19:53:22  
19:53:22  Skipping due to fail_fast
19:53:22  
19:53:22  Done. PASS=1 WARN=0 ERROR=1 SKIP=79 TOTAL=81

</details>

I think either we'd want to:
- specify which model was being skipped in each message "Skipping model_x due to fail_fast"
- have a single message, that specifies the number of models being skipped "Skipping ### models due to fail_fast" (your suggestion!)

Since we already show which models are "CANCEL"ed in the logs, I think option 2 would be a better user experience. I'm not sure what the technical implications would be of doing this, but I'll queue this up for our team to take a look at
I suggest having a single message, that specifies the number of models being skipped "Skipping ### models due to fail_fast"

@graciegoheen @dbeatty10 is there any plan to address this? 
Honestly, I would be fine with simply moving this message to DEBUG level verbosity. If I've specified fail-fast, the only thing I want to see after the first failure is what failed and why. In the current behaviour, I can't even see the error message because my terminal history is full of "Skipping <model> due to fail_fast", which is pretty ironic.

Resolving patch:
diff --git a/core/dbt/task/printer.py b/core/dbt/task/printer.py
--- a/core/dbt/task/printer.py
+++ b/core/dbt/task/printer.py
@@ -13,6 +13,7 @@
     StatsLine,
 )
 from dbt.node_types import NodeType
+from dbt_common.events.base_types import EventLevel
 from dbt_common.events.format import pluralize
 from dbt_common.events.functions import fire_event
 from dbt_common.events.types import Formatting
@@ -68,14 +69,13 @@ def print_run_status_line(results) -> None:
 
 
 def print_run_result_error(result, newline: bool = True, is_warning: bool = False) -> None:
-    if newline:
-        fire_event(Formatting(""))
-
     # set node_info for logging events
     node_info = None
     if hasattr(result, "node") and result.node:
         node_info = result.node.node_info
     if result.status == NodeStatus.Fail or (is_warning and result.status == NodeStatus.Warn):
+        if newline:
+            fire_event(Formatting(""))
         if is_warning:
             fire_event(
                 RunResultWarning(
@@ -112,8 +112,13 @@ def print_run_result_error(result, newline: bool = True, is_warning: bool = Fals
             fire_event(
                 CheckNodeTestFailure(relation_name=result.node.relation_name, node_info=node_info)
             )
-
+    elif result.status == NodeStatus.Skipped and result.message is not None:
+        if newline:
+            fire_event(Formatting(""), level=EventLevel.DEBUG)
+        fire_event(RunResultError(msg=result.message), level=EventLevel.DEBUG)
     elif result.message is not None:
+        if newline:
+            fire_event(Formatting(""))
         fire_event(RunResultError(msg=result.message, node_info=node_info))
 
 


Test patch:
diff --git a/tests/functional/minimal_cli/test_minimal_cli.py b/tests/functional/minimal_cli/test_minimal_cli.py
--- a/tests/functional/minimal_cli/test_minimal_cli.py
+++ b/tests/functional/minimal_cli/test_minimal_cli.py
@@ -53,6 +53,38 @@ def test_build(self, runner, project):
         assert "SKIP=1" in result.output
 
 
+class TestBuildFailFast(BaseConfigProject):
+    def test_build(self, runner, project):
+        runner.invoke(cli, ["deps"])
+        result = runner.invoke(cli, ["build", "--fail-fast"])
+        # 1 seed, 1 model, 2 data tests
+        assert "PASS=4" in result.output
+        # 2 data tests
+        assert "ERROR=2" in result.output
+        # Singular test
+        assert "WARN=1" in result.output
+        # 1 snapshot
+        assert "SKIP=1" in result.output
+        # Skipping due to fail_fast is not shown when --debug is not specified.
+        assert "Skipping due to fail_fast" not in result.output
+
+
+class TestBuildFailFastDebug(BaseConfigProject):
+    def test_build(self, runner, project):
+        runner.invoke(cli, ["deps"])
+        result = runner.invoke(cli, ["build", "--fail-fast", "--debug"])
+        # 1 seed, 1 model, 2 data tests
+        assert "PASS=4" in result.output
+        # 2 data tests
+        assert "ERROR=2" in result.output
+        # Singular test
+        assert "WARN=1" in result.output
+        # 1 snapshot
+        assert "SKIP=1" in result.output
+        # Skipping due to fail_fast is shown when --debug is specified.
+        assert "Skipping due to fail_fast" in result.output
+
+
 class TestDocsGenerate(BaseConfigProject):
     def test_docs_generate(self, runner, project):
         runner.invoke(cli, ["deps"])