Repository: dbt-labs/dbt-core

Issue:
[CT-1737] Support all "global" flags after all subcommands
**See comment below for updated proposal**

We currently distinguish between two types of flags:
- those which precede the subcommand ("global"), e.g. `--no-use-color`
- those which follow the subcommand, and may have different behavior depending on the subcommand, e.g. `--select`, `--full-refresh`

IMO this is pretty confusing for end users! We should really seek to support flags in any order (and resolve any resulting ambiguities by actually renaming flags). I know we implemented the new CLI modeling in accordance with the old CLI model — parity is a good goal to shoot for with any refactor — but it would be great to just support both of these, seamlessly and equivalently:
```
$ dbt --no-use-color run --select one_model
$ dbt run --select one_model --no-use-color
```
[CT-1737] Support all "global" flags after all subcommands
**See comment below for updated proposal**

We currently distinguish between two types of flags:
- those which precede the subcommand ("global"), e.g. `--no-use-color`
- those which follow the subcommand, and may have different behavior depending on the subcommand, e.g. `--select`, `--full-refresh`

IMO this is pretty confusing for end users! We should really seek to support flags in any order (and resolve any resulting ambiguities by actually renaming flags). I know we implemented the new CLI modeling in accordance with the old CLI model — parity is a good goal to shoot for with any refactor — but it would be great to just support both of these, seamlessly and equivalently:
```
$ dbt --no-use-color run --select one_model
$ dbt run --select one_model --no-use-color
```


Discussion:
Per BLG on 1/17/23 converting to a spike.. this is a lot of work and has a lot of potential pitfalls.
This issue has been marked as Stale because it has been open for 180 days with no activity. If you would like the issue to remain open, please comment on the issue or else it will be closed in 7 days.
One way to accomplish this goal: [all global/root parameters](https://github.com/dbt-labs/dbt-core/blob/991618dfc152e3316d4900a29c3bc8b44c818664/core/dbt/cli/main.py#L129-L159) should also be defined on every single subcommand, so that you can pass them before or after that subcommand.

Considerations:
- We already have some logic that raises an error if you try to pass it in both places: `dbt --fail-fast run --no-fail-fast`
- We would want to ensure complete coverage going forward: do this programmatically (”global” decorator-of-decorators?), or have a test to ensure that every relevant param is shared across both sets

Without being too clever, I think this could be a reasonable way to go for now.

Then, we should update [our documentation](https://docs.getdbt.com/reference/global-configs/about-global-configs) to encourage that users always pass flags _after_ the subcommand.
From refinement meeting: 
- would we want to deprecate the global parameters in the future? maybe! when we do, we'll add deprecation warnings before doing so
- will need a documentation update
- update old tests, write new tests
- for click, you have to inject the flag in, not as easy to just add extra flags in
@jtcohen6 What's the expected behavior in the following scenarios:
1. A flag is present at global and sub-command levels, but has the same value at each level?
2. A flag is present at global and sub-command levels, but has different values at each level?
@aranke Current state (since v1.5) is that we raise an error in both cases. I think we should preserve that behavior.

https://github.com/dbt-labs/dbt-core/blob/d597b80486ebb409685fafb23d972f5361ddf7fc/core/dbt/cli/flags.py#L97-L105

(Implemented in https://github.com/dbt-labs/dbt-core/pull/7303)
Per BLG on 1/17/23 converting to a spike.. this is a lot of work and has a lot of potential pitfalls.
This issue has been marked as Stale because it has been open for 180 days with no activity. If you would like the issue to remain open, please comment on the issue or else it will be closed in 7 days.
One way to accomplish this goal: [all global/root parameters](https://github.com/dbt-labs/dbt-core/blob/991618dfc152e3316d4900a29c3bc8b44c818664/core/dbt/cli/main.py#L129-L159) should also be defined on every single subcommand, so that you can pass them before or after that subcommand.

Considerations:
- We already have some logic that raises an error if you try to pass it in both places: `dbt --fail-fast run --no-fail-fast`
- We would want to ensure complete coverage going forward: do this programmatically (”global” decorator-of-decorators?), or have a test to ensure that every relevant param is shared across both sets

Without being too clever, I think this could be a reasonable way to go for now.

Then, we should update [our documentation](https://docs.getdbt.com/reference/global-configs/about-global-configs) to encourage that users always pass flags _after_ the subcommand.
From refinement meeting: 
- would we want to deprecate the global parameters in the future? maybe! when we do, we'll add deprecation warnings before doing so
- will need a documentation update
- update old tests, write new tests
- for click, you have to inject the flag in, not as easy to just add extra flags in
@jtcohen6 What's the expected behavior in the following scenarios:
1. A flag is present at global and sub-command levels, but has the same value at each level?
2. A flag is present at global and sub-command levels, but has different values at each level?
@aranke Current state (since v1.5) is that we raise an error in both cases. I think we should preserve that behavior.

https://github.com/dbt-labs/dbt-core/blob/d597b80486ebb409685fafb23d972f5361ddf7fc/core/dbt/cli/flags.py#L97-L105

(Implemented in https://github.com/dbt-labs/dbt-core/pull/7303)

Resolving patch:
diff --git a/core/dbt/cli/main.py b/core/dbt/cli/main.py
--- a/core/dbt/cli/main.py
+++ b/core/dbt/cli/main.py
@@ -1,3 +1,4 @@
+import functools
 from copy import copy
 from dataclasses import dataclass
 from typing import Callable, List, Optional, Union
@@ -118,6 +119,44 @@ def invoke(self, args: List[str], **kwargs) -> dbtRunnerResult:
             )
 
 
+# approach from https://github.com/pallets/click/issues/108#issuecomment-280489786
+def global_flags(func):
+    @p.cache_selected_only
+    @p.debug
+    @p.deprecated_print
+    @p.enable_legacy_logger
+    @p.fail_fast
+    @p.log_cache_events
+    @p.log_file_max_bytes
+    @p.log_format_file
+    @p.log_level
+    @p.log_level_file
+    @p.log_path
+    @p.macro_debugging
+    @p.partial_parse
+    @p.partial_parse_file_path
+    @p.partial_parse_file_diff
+    @p.populate_cache
+    @p.print
+    @p.printer_width
+    @p.quiet
+    @p.record_timing_info
+    @p.send_anonymous_usage_stats
+    @p.single_threaded
+    @p.static_parser
+    @p.use_colors
+    @p.use_colors_file
+    @p.use_experimental_parser
+    @p.version
+    @p.version_check
+    @p.write_json
+    @functools.wraps(func)
+    def wrapper(*args, **kwargs):
+        return func(*args, **kwargs)
+
+    return wrapper
+
+
 # dbt
 @click.group(
     context_settings={"help_option_names": ["-h", "--help"]},
@@ -126,38 +165,10 @@ def invoke(self, args: List[str], **kwargs) -> dbtRunnerResult:
     epilog="Specify one of these sub-commands and you can find more help from there.",
 )
 @click.pass_context
-@p.cache_selected_only
-@p.debug
-@p.deprecated_print
-@p.enable_legacy_logger
-@p.fail_fast
-@p.log_cache_events
-@p.log_file_max_bytes
-@p.log_format
-@p.log_format_file
-@p.log_level
-@p.log_level_file
-@p.log_path
-@p.macro_debugging
-@p.partial_parse
-@p.partial_parse_file_path
-@p.partial_parse_file_diff
-@p.populate_cache
-@p.print
-@p.printer_width
-@p.quiet
-@p.record_timing_info
-@p.send_anonymous_usage_stats
-@p.single_threaded
-@p.static_parser
-@p.use_colors
-@p.use_colors_file
-@p.use_experimental_parser
-@p.version
-@p.version_check
+@global_flags
 @p.warn_error
 @p.warn_error_options
-@p.write_json
+@p.log_format
 def cli(ctx, **kwargs):
     """An ELT tool for managing your SQL transformations and data models.
     For more documentation on these commands, visit: docs.getdbt.com
@@ -167,10 +178,10 @@ def cli(ctx, **kwargs):
 # dbt build
 @cli.command("build")
 @click.pass_context
+@global_flags
 @p.defer
 @p.deprecated_defer
 @p.exclude
-@p.fail_fast
 @p.favor_state
 @p.deprecated_favor_state
 @p.full_refresh
@@ -190,7 +201,6 @@ def cli(ctx, **kwargs):
 @p.target_path
 @p.threads
 @p.vars
-@p.version_check
 @requires.postflight
 @requires.preflight
 @requires.profile
@@ -213,6 +223,7 @@ def build(ctx, **kwargs):
 # dbt clean
 @cli.command("clean")
 @click.pass_context
+@global_flags
 @p.profile
 @p.profiles_dir
 @p.project_dir
@@ -235,6 +246,7 @@ def clean(ctx, **kwargs):
 # dbt docs
 @cli.group()
 @click.pass_context
+@global_flags
 def docs(ctx, **kwargs):
     """Generate or serve the documentation website for your project"""
 
@@ -242,6 +254,7 @@ def docs(ctx, **kwargs):
 # dbt docs generate
 @docs.command("generate")
 @click.pass_context
+@global_flags
 @p.compile_docs
 @p.defer
 @p.deprecated_defer
@@ -261,7 +274,6 @@ def docs(ctx, **kwargs):
 @p.target_path
 @p.threads
 @p.vars
-@p.version_check
 @requires.postflight
 @requires.preflight
 @requires.profile
@@ -284,6 +296,7 @@ def docs_generate(ctx, **kwargs):
 # dbt docs serve
 @docs.command("serve")
 @click.pass_context
+@global_flags
 @p.browser
 @p.port
 @p.profile
@@ -312,6 +325,7 @@ def docs_serve(ctx, **kwargs):
 # dbt compile
 @cli.command("compile")
 @click.pass_context
+@global_flags
 @p.defer
 @p.deprecated_defer
 @p.exclude
@@ -335,7 +349,6 @@ def docs_serve(ctx, **kwargs):
 @p.target_path
 @p.threads
 @p.vars
-@p.version_check
 @requires.postflight
 @requires.preflight
 @requires.profile
@@ -359,6 +372,7 @@ def compile(ctx, **kwargs):
 # dbt show
 @cli.command("show")
 @click.pass_context
+@global_flags
 @p.defer
 @p.deprecated_defer
 @p.exclude
@@ -382,7 +396,6 @@ def compile(ctx, **kwargs):
 @p.target_path
 @p.threads
 @p.vars
-@p.version_check
 @requires.postflight
 @requires.preflight
 @requires.profile
@@ -406,6 +419,7 @@ def show(ctx, **kwargs):
 # dbt debug
 @cli.command("debug")
 @click.pass_context
+@global_flags
 @p.debug_connection
 @p.config_dir
 @p.profile
@@ -413,7 +427,6 @@ def show(ctx, **kwargs):
 @p.project_dir
 @p.target
 @p.vars
-@p.version_check
 @requires.postflight
 @requires.preflight
 def debug(ctx, **kwargs):
@@ -432,6 +445,7 @@ def debug(ctx, **kwargs):
 # dbt deps
 @cli.command("deps")
 @click.pass_context
+@global_flags
 @p.profile
 @p.profiles_dir_exists_false
 @p.project_dir
@@ -452,6 +466,7 @@ def deps(ctx, **kwargs):
 # dbt init
 @cli.command("init")
 @click.pass_context
+@global_flags
 # for backwards compatibility, accept 'project_name' as an optional positional argument
 @click.argument("project_name", required=False)
 @p.profile
@@ -474,6 +489,7 @@ def init(ctx, **kwargs):
 # dbt list
 @cli.command("list")
 @click.pass_context
+@global_flags
 @p.exclude
 @p.indirect_selection
 @p.models
@@ -519,6 +535,7 @@ def list(ctx, **kwargs):
 # dbt parse
 @cli.command("parse")
 @click.pass_context
+@global_flags
 @p.profile
 @p.profiles_dir
 @p.project_dir
@@ -526,7 +543,6 @@ def list(ctx, **kwargs):
 @p.target_path
 @p.threads
 @p.vars
-@p.version_check
 @requires.postflight
 @requires.preflight
 @requires.profile
@@ -543,12 +559,12 @@ def parse(ctx, **kwargs):
 # dbt run
 @cli.command("run")
 @click.pass_context
+@global_flags
 @p.defer
 @p.deprecated_defer
 @p.favor_state
 @p.deprecated_favor_state
 @p.exclude
-@p.fail_fast
 @p.full_refresh
 @p.profile
 @p.profiles_dir
@@ -562,7 +578,6 @@ def parse(ctx, **kwargs):
 @p.target_path
 @p.threads
 @p.vars
-@p.version_check
 @requires.postflight
 @requires.preflight
 @requires.profile
@@ -585,6 +600,7 @@ def run(ctx, **kwargs):
 # dbt retry
 @cli.command("retry")
 @click.pass_context
+@global_flags
 @p.project_dir
 @p.profiles_dir
 @p.vars
@@ -592,7 +608,6 @@ def run(ctx, **kwargs):
 @p.target
 @p.state
 @p.threads
-@p.fail_fast
 @requires.postflight
 @requires.preflight
 @requires.profile
@@ -615,6 +630,7 @@ def retry(ctx, **kwargs):
 # dbt clone
 @cli.command("clone")
 @click.pass_context
+@global_flags
 @p.defer_state
 @p.exclude
 @p.full_refresh
@@ -629,7 +645,6 @@ def retry(ctx, **kwargs):
 @p.target_path
 @p.threads
 @p.vars
-@p.version_check
 @requires.preflight
 @requires.profile
 @requires.project
@@ -652,6 +667,7 @@ def clone(ctx, **kwargs):
 # dbt run operation
 @cli.command("run-operation")
 @click.pass_context
+@global_flags
 @click.argument("macro")
 @p.args
 @p.profile
@@ -683,6 +699,7 @@ def run_operation(ctx, **kwargs):
 # dbt seed
 @cli.command("seed")
 @click.pass_context
+@global_flags
 @p.exclude
 @p.full_refresh
 @p.profile
@@ -698,7 +715,6 @@ def run_operation(ctx, **kwargs):
 @p.target_path
 @p.threads
 @p.vars
-@p.version_check
 @requires.postflight
 @requires.preflight
 @requires.profile
@@ -720,6 +736,7 @@ def seed(ctx, **kwargs):
 # dbt snapshot
 @cli.command("snapshot")
 @click.pass_context
+@global_flags
 @p.defer
 @p.deprecated_defer
 @p.exclude
@@ -759,6 +776,7 @@ def snapshot(ctx, **kwargs):
 # dbt source
 @cli.group()
 @click.pass_context
+@global_flags
 def source(ctx, **kwargs):
     """Manage your project's sources"""
 
@@ -766,6 +784,7 @@ def source(ctx, **kwargs):
 # dbt source freshness
 @source.command("freshness")
 @click.pass_context
+@global_flags
 @p.exclude
 @p.output_path  # TODO: Is this ok to re-use?  We have three different output params, how much can we consolidate?
 @p.profile
@@ -808,10 +827,10 @@ def freshness(ctx, **kwargs):
 # dbt test
 @cli.command("test")
 @click.pass_context
+@global_flags
 @p.defer
 @p.deprecated_defer
 @p.exclude
-@p.fail_fast
 @p.favor_state
 @p.deprecated_favor_state
 @p.indirect_selection
@@ -828,7 +847,6 @@ def freshness(ctx, **kwargs):
 @p.target_path
 @p.threads
 @p.vars
-@p.version_check
 @requires.postflight
 @requires.preflight
 @requires.profile


Test patch:
diff --git a/tests/unit/test_cli_flags.py b/tests/unit/test_cli_flags.py
--- a/tests/unit/test_cli_flags.py
+++ b/tests/unit/test_cli_flags.py
@@ -1,10 +1,10 @@
-import pytest
-
-import click
 from multiprocessing import get_context
 from pathlib import Path
 from typing import List, Optional
 
+import click
+import pytest
+
 from dbt.cli.exceptions import DbtUsageException
 from dbt.cli.flags import Flags
 from dbt.cli.main import cli
@@ -356,6 +356,19 @@ def test_duplicate_flags_raises_error(self):
         with pytest.raises(DbtUsageException):
             Flags(context)
 
+    def test_global_flag_at_child_context(self):
+        parent_context_a = self.make_dbt_context("parent_context_a", ["--no-use-colors"])
+        child_context_a = self.make_dbt_context("child_context_a", ["run"], parent_context_a)
+        flags_a = Flags(child_context_a)
+
+        parent_context_b = self.make_dbt_context("parent_context_b", ["run"])
+        child_context_b = self.make_dbt_context(
+            "child_context_b", ["--no-use-colors"], parent_context_b
+        )
+        flags_b = Flags(child_context_b)
+
+        assert flags_a.USE_COLORS == flags_b.USE_COLORS
+
     def _create_flags_from_dict(self, cmd, d):
         write_file("", "profiles.yml")
         result = Flags.from_dict(cmd, d)