Repository: dbt-labs/dbt-core

Issue:
[CT-2649] [Feature] Move `--target` and `--profile` command line flags to global config
### Is this your first time submitting a feature request?

- [X] I have read the [expectations for open source contributors](https://docs.getdbt.com/docs/contributing/oss-expectations)
- [X] I have searched the existing issues, and I could not find an existing issue for this feature
- [X] I am requesting a straightforward extension of existing dbt functionality, rather than a Big Idea better suited to a discussion

### Describe the feature

When attempting to dynamically configure a user's profile and target, We should prefer `dbt --target TARGET_NAME` over `dbt run --target TARGET_NAME`, and similarly for `--profile`.

`--profiles-dir` is essentially already a global config with [its exposure as an environment variable](https://docs.getdbt.com/docs/core/connect-data-platform/connection-profiles#2-use-the-dbt_profiles_dir-environment-variable-to-change-the-default-location), so I believe other associated configurations with profiles (e.g. `--target` and `--profile`) should also be a global config.







### Describe alternatives you've considered

To achieve dynamic behavior, environment variables (e.g. `DBT_TARGET` and `DBT_PROFILE`) can be used to configure `--target` and `--profile` by templating these values into `profiles.yml`. 

However: 

- These environment variables are less explicit than configuring CLI flags, and
- This requires the user to use `env_var(...)` for their `profile.yml`. Ideally, this feature should come out of the box.

### Who will this benefit?

_No response_

### Are you interested in contributing this feature?

_No response_

### Anything else?

_No response_


Discussion:
Thanks for reaching out @rexledesma !

Can you help me better understand the things you're trying to unlock that aren't possible with using `--target` and `--profile` CLI arguments like this?
```shell
dbt run --target TARGET_NAME --profile PROFILE_NAME
```

Would these things you're thinking of be primarily aimed at users invoking dbt manually or systems invoking dynamic configurations on behalf of a user?
> Thanks for reaching out @rexledesma !
> 
> Can you help me better understand the things you're trying to unlock that aren't possible with using `--target` and `--profile` CLI arguments like this?
> 
> ```shell
> dbt run --target TARGET_NAME --profile PROFILE_NAME
> ```
> 
> Would these things you're thinking of be primarily aimed at users invoking dbt manually or systems invoking dynamic configurations on behalf of a user?

This is more of just ergonomics managing configuration manually, as well as on behalf of a user. `--target` and `--profile` are both already configured on every single dbt command, so might as well be elevated to global configuration.

There are cases where I am developing my dbt models and the default profile and target are different from the profile + target that I want to use for local development. Having the ability to customize this via env var would be nice, so I can set this one time and not have to think about it. 
I buy it!

The `--target` and `--profile` params are supported by all(?) commands — although they don't actually _do_ anything for `deps`, `clean`, `init` (until https://github.com/dbt-labs/dbt-core/pull/7450), `docs serve`, ... (possibly a few others) ...

I think the change here might be as simple as:
- adding `@p.profile` and `@p.target` to the list of "global" CLI flags defined [here](https://github.com/dbt-labs/dbt-core/blob/de65697ff9e0746e0b7dec39880381decebd3a85/core/dbt/cli/main.py#L129-L157)
- adding env vars (`DBT_PROFILE`, `DBT_TARGET`) to the params [here](https://github.com/dbt-labs/dbt-core/blob/de65697ff9e0746e0b7dec39880381decebd3a85/core/dbt/cli/params.py#L480-L485) and [here](https://github.com/dbt-labs/dbt-core/blob/de65697ff9e0746e0b7dec39880381decebd3a85/core/dbt/cli/params.py#L281-L285)
- Adding a unit test [here](https://github.com/dbt-labs/dbt-core/blob/main/tests/unit/test_cli_flags.py) to prove these flags are supported in the "global" position, in addition to the "subcommand" position. (We already have [a test](https://github.com/dbt-labs/dbt-core/blob/de65697ff9e0746e0b7dec39880381decebd3a85/tests/unit/test_cli_flags.py#L347-L352) to verify that dbt should raise an error if the same flag is passed in both positions.)

Going to mark this one a GFI :)
> The `--target` and `--profile` params are supported by all(?) commands

👍 Yep, looks like it is already included in all commands! See below.

<details>
<summary>Pivot table of dbt commands and supported flags</summary>

|            option            |  # dbt build  |  # dbt clean  |  # dbt compile  |  # dbt debug  |  # dbt deps  |  # dbt docs generate  |  # dbt docs serve  |  # dbt init  |  # dbt list  |  # dbt parse  |  # dbt retry  |  # dbt run  |  # dbt run operation  |  # dbt seed  |  # dbt show  |  # dbt snapshot  |  # dbt source freshness  |  # dbt test  |
|:----------------------------:|:-------------:|:-------------:|:---------------:|:-------------:|:------------:|:---------------------:|:------------------:|:------------:|:------------:|:-------------:|:-------------:|:-----------:|:---------------------:|:------------:|:------------:|:----------------:|:------------------------:|:------------:|
|           @p.args            |               |               |                 |               |              |                       |                    |              |              |               |               |             |          ✅           |              |              |                  |                          |              |
|          @p.browser          |               |               |                 |               |              |                       |         ✅         |              |              |               |               |             |                       |              |              |                  |                          |              |
|       @p.compile_docs        |               |               |                 |               |              |          ✅           |                    |              |              |               |               |             |                       |              |              |                  |                          |              |
|        @p.config_dir         |               |               |                 |      ✅       |              |                       |                    |              |              |               |               |             |                       |              |              |                  |                          |              |
|     @p.debug_connection      |               |               |                 |      ✅       |              |                       |                    |              |              |               |               |             |                       |              |              |                  |                          |              |
|           @p.defer           |      ✅       |               |       ✅        |               |              |          ✅           |                    |              |              |               |               |     ✅      |                       |              |      ✅      |        ✅        |                          |      ✅      |
|        @p.defer_state        |      ✅       |               |       ✅        |               |              |          ✅           |                    |              |      ✅      |               |               |     ✅      |                       |      ✅      |      ✅      |        ✅        |            ✅            |      ✅      |
|     @p.deprecated_defer      |      ✅       |               |       ✅        |               |              |          ✅           |                    |              |              |               |               |     ✅      |                       |              |      ✅      |        ✅        |                          |      ✅      |
|  @p.deprecated_favor_state   |      ✅       |               |       ✅        |               |              |          ✅           |                    |              |              |               |               |     ✅      |                       |              |      ✅      |        ✅        |                          |      ✅      |
|     @p.deprecated_state      |      ✅       |               |       ✅        |               |              |          ✅           |                    |              |      ✅      |               |               |     ✅      |                       |      ✅      |      ✅      |        ✅        |            ✅            |      ✅      |
|       @p.empty_catalog       |               |               |                 |               |              |          ✅           |                    |              |              |               |               |             |                       |              |              |                  |                          |              |
|          @p.exclude          |      ✅       |               |       ✅        |               |              |          ✅           |                    |              |      ✅      |               |               |     ✅      |                       |      ✅      |      ✅      |        ✅        |            ✅            |      ✅      |
|         @p.fail_fast         |      ✅       |               |                 |               |              |                       |                    |              |              |               |      ✅       |     ✅      |                       |              |              |                  |                          |      ✅      |
|        @p.favor_state        |      ✅       |               |       ✅        |               |              |          ✅           |                    |              |              |               |               |     ✅      |                       |              |      ✅      |        ✅        |                          |      ✅      |
|       @p.full_refresh        |      ✅       |               |       ✅        |               |              |                       |                    |              |              |               |               |     ✅      |                       |      ✅      |      ✅      |                  |                          |              |
|    @p.indirect_selection     |      ✅       |               |       ✅        |               |              |                       |                    |              |      ✅      |               |               |             |                       |              |      ✅      |                  |                          |      ✅      |
|          @p.inline           |               |               |       ✅        |               |              |                       |                    |              |              |               |               |             |                       |              |      ✅      |                  |                          |              |
|        @p.introspect         |               |               |       ✅        |               |              |                       |                    |              |              |               |               |             |                       |              |      ✅      |                  |                          |              |
|          @p.models           |               |               |                 |               |              |                       |                    |              |      ✅      |               |               |             |                       |              |              |                  |                          |              |
|          @p.output           |               |               |                 |               |              |                       |                    |              |      ✅      |               |               |             |                       |              |              |                  |                          |              |
|        @p.output_keys        |               |               |                 |               |              |                       |                    |              |      ✅      |               |               |             |                       |              |              |                  |                          |              |
|        @p.output_path        |               |               |                 |               |              |                       |                    |              |              |               |               |             |                       |              |              |                  |            ✅            |              |
|           @p.port            |               |               |                 |               |              |                       |         ✅         |              |              |               |               |             |                       |              |              |                  |                          |              |
|          @p.profile          |      ✅       |      ✅       |       ✅        |      ✅       |      ✅      |          ✅           |         ✅         |      ✅      |      ✅      |      ✅       |      ✅       |     ✅      |          ✅           |      ✅      |      ✅      |        ✅        |            ✅            |      ✅      |
|       @p.profiles_dir        |      ✅       |      ✅       |       ✅        |               |              |          ✅           |         ✅         |              |      ✅      |      ✅       |      ✅       |     ✅      |          ✅           |      ✅      |      ✅      |        ✅        |            ✅            |      ✅      |
| @p.profiles_dir_exists_false |               |               |                 |      ✅       |      ✅      |                       |                    |      ✅      |              |               |               |             |                       |              |              |                  |                          |              |
|        @p.project_dir        |      ✅       |      ✅       |       ✅        |      ✅       |      ✅      |          ✅           |         ✅         |      ✅      |      ✅      |      ✅       |      ✅       |     ✅      |          ✅           |      ✅      |      ✅      |        ✅        |            ✅            |      ✅      |
|        @p.raw_select         |               |               |                 |               |              |                       |                    |              |      ✅      |               |               |             |                       |              |              |                  |                          |              |
|       @p.resource_type       |      ✅       |               |                 |               |              |                       |                    |              |      ✅      |               |               |             |                       |              |              |                  |                          |              |
|          @p.select           |      ✅       |               |       ✅        |               |              |          ✅           |                    |              |              |               |               |     ✅      |                       |      ✅      |      ✅      |        ✅        |            ✅            |      ✅      |
|         @p.selector          |      ✅       |               |       ✅        |               |              |          ✅           |                    |              |      ✅      |               |               |     ✅      |                       |      ✅      |      ✅      |        ✅        |            ✅            |      ✅      |
|           @p.show            |      ✅       |               |                 |               |              |                       |                    |              |              |               |               |             |                       |      ✅      |              |                  |                          |              |
|        @p.show_limit         |               |               |                 |               |              |                       |                    |              |              |               |               |             |                       |              |      ✅      |                  |                          |              |
|    @p.show_output_format     |               |               |       ✅        |               |              |                       |                    |              |              |               |               |             |                       |              |      ✅      |                  |                          |              |
|    @p.skip_profile_setup     |               |               |                 |               |              |                       |                    |      ✅      |              |               |               |             |                       |              |              |                  |                          |              |
|           @p.state           |      ✅       |               |       ✅        |               |              |          ✅           |                    |              |      ✅      |               |      ✅       |     ✅      |                       |      ✅      |      ✅      |        ✅        |            ✅            |      ✅      |
|      @p.store_failures       |      ✅       |               |                 |               |              |                       |                    |              |              |               |               |             |                       |              |              |                  |                          |      ✅      |
|          @p.target           |      ✅       |      ✅       |       ✅        |      ✅       |      ✅      |          ✅           |         ✅         |      ✅      |      ✅      |      ✅       |      ✅       |     ✅      |          ✅           |      ✅      |      ✅      |        ✅        |            ✅            |      ✅      |
|        @p.target_path        |      ✅       |      ✅       |       ✅        |               |              |          ✅           |         ✅         |              |      ✅      |      ✅       |               |     ✅      |          ✅           |      ✅      |      ✅      |        ✅        |            ✅            |      ✅      |
|          @p.threads          |      ✅       |               |       ✅        |               |              |          ✅           |                    |              |              |      ✅       |      ✅       |     ✅      |          ✅           |      ✅      |      ✅      |        ✅        |            ✅            |      ✅      |
|           @p.vars            |      ✅       |      ✅       |       ✅        |      ✅       |      ✅      |          ✅           |         ✅         |      ✅      |      ✅      |      ✅       |      ✅       |     ✅      |          ✅           |      ✅      |      ✅      |        ✅        |            ✅            |      ✅      |
|       @p.version_check       |      ✅       |               |       ✅        |      ✅       |              |          ✅           |                    |              |              |      ✅       |               |     ✅      |                       |      ✅      |      ✅      |                  |                          |      ✅      |

</details>


<details>
<summary>Source code for pivot table</summary>

`dbt_subcommand_flags.py`

```python
import re
import pandas as pd
import numpy as np


def find_instances(text):
    # Matches any string of format '# dbt subcommand'
    subcommand_pattern = r"# dbt .+"

    # Matches any string of format '@p.cli_option'
    option_pattern = r"@p\.\w+"

    # Store results in a list of dictionaries
    result_list = []
    last_subcommand_match = None

    for line in text.splitlines():
        subcommand_match = re.search(subcommand_pattern, line)
        if subcommand_match:
            last_subcommand_match = subcommand_match.group()

        option_match = re.search(option_pattern, line)
        if option_match and last_subcommand_match:
            result_list.append(
                {"subcommand": last_subcommand_match, "option": option_match.group()}
            )

    return pd.DataFrame(result_list)


def read_file(file_path):
    with open(file_path, "r") as file:
        text = file.read()
    return text


def main():
    file_path = "main.py"
    text = read_file(file_path)
    result_df = find_instances(text)

    # Represents the presence of a pair with the value 1
    result_df["count"] = 1

    # Create a pivot table
    table = pd.pivot_table(
        result_df,
        values="count",
        index=["option"],
        columns=["subcommand"],
        aggfunc=np.sum,
        fill_value=0,
    )

    # Convert 1's and 0's to applicable icons
    table.replace(to_replace=1, value="✅", inplace=True)
    table.replace(to_replace=0, value="", inplace=True)

    # Export to Markdown
    print(table.to_markdown(stralign="center"))


if __name__ == "__main__":
    main()
```

1. Download latest version of [`core/dbt/cli/main.py`](https://github.com/dbt-labs/dbt-core/blob/main/core/dbt/cli/main.py) to your current working directory
2. pip install pandas and numpy if needed
3. `python dbt_subcommand_flags.py`

</details>

Resolving patch:
diff --git a/core/dbt/cli/main.py b/core/dbt/cli/main.py
--- a/core/dbt/cli/main.py
+++ b/core/dbt/cli/main.py
@@ -127,12 +127,14 @@ def global_flags(func):
     @p.populate_cache
     @p.print
     @p.printer_width
+    @p.profile
     @p.quiet
     @p.record_timing_info
     @p.send_anonymous_usage_stats
     @p.single_threaded
     @p.state
     @p.static_parser
+    @p.target
     @p.use_colors
     @p.use_colors_file
     @p.use_experimental_parser
@@ -172,7 +174,6 @@ def cli(ctx, **kwargs):
 @p.export_saved_queries
 @p.full_refresh
 @p.deprecated_include_saved_query
-@p.profile
 @p.profiles_dir
 @p.project_dir
 @p.resource_type
@@ -181,7 +182,6 @@ def cli(ctx, **kwargs):
 @p.selector
 @p.show
 @p.store_failures
-@p.target
 @p.target_path
 @p.threads
 @p.vars
@@ -211,10 +211,8 @@ def build(ctx, **kwargs):
 @click.pass_context
 @global_flags
 @p.clean_project_files_only
-@p.profile
 @p.profiles_dir
 @p.project_dir
-@p.target
 @p.target_path
 @p.vars
 @requires.postflight
@@ -246,14 +244,12 @@ def docs(ctx, **kwargs):
 @global_flags
 @p.compile_docs
 @p.exclude
-@p.profile
 @p.profiles_dir
 @p.project_dir
 @p.select
 @p.selector
 @p.empty_catalog
 @p.static
-@p.target
 @p.target_path
 @p.threads
 @p.vars
@@ -284,10 +280,8 @@ def docs_generate(ctx, **kwargs):
 @global_flags
 @p.browser
 @p.port
-@p.profile
 @p.profiles_dir
 @p.project_dir
-@p.target
 @p.target_path
 @p.vars
 @requires.postflight
@@ -317,7 +311,6 @@ def docs_serve(ctx, **kwargs):
 @p.full_refresh
 @p.show_output_format
 @p.introspect
-@p.profile
 @p.profiles_dir
 @p.project_dir
 @p.empty
@@ -325,7 +318,6 @@ def docs_serve(ctx, **kwargs):
 @p.selector
 @p.inline
 @p.compile_inject_ephemeral_ctes
-@p.target
 @p.target_path
 @p.threads
 @p.vars
@@ -360,13 +352,11 @@ def compile(ctx, **kwargs):
 @p.show_output_format
 @p.show_limit
 @p.introspect
-@p.profile
 @p.profiles_dir
 @p.project_dir
 @p.select
 @p.selector
 @p.inline
-@p.target
 @p.target_path
 @p.threads
 @p.vars
@@ -398,10 +388,8 @@ def show(ctx, **kwargs):
 @global_flags
 @p.debug_connection
 @p.config_dir
-@p.profile
 @p.profiles_dir_exists_false
 @p.project_dir
-@p.target
 @p.vars
 @requires.postflight
 @requires.preflight
@@ -423,10 +411,8 @@ def debug(ctx, **kwargs):
 @cli.command("deps")
 @click.pass_context
 @global_flags
-@p.profile
 @p.profiles_dir_exists_false
 @p.project_dir
-@p.target
 @p.vars
 @p.source
 @p.lock
@@ -468,11 +454,9 @@ def deps(ctx, **kwargs):
 @global_flags
 # for backwards compatibility, accept 'project_name' as an optional positional argument
 @click.argument("project_name", required=False)
-@p.profile
 @p.profiles_dir_exists_false
 @p.project_dir
 @p.skip_profile_setup
-@p.target
 @p.vars
 @requires.postflight
 @requires.preflight
@@ -495,14 +479,12 @@ def init(ctx, **kwargs):
 @p.models
 @p.output
 @p.output_keys
-@p.profile
 @p.profiles_dir
 @p.project_dir
 @p.resource_type
 @p.exclude_resource_type
 @p.raw_select
 @p.selector
-@p.target
 @p.target_path
 @p.vars
 @requires.postflight
@@ -536,10 +518,8 @@ def list(ctx, **kwargs):
 @cli.command("parse")
 @click.pass_context
 @global_flags
-@p.profile
 @p.profiles_dir
 @p.project_dir
-@p.target
 @p.target_path
 @p.threads
 @p.vars
@@ -561,13 +541,11 @@ def parse(ctx, **kwargs):
 @global_flags
 @p.exclude
 @p.full_refresh
-@p.profile
 @p.profiles_dir
 @p.project_dir
 @p.empty
 @p.select
 @p.selector
-@p.target
 @p.target_path
 @p.threads
 @p.vars
@@ -599,8 +577,6 @@ def run(ctx, **kwargs):
 @p.project_dir
 @p.profiles_dir
 @p.vars
-@p.profile
-@p.target
 @p.target_path
 @p.threads
 @p.full_refresh
@@ -630,14 +606,12 @@ def retry(ctx, **kwargs):
 @global_flags
 @p.exclude
 @p.full_refresh
-@p.profile
 @p.profiles_dir
 @p.project_dir
 @p.resource_type
 @p.exclude_resource_type
 @p.select
 @p.selector
-@p.target
 @p.target_path
 @p.threads
 @p.vars
@@ -668,10 +642,8 @@ def clone(ctx, **kwargs):
 @global_flags
 @click.argument("macro")
 @p.args
-@p.profile
 @p.profiles_dir
 @p.project_dir
-@p.target
 @p.target_path
 @p.threads
 @p.vars
@@ -702,13 +674,11 @@ def run_operation(ctx, **kwargs):
 @global_flags
 @p.exclude
 @p.full_refresh
-@p.profile
 @p.profiles_dir
 @p.project_dir
 @p.select
 @p.selector
 @p.show
-@p.target
 @p.target_path
 @p.threads
 @p.vars
@@ -737,12 +707,10 @@ def seed(ctx, **kwargs):
 @click.pass_context
 @global_flags
 @p.exclude
-@p.profile
 @p.profiles_dir
 @p.project_dir
 @p.select
 @p.selector
-@p.target
 @p.target_path
 @p.threads
 @p.vars
@@ -781,12 +749,10 @@ def source(ctx, **kwargs):
 @global_flags
 @p.exclude
 @p.output_path  # TODO: Is this ok to re-use?  We have three different output params, how much can we consolidate?
-@p.profile
 @p.profiles_dir
 @p.project_dir
 @p.select
 @p.selector
-@p.target
 @p.target_path
 @p.threads
 @p.vars
@@ -822,13 +788,11 @@ def freshness(ctx, **kwargs):
 @click.pass_context
 @global_flags
 @p.exclude
-@p.profile
 @p.profiles_dir
 @p.project_dir
 @p.select
 @p.selector
 @p.store_failures
-@p.target
 @p.target_path
 @p.threads
 @p.vars
diff --git a/core/dbt/cli/params.py b/core/dbt/cli/params.py
--- a/core/dbt/cli/params.py
+++ b/core/dbt/cli/params.py
@@ -342,7 +342,7 @@
 
 profile = click.option(
     "--profile",
-    envvar=None,
+    envvar="DBT_PROFILE",
     help="Which existing profile to load. Overrides setting in dbt_project.yml.",
 )
 
@@ -603,7 +603,7 @@
 target = click.option(
     "--target",
     "-t",
-    envvar=None,
+    envvar="DBT_TARGET",
     help="Which target to load for the given profile",
 )
 


Test patch: