Repository: meltano/meltano

Issue:
bug:  Conflicting behavior flags in `meltano config set`
### Meltano Version

latest

### Python Version

NA

### Bug scope

CLI (options, error messages, logging, etc.)

### Operating System

any

### Description

From Slack: https://meltano.slack.com/archives/C01TCRBBJD7/p1659032672217559?thread_ts=1658965446.758529&cid=C01TCRBBJD7

With the example invocation:

```console
meltano --environment=prod config target-csv set destination_path ./output
```

The expectation is that meltano would store the value within the environment config. Instead, the value is stored at the top-level meltano config.

## Workaround

The correct behavior can be achieved by also overriding `--store` as in:

```
meltano --environment=prod config target-csv set destination_path ./output --store=meltano_environment
```

## Proposed solution

The default behavior (without specifying `--store` or by using `--store=auto`) should be updated to prefer `meltano_environment` over `meltano_yml` if `--environment` is set to a valid environment name.

## Alternative Approaches

Another option is to remove or deprecate `--store=meltano_environment` in deference to the `--environment` flag being specified or not.

In that case `--store=meltano_yml` would indicate _either_ the top-level yaml declarations (if no environment is set) _or_ the environment-specific section of the yaml (if an environment is set).

Note:

- There are probably good reasons why this alternative wouldn't work, for instance if internally we need to treat these as two different stores and due to the fact that internally we would step through each layer with the top-level and environment-level yaml each being a different important part of that resolution.

### Code

_No response_


Discussion:
> Another option is to remove or deprecate --store=meltano_environment in deference to the --environment flag being specified or not.

@aaronsteers we don't mention `meltano_enviornment` as a store option anywhere in the docs. And it's fairly confusing behavior with the --environment flag imo. I'd be in favor of just getting rid of it.

Either way, this is definitely a bug and should be fixed!
@aaronsteers @tayloramurphy -- just to confirm, the scope here would be:
1. `--store=meltano_environment` will now raise the same `Invalid value for --store:` error that other invalid/nonexistent `--store` arguments raise
1. If an `--environment` flag is provided, the value will be set in that environment's config
1. If no `--environment` flag is provided, the value will be set in the top-level of `meltano.yml` **NOT** in the default environment

Does that sound right? I particularly want to confirm point (3) since that might not be completely intuitive.

Resolving patch:
diff --git a/src/meltano/cli/config.py b/src/meltano/cli/config.py
--- a/src/meltano/cli/config.py
+++ b/src/meltano/cli/config.py
@@ -6,7 +6,9 @@
 import json
 import logging
 import tempfile
+from functools import wraps
 from pathlib import Path
+from typing import Any
 
 import click
 import dotenv
@@ -31,6 +33,65 @@
 logger = logging.getLogger(__name__)
 
 
+def _get_ctx_arg(*args: Any) -> click.core.Context:
+    """Get the click.core.Context arg from a set of args.
+
+    Args:
+        args: the args to get Context from
+
+    Returns:
+        The click.core.Context arg.
+
+    Raises:
+        ValueError: if there is no click.core.Context in the given args.
+    """
+    for arg in args:
+        if isinstance(arg, click.core.Context):
+            return arg
+    raise ValueError("No click.core.Context provided in *args")
+
+
+def _get_store_choices() -> list[SettingValueStore]:
+    """Get a list of valid choices for the --store flag.
+
+    Returns:
+        SettingValueStore.writables(), without meltano_env
+    """
+    writables = SettingValueStore.writables()
+    writables.remove(SettingValueStore.MELTANO_ENV)
+    return writables
+
+
+def _use_meltano_env(func):
+    """Override the 'meltano_yml' choice for a config command's 'store' argument.
+
+    If an --environment flag is passed, the decorated command will use
+    the MELTANO_ENV store instead of MELTANO_YML but _will not_ use MELTANO_ENV
+    store if the active environment is set via the default environment.
+
+    Args:
+       func: the command to override
+
+    Returns:
+       A wrapped function with overridden store argument
+    """
+
+    @wraps(func)
+    def _wrapper(*args, **kwargs):
+        store = kwargs.pop("store")
+        if store not in {SettingValueStore.MELTANO_YML, SettingValueStore.MELTANO_ENV}:
+            return func(*args, **kwargs, store=store)
+        ctx = _get_ctx_arg(*args)
+        store = (
+            SettingValueStore.MELTANO_YML
+            if ctx.obj["is_default_environment"]
+            else SettingValueStore.MELTANO_ENV
+        )
+        return func(*args, **kwargs, store=store)
+
+    return _wrapper
+
+
 def get_label(metadata) -> str:
     """Get the label for an environment variable's source.
 
@@ -149,6 +210,7 @@ def config(  # noqa: WPS231
         session.close()
 
 
+@_use_meltano_env
 @config.command(
     cls=PartialInstrumentedCmd,
     name="list",
@@ -242,10 +304,11 @@ def list_settings(ctx, extras: bool):
 @config.command(cls=PartialInstrumentedCmd)
 @click.option(
     "--store",
-    type=click.Choice(SettingValueStore.writables()),
+    type=click.Choice(_get_store_choices()),
     default=SettingValueStore.AUTO,
 )
 @click.pass_context
+@_use_meltano_env
 def reset(ctx, store):
     """Clear the configuration (back to defaults)."""
     store = SettingValueStore(store)
@@ -276,10 +339,11 @@ def reset(ctx, store):
 @click.argument("value", required=False)
 @click.option(
     "--store",
-    type=click.Choice(SettingValueStore.writables()),
+    type=click.Choice(_get_store_choices()),
     default=SettingValueStore.AUTO,
 )
 @click.pass_context
+@_use_meltano_env
 def set_(ctx, setting_name, value, store, interactive):
     """Set the configurations' setting `<name>` to `<value>`."""
     interaction = InteractiveConfig(ctx=ctx, store=store, extras=False)
@@ -324,10 +388,11 @@ async def _validate():  # noqa: WPS430
 @click.argument("setting_name", nargs=-1, required=True)
 @click.option(
     "--store",
-    type=click.Choice(SettingValueStore.writables()),
+    type=click.Choice(_get_store_choices()),
     default=SettingValueStore.AUTO,
 )
 @click.pass_context
+@_use_meltano_env
 def unset(ctx, setting_name, store):
     """Unset the configurations' setting called `<name>`."""
     store = SettingValueStore(store)


Test patch:
diff --git a/tests/meltano/cli/test_config.py b/tests/meltano/cli/test_config.py
--- a/tests/meltano/cli/test_config.py
+++ b/tests/meltano/cli/test_config.py
@@ -146,6 +146,22 @@ def test_environments_order_of_precedence(
             )
             assert base_tap_config["config"]["test"] == "base-mock"
 
+            # set base config in `meltano.yml` -- ignore default environment
+            result = cli_runner.invoke(
+                cli,
+                ["config", "tap-mock", "set", "test", "base-mock-no-default"],
+            )
+            assert_cli_runner(result)
+            base_tap_config = next(
+                (
+                    tap
+                    for tap in project.meltano["plugins"]["extractors"]
+                    if tap["name"] == "tap-mock"
+                ),
+                {},
+            )
+            assert base_tap_config["config"]["test"] == "base-mock-no-default"
+
             # set dev environment config in `meltano.yml`
             result = cli_runner.invoke(
                 cli,