Repository: meltano/meltano

Issue:
Unexpected behaviour when handling default values with Environments
### What is the current *bug* behavior?

Meltano does not take Environment inheritance into account when setting a value to its default in an Environment.

### What is the expected *correct* behavior?

Currently, when setting a setting value to it's default, Meltano will `unset` all values ("to fall back to default") rather than adding a new entry with the default value to `meltano.yml`. This behaviour was correct pre-environments, but does not take into account inheritance now that environments exist. The behaviour I would expect is that a new setting value would be added to the specific Environment with the default value, rather than the current behaviour that `unsets` values elsewhere in the inheritance chain.

### Steps to reproduce

```
# add example tap
meltano add extractor tap-gitlab

# configure example setting with non-default value in base
meltano --no-environment config tap-gitlab set fetch_merge_request_commits true

# configure same setting with default value in an Environment
meltano environment=dev config tap-gitlab set fetch_merge_request_commits false
```

The above results in the removal of the `True` setting in base, rather than the addition of a `False` setting value in the environment `dev`.

### Relevant logs and/or screenshots

<img width="1336" alt="Screenshot 2022-06-13 at 15 02 48" src="https://user-images.githubusercontent.com/5585874/173373187-e42fd497-2c58-4172-a0b7-7fa173c77fef.png">

Bug behaviour - removed `True` setting in plugin base configuration and no additional `False` setting in Environment `dev`.
Expected behaviour:

<img width="618" alt="Screenshot 2022-06-13 at 15 03 54" src="https://user-images.githubusercontent.com/5585874/173373791-2c20bdd0-6c21-4682-bf6d-2593e66f7daa.png">

`true` retained in base and `false` added to Environment `dev` (even though it happens to be the default value).

### Possible fixes


### Further regression test
_Ensure we automatically catch similar issues in the future_

- [ ] Write additional adequate test cases and submit test results
- [ ] Test results should be reviewed by a person from the team

/label ~bug
/label ~"To Do"


Discussion:
@kgpayne Does this look related to https://github.com/meltano/meltano/issues/6116? 
> @kgpayne Does this look related to https://github.com/meltano/meltano/issues/6116?

I think it is similar but with a different root cause 🙂 In the case of setting a value to the default, we have a code path that specifically [unsets all available stores](https://github.com/meltano/meltano/blob/dfd9650170ff97fd69fdff7ac8dc7dc0841bf663/src/meltano/core/settings_store.py#L1221) (including the current, but not _all_, environments). This appears to be different to the "choosing the wrong location when setting" unexpected behaviour documented in #6116 🤔
@tayloramurphy I've put this one into the current milestone assuming you agree that fixing config/environments related bugs is a good use of our time right now!

Resolving patch:
diff --git a/src/meltano/core/settings_store.py b/src/meltano/core/settings_store.py
--- a/src/meltano/core/settings_store.py
+++ b/src/meltano/core/settings_store.py
@@ -1280,15 +1280,6 @@ def set(self, name: str, path: list[str], value, setting_def=None) -> dict:
             StoreNotSupportedError: exception encountered when attempting to write to store.
         """
         setting_def = setting_def or self.find_setting(name)
-
-        current_value, metadata = self.get(name, setting_def=setting_def)
-
-        if setting_def:
-            if value == setting_def.value:
-                # Unset everything so we fall down on default
-                self.unset(name, path, setting_def=setting_def)
-                return {"store": SettingValueStore.DEFAULT}
-
         store = self.auto_store(name, setting_def=setting_def)
         logger.debug(f"AutoStoreManager returned store '{store}'")
         if store is None:
@@ -1296,16 +1287,7 @@ def set(self, name: str, path: list[str], value, setting_def=None) -> dict:
 
         # May raise StoreNotSupportedError, but that's good.
         manager = self.manager_for(store)
-
-        # Even if the global current value isn't equal,
-        # the value in this store might be
-        current_value, _ = manager.get(name, setting_def=setting_def)
-        if value == current_value:
-            # No need to do anything
-            return {"store": store}
-
         metadata = manager.set(name, path, value, setting_def=setting_def)
-
         metadata["store"] = store
         return metadata
 


Test patch:
diff --git a/tests/meltano/core/test_settings_store.py b/tests/meltano/core/test_settings_store.py
--- a/tests/meltano/core/test_settings_store.py
+++ b/tests/meltano/core/test_settings_store.py
@@ -13,6 +13,7 @@
     InheritedStoreManager,
     MeltanoEnvStoreManager,
     MeltanoYmlStoreManager,
+    SettingsStoreManager,
     SettingValueStore,
     StoreNotSupportedError,
 )
@@ -283,7 +284,7 @@ def test_get(  # noqa: WPS213
 
     def test_set(
         self,
-        subject,
+        subject: SettingsStoreManager,
         project,
         unsupported,
         set_value_store,
@@ -294,20 +295,11 @@ def test_set(
         def set_value(value):
             return subject.set("regular", ["regular"], value)
 
-        # Returns silently when new value matches current value,
-        # even if source is not writable
+        # Allow setting to a default value
         assert_value_source("from_default", Store.DEFAULT)
         metadata = set_value("from_default")
-        assert metadata["store"] == Store.DEFAULT
-        assert_value_source("from_default", Store.DEFAULT)
-
-        set_value_store("from_db", Store.DB)
-        with mock.patch.object(
-            Store.DB.manager, "set", side_effect=StoreNotSupportedError
-        ):
-            metadata = set_value("from_db")
-            assert metadata["store"] == Store.MELTANO_YML
-            assert_value_source("from_db", Store.MELTANO_YML)
+        assert metadata["store"] == Store.MELTANO_YML
+        assert_value_source("from_default", Store.MELTANO_YML)
 
         # Falls back on `meltano.yml` when current source is not writable
         with unsupported(Store.DB):
@@ -315,17 +307,12 @@ def set_value(value):
             assert metadata["store"] == Store.MELTANO_YML
             assert_value_source("from_meltano_yml", Store.MELTANO_YML)
 
-        # Unsets in all writable stores when new value matches default value
-        metadata = set_value("from_default")
-        assert metadata["store"] == Store.DEFAULT
-        assert_value_source("from_default", Store.DEFAULT)
-
         # Stores in Meltano Environment if active
         with monkeypatch.context() as mpc:
             mpc.setattr(project, "active_environment", environment)
-            metadata = set_value("from_meltano_env_new")
+            metadata = set_value("from_meltano_env")
             assert metadata["store"] == Store.MELTANO_ENV
-            assert_value_source("from_meltano_env_new", Store.MELTANO_ENV)
+            assert_value_source("from_meltano_env", Store.MELTANO_ENV)
 
         # Stores in `meltano.yml` by default
         metadata = set_value("from_meltano_yml")