Repository: meltano/meltano

Issue:
Bug: Env var strict mode not working as expected
While [testing](https://github.com/meltano/squared/pull/312#issuecomment-1191828551) within the `squared` project, and also confirmed by @visch [in Slack](https://meltano.slack.com/archives/CFG3C3C66/p1660078390407119?thread_ts=1659968745.750009&cid=CFG3C3C66), we've not been seeing the strict mode flag working as expected to raise errors.

It could be the case, in the multiple recent changes to this part of the codebase (`env` resolution, specifically) that we may have had a regression. 




Discussion:
@aaronsteers @pnadolny13 -- [I think this is the source of the issue](https://github.com/meltano/meltano/blob/e4cf960955befe90c66218d50a1d73f3572bc08f/src/meltano/core/utils/__init__.py#L469), basically an exception isn't raised for the case where the string being expanded is a full match rather than a partial substitution. I can get a PR up this afternoon with a fix and a new unit test for this case.
@cjohnhanson - Great - thanks for looking into this! I've assigned to you in next iteration but also okay if you have time to pick up this week.

> [I think this is the source of the issue](https://github.com/meltano/meltano/blob/e4cf960955befe90c66218d50a1d73f3572bc08f/src/meltano/core/utils/__init__.py#L469), basically an exception isn't raised for the case where the string being expanded is a full match rather than a partial substitution. I can get a PR up this afternoon with a fix and a new unit test for this case.

^ I was incorrect there, after spending most of yesterday diving into this, I discovered that the issue is more complex than that. 

NOTE: this is a hypothesis, I haven't fully confirmed this yet.

The `EnvironmentVariableNotFound` error is raised [in the `expand_env_vars` utility function](https://github.com/meltano/meltano/blob/f0dd8f935a092212ecce911eb4c3492b4d5bd351/src/meltano/core/utils/__init__.py#L463). The function doesn't have a `SettingsService` in its scope, so it relies on its caller to pass a `raise_if_missing` argument. It also relies on its caller to pass a dict to use for expanding environment variables. Basically it just does a regex replace, there's not much complex or context-dependent logic (which is as it should be--it's a utility function).

When we implemented [environment variable inheritance across configuration layers](https://github.com/meltano/meltano/pull/6563), we put the new expansion logic/precedence order into [`PluginInvoker.env`](https://github.com/meltano/meltano/blob/f0dd8f935a092212ecce911eb4c3492b4d5bd351/src/meltano/core/plugin_invoker.py#L330).


I _think_ what could be happening is that in the new expansion logic, environment variables for settings are being set to empty strings as a default somewhere in the expansion order. I.e., if we expand environments variables in the active environment's plugin definition using the environment from the root plugin definition, I think the dict we're using as the expansion source would have all the environments variables that are mapped from settings set to the default, probably an empty string. 

If that's the case, then the scope of this issue would be:
- Add unit tests in `test_environment_variables.py` to test for the cases Pat found.
- Raise an `EnvironmentNotFoundError` [here](https://github.com/meltano/meltano/blob/f0dd8f935a092212ecce911eb4c3492b4d5bd351/src/meltano/core/utils/__init__.py#L458) so that empty strings are considered exceptions under `env_var_strict_mode`


Resolving patch:
diff --git a/src/meltano/core/plugin_invoker.py b/src/meltano/core/plugin_invoker.py
--- a/src/meltano/core/plugin_invoker.py
+++ b/src/meltano/core/plugin_invoker.py
@@ -327,7 +327,7 @@ def env(self):
         project_settings_service = ProjectSettingsService(
             self.project, config_service=self.plugins_service.config_service
         )
-        with self.settings_service.feature_flag(
+        with project_settings_service.feature_flag(
             FeatureFlags.STRICT_ENV_VAR_MODE, raise_error=False
         ) as strict_env_var_mode:
 
diff --git a/src/meltano/core/utils/__init__.py b/src/meltano/core/utils/__init__.py
--- a/src/meltano/core/utils/__init__.py
+++ b/src/meltano/core/utils/__init__.py
@@ -456,7 +456,8 @@ def subst(match) -> str:
 
             if not val:
                 logger.debug(f"Variable '${var}' is empty.")
-
+                if raise_if_missing:
+                    raise EnvironmentVariableNotSetError(var)
             return val
         except KeyError as e:
             if raise_if_missing:


Test patch:
diff --git a/tests/meltano/core/test_environment_variables.py b/tests/meltano/core/test_environment_variables.py
--- a/tests/meltano/core/test_environment_variables.py
+++ b/tests/meltano/core/test_environment_variables.py
@@ -7,6 +7,9 @@
 
 from asserts import assert_cli_runner
 from meltano.cli import cli
+from meltano.core.project_settings_service import ProjectSettingsService
+from meltano.core.settings_service import FEATURE_FLAG_PREFIX, FeatureFlags
+from meltano.core.utils import EnvironmentVariableNotSetError
 
 
 class EnvVarResolutionExpectation(NamedTuple):
@@ -343,3 +346,43 @@ def test_environment_variable_inheritance_meltano_env_only(
     )
     assert_cli_runner(result)
     assert result.stdout.strip() == "STACKED=12"
+
+
+def test_strict_env_var_mode_raises_full_replace(cli_runner, project):
+    project_settings_service = ProjectSettingsService(project)
+    project_settings_service.set(
+        [FEATURE_FLAG_PREFIX, str(FeatureFlags.STRICT_ENV_VAR_MODE)], True
+    )
+    with project.meltano_update() as meltanofile:
+        meltanofile.update(_meltanofile_update_dict())
+        meltanofile.update({"env": {"some_var": "$NONEXISTENT"}})
+    result = cli_runner.invoke(
+        cli,
+        [
+            "invoke",
+            "--print-var",
+            "STACKED",
+            "test-env-var-resolution",
+        ],
+    )
+    assert isinstance(result.exception, EnvironmentVariableNotSetError)
+
+
+def test_strict_env_var_mode_raises_partial_replace(cli_runner, project):
+    project_settings_service = ProjectSettingsService(project)
+    project_settings_service.set(
+        [FEATURE_FLAG_PREFIX, str(FeatureFlags.STRICT_ENV_VAR_MODE)], True
+    )
+    with project.meltano_update() as meltanofile:
+        meltanofile.update(_meltanofile_update_dict())
+        meltanofile.update({"env": {"some_var": "some_${NONEXISTENT}_value"}})
+    result = cli_runner.invoke(
+        cli,
+        [
+            "invoke",
+            "--print-var",
+            "STACKED",
+            "test-env-var-resolution",
+        ],
+    )
+    assert isinstance(result.exception, EnvironmentVariableNotSetError)