Repository: meltano/meltano

Issue:
bug: env vars in `meltano.yml` cause exception while running meltano install since v2.13.0
### Meltano Version

2.13.0

### Python Version

3.9

### Bug scope

CLI (options, error messages, logging, etc.)

### Operating System

Windows 10 (using WSL2), same issue occurs on Ubuntu 22.04 VM

### Description

[Slack thread](https://app.slack.com/client/TFG99TU9K/C01TCRBBJD7/thread/C01TCRBBJD7-1674066405.617239)

Using Meltano's Docker image I set up env vars in the `meltano.yml` file to utilize dynamically created database connections (using docker compose) cross-projects. This worked like a charm in `v2.12.0`, however since `v2.13.0` I'm receiving an exception during the meltano install step in the Dockerfile.

This exception occurs:
```
invalid literal for int() with base 10: ''
```

For example, the meltano.yml contains the following loader:

```
loaders:
  - name: target-postgres
    variant: transferwise
    pip_url: pipelinewise-target-postgres
    config:
      host: ${DB_HOST}
      port: ${DB_PORT}
      user: ${DB_USER}
      password: ${DB_PASS}
```

And in the docker-compose.yml I set up the env vars:

```
services:
  meltano:
    <<: *meltano-image
    command: ui
    environment:
      DB_HOST: ${DB_HOST}
      DB_PORT: ${DB_PORT}
      DB_USER: ${DB_USER}
      DB_PASS: ${DB_PASS}
```

If I remove the env vars from the meltano.yml, the exception doesn't occur.
Also, if I comment out the `meltano install` command from the `Dockerfile`, run the container and `bash` into it, the exception doesn't occur if I manually execute `meltano install`. In that case the env vars are also passed correctly, which I checked using the `meltano config target-postgres list` command.

The `Dockerfile` for reference:

```
# registry.gitlab.com/meltano/meltano:latest is also available in GitLab Registry
ARG MELTANO_IMAGE=meltano/meltano:latest-python3.9
FROM $MELTANO_IMAGE

RUN apt-get update && apt-get install libpq-dev -y

WORKDIR /project

# Install 
# Install any additional requirements
COPY ./requirements.txt .
RUN pip install -r requirements.txt

# Copy over Meltano project directory
COPY . .

RUN meltano install

# Don't allow changes to containerized project files
# ENV MELTANO_PROJECT_READONLY 1

ENTRYPOINT ["meltano"]
```

### Code

_No response_


Discussion:
[The old behavior when an env var wasn't found within a string setting value while expanding env vars was to return `None`](https://github.com/meltano/meltano/pull/7123/files#diff-00d9669cbb6c978cd7c6d0b0fd4d4b8fd304716f7c39f64c0a9fe5ad7b8e2eb9L510). `re.sub` interprets this as the empty string, but if the whole setting string matches the regex, then instead of calling `re.sub` the substitution function was called directly, so `None` was returned instead of a string. This is... weird. I don't know why it was implemented like this, but it seems like this behaviour was relied on by `src/meltano/core/settings_service.py::SettingsService.get_with_metadata`, since it passes the value returned by `expand_env_vars` to `setting_def.cast_value`, which is a no-op when the value is `None`, but errors if the value is the empty string and its trying to cast it to int.

I'm hesitant to restore the old behaviour because it seems incorrect. A better way to do what it was doing without changing too much is probably to detect whether the setting string was not empty before expansion and is empty after expansion. If that's the case, then we replace the value with `None`. 

Resolving patch:
diff --git a/src/meltano/core/settings_service.py b/src/meltano/core/settings_service.py
--- a/src/meltano/core/settings_service.py
+++ b/src/meltano/core/settings_service.py
@@ -360,6 +360,9 @@ def get_with_metadata(  # noqa: WPS210, WPS615
                 env=expandable_env,
                 if_missing=EnvVarMissingBehavior(strict_env_var_mode),
             )
+            # https://github.com/meltano/meltano/issues/7189#issuecomment-1396112167
+            if value and not expanded_value:  # The whole string was missing env vars
+                expanded_value = None
 
             if expanded_value != value:
                 metadata["expanded"] = True


Test patch:
diff --git a/tests/meltano/core/plugin/test_plugin_settings.py b/tests/meltano/core/plugin/test_plugin_settings.py
--- a/tests/meltano/core/plugin/test_plugin_settings.py
+++ b/tests/meltano/core/plugin/test_plugin_settings.py
@@ -538,7 +538,7 @@ def test_env_var_expansion(
 
         assert config["var"] == "hello world!"
         assert config["foo"] == "42"
-        assert config["missing"] == ""
+        assert config["missing"] is None
         assert config["multiple"] == "rock paper scissors"
         assert config["info"] == "tap-mock"
 
@@ -939,4 +939,4 @@ def test_strict_env_var_mode_off_no_raise_error(self, subject):
             [FEATURE_FLAG_PREFIX, str(FeatureFlags.STRICT_ENV_VAR_MODE)], False
         )
         subject.set("stacked_env_var", "${NONEXISTENT_ENV_VAR}")
-        assert subject.get("stacked_env_var") == ""
+        assert subject.get("stacked_env_var") is None
diff --git a/tests/meltano/core/test_project_settings_service.py b/tests/meltano/core/test_project_settings_service.py
--- a/tests/meltano/core/test_project_settings_service.py
+++ b/tests/meltano/core/test_project_settings_service.py
@@ -188,3 +188,12 @@ def test_meltano_settings_with_active_environment(
         value, source = subject.get_with_source("database_max_retries")
         assert source == SettingValueStore.MELTANO_YML
         assert value == 10000
+
+    def test_fully_missing_env_var_setting_is_none(
+        self, subject: ProjectSettingsService
+    ):
+        subject.set([FEATURE_FLAG_PREFIX, str(FeatureFlags.STRICT_ENV_VAR_MODE)], False)
+        # https://github.com/meltano/meltano/issues/7189#issuecomment-1396112167
+        with pytest.warns(RuntimeWarning, match="Unknown setting 'port'"):
+            subject.set("port", "${UNSET_PORT_ENV_VAR}")
+        assert subject.get("port") is None