Repository: PrefectHQ/prefect

Issue:
Newlines in JSON value from environment variable fails to convert to Config / Can't parse Keyfile as GCP_CREDENTIALS
## Description
<!-- A clear description of the bug -->
I want to run Prefect on premise, and need to provide the GCP credentials as environment variable. The format of GCP contains newlines in the `private_key` value, which somehow Prefect fails to parse. Instead the returned type is `str` instead of an `configuration.Config`.

Of course, this breaks somewhere downstream. Specifically, when it is trying to load the keyfile: https://github.com/PrefectHQ/prefect/blob/master/src/prefect/utilities/gcp.py#L29 The GCP library expects a dict-like, and not a string. 


## Expected Behavior
<!-- What did you expect to happen instead? -->

That it parses newlines without problems.



## Reproduction
<!-- A minimal example that exhibits the behavior. -->

I constructed a minimal reproducible script. I also demo that if it was parsed directly, Python can construct the expected `dict`, displaying that it's not a fault with passing in the environment variable.
```python
import prefect
import os
import json

print('WITH_NEWLINE_IN_SCHEMA')
env_newline_in_schema = json.loads(os.environ['PREFECT__CONTEXT__SECRETS__WITH_NEWLINE_IN_SCHEMA'])
print('env:   ', type(env_newline_in_schema))
print('env:   ', env_newline_in_schema)

secret_newline_in_schema = prefect.context.get('secrets', {}).get("WITH_NEWLINE_IN_SCHEMA")
print('secret:', type(secret_newline_in_schema))
print('secret:', secret_newline_in_schema)


print()
print('WITH_NEWLINE_IN_VALUE')

env_newline_in_value = json.loads(os.environ['PREFECT__CONTEXT__SECRETS__WITH_NEWLINE_IN_VALUE'])
print('env:   ', type(env_newline_in_value))
print('env:   ', env_newline_in_value)

secret_newline_in_value = prefect.context.get("secrets", {}).get("WITH_NEWLINE_IN_VALUE")
print('secret:', type(secret_newline_in_value))
print('secret:', secret_newline_in_value)
```

Ran with:
```bash
PREFECT__CONTEXT__SECRETS__WITH_NEWLINE_IN_SCHEMA=$'{"a": "1"\n, "text": "foo bar"}' \
   PREFECT__CONTEXT__SECRETS__WITH_NEWLINE_IN_VALUE=$'{"a": "1", "text": "foo\\nbar"}' \
   python flows/test.py
```

Output:
```plain
WITH_NEWLINE_IN_SCHEMA
env:    <class 'dict'>
env:    {'a': '1', 'text': 'foo bar'}
secret: <class 'prefect.configuration.Config'>
secret: {'a': '1', 'text': 'foo bar'}

WITH_NEWLINE_IN_VALUE
env:    <class 'dict'>
env:    {'a': '1', 'text': 'foo\nbar'}
secret: <class 'str'>
secret: {"a": "1", "text": "foo
bar"}
```


## Environment
<!-- Any additional information about your environment

Optionally run `prefect diagnostics` from the command line and paste the information here. -->
```bash
{
  "config_overrides": {},
  "env_vars": [],
  "system_information": {
    "platform": "macOS-10.15.7-x86_64-i386-64bit",
    "prefect_backend": "server",
    "prefect_version": "0.13.13",
    "python_version": "3.8.6"
  }
}
```

I'd be willing to dive into this and open up a PR if I can find it, any pointers are welcome!


Discussion:
If I double escape the backslashes, it works: 
```
PREFECT__CONTEXT__SECRETS__WITH_NEWLINE_IN_SCHEMA=$'{"a": "1"\n, "text": "foo bar"}' \                                                                                                                                                                                                                                  
   PREFECT__CONTEXT__SECRETS__WITH_NEWLINE_IN_VALUE=$'{"a": "1", "text": "foo\\\\nbar"}' \
   python flows/test.py

WITH_NEWLINE_IN_SCHEMA
env:    <class 'dict'>
env:    {'a': '1', 'text': 'foo bar'}
secret: <class 'prefect.configuration.Config'>
secret: {'a': '1', 'text': 'foo bar'}

WITH_NEWLINE_IN_VALUE
env:    <class 'dict'>
env:    {'a': '1', 'text': 'foo\\nbar'}
secret: <class 'prefect.configuration.Config'>
secret: {'a': '1', 'text': 'foo\nbar'}
```

But that doesn't look optimal to me, is this the intended usage? Because, then I specifically have to modify the keyfile for Prefect, while for other applications the original works.

--

This is caused by https://github.com/PrefectHQ/prefect/blob/master/src/prefect/configuration.py#L209-L212, if I comment out the cast, the original usage works as expected.
Hi @joelluijmes looking through the code I don't have a good reason as to why this line can't be removed. IIRC a long time ago there was an issue where the newlines were getting stripped out of the env vars or something along those lines. Maybe instead of that line in the configuration code we should instead do something that loads the literal string and collapses `\\` into `\`. Not fully sure what the implications of this is but I think we could attempt the change and see what happens.

Resolving patch:
diff --git a/src/prefect/configuration.py b/src/prefect/configuration.py
--- a/src/prefect/configuration.py
+++ b/src/prefect/configuration.py
@@ -206,11 +206,6 @@ def interpolate_config(config: dict, env_var_prefix: str = None) -> Config:
                 if "__" not in env_var:
                     continue
 
-                # env vars with escaped characters are interpreted as literal "\", which
-                # Python helpfully escapes with a second "\". This step makes sure that
-                # escaped characters are properly interpreted.
-                value = cast(str, env_var_value.encode().decode("unicode_escape"))
-
                 # place the env var in the flat config as a compound key
                 if env_var_option.upper().startswith("CONTEXT__SECRETS"):
                     formatted_option = env_var_option.split("__")
@@ -224,7 +219,7 @@ def interpolate_config(config: dict, env_var_prefix: str = None) -> Config:
                     )
 
                 flat_config[config_option] = string_to_type(
-                    cast(str, interpolate_env_vars(value))
+                    cast(str, interpolate_env_vars(env_var_value))
                 )
 
     # interpolate any env vars referenced


Test patch:
diff --git a/tests/test_configuration.py b/tests/test_configuration.py
--- a/tests/test_configuration.py
+++ b/tests/test_configuration.py
@@ -67,7 +67,7 @@ def config(test_config_file_path, monkeypatch):
     )
     monkeypatch.setenv("PATH", "1/2/3")
     monkeypatch.setenv(
-        "PREFECT_TEST__ENV_VARS__ESCAPED_CHARACTERS", r"line 1\nline 2\rand 3\tand 4"
+        "PREFECT_TEST__ENV_VARS__ESCAPED_CHARACTERS", "line 1\nline 2\rand 3\tand 4"
     )
 
     yield configuration.load_configuration(