Repository: peopledoc/vault-cli

Issue:
Feature request: bootstrap-env
The idea would be:
```console
$ vault bootstrap-env path1/ path2/ some_secret -- some command
```
Would read all secrets from path1, path2 and some_secret, put them in the environment, and then `exec`  `some command`.

A `--backup /some/location` would make it so that a yaml file with those secrets would be written locally at that location. If the vault is not available and the backup file is, then the backup file is used transparently and the command runs (though with a `logger.exception`).

Use case: start an app that reads settings from the environment (this would be the systemd entrypoint). The `exec` part is important so that we don't have to manage signals or what, the process is replaced with the real process as soon as possible.

The ``--backup`` flag would allow to use this way of starting the app even if your vault instance doesn't have 100% high availability yet (if the app restarts when your vault is down, your app can still run)

Discussion:
codecov-io: # [Codecov](https://codecov.io/gh/peopledoc/vault-cli/pull/47?src=pr&el=h1) Report
> Merging [#47](https://codecov.io/gh/peopledoc/vault-cli/pull/47?src=pr&el=desc) into [master](https://codecov.io/gh/peopledoc/vault-cli/commit/69fc6e4f361b103a8263bf38cf8a5eaca6c4f161?src=pr&el=desc) will **not change** coverage.
> The diff coverage is `100%`.

[![Impacted file tree graph](https://codecov.io/gh/peopledoc/vault-cli/pull/47/graphs/tree.svg?width=650&token=fjUxXKxpOh&height=150&src=pr)](https://codecov.io/gh/peopledoc/vault-cli/pull/47?src=pr&el=tree)

```diff
@@          Coverage Diff          @@
##           master    #47   +/-   ##
=====================================
  Coverage     100%   100%           
=====================================
  Files           7      8    +1     
  Lines         347    410   +63     
  Branches       41     48    +7     
=====================================
+ Hits          347    410   +63
```

| Flag | Coverage Δ | |
|---|---|---|
| #integration | `62.92% <69.89%> (-3.94%)` | :arrow_down: |
| #unit | `100% <100%> (ø)` | :arrow_up: |

| [Impacted Files](https://codecov.io/gh/peopledoc/vault-cli/pull/47?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [vault\_cli/client.py](https://codecov.io/gh/peopledoc/vault-cli/pull/47/diff?src=pr&el=tree#diff-dmF1bHRfY2xpL2NsaWVudC5weQ==) | `100% <100%> (ø)` | :arrow_up: |
| [vault\_cli/types.py](https://codecov.io/gh/peopledoc/vault-cli/pull/47/diff?src=pr&el=tree#diff-dmF1bHRfY2xpL3R5cGVzLnB5) | `100% <100%> (ø)` | |
| [vault\_cli/requests.py](https://codecov.io/gh/peopledoc/vault-cli/pull/47/diff?src=pr&el=tree#diff-dmF1bHRfY2xpL3JlcXVlc3RzLnB5) | `100% <100%> (ø)` | :arrow_up: |
| [vault\_cli/settings.py](https://codecov.io/gh/peopledoc/vault-cli/pull/47/diff?src=pr&el=tree#diff-dmF1bHRfY2xpL3NldHRpbmdzLnB5) | `100% <100%> (ø)` | :arrow_up: |
| [vault\_cli/\_\_main\_\_.py](https://codecov.io/gh/peopledoc/vault-cli/pull/47/diff?src=pr&el=tree#diff-dmF1bHRfY2xpL19fbWFpbl9fLnB5) | `100% <100%> (ø)` | :arrow_up: |
| [vault\_cli/\_\_init\_\_.py](https://codecov.io/gh/peopledoc/vault-cli/pull/47/diff?src=pr&el=tree#diff-dmF1bHRfY2xpL19faW5pdF9fLnB5) | `100% <100%> (ø)` | :arrow_up: |
| [vault\_cli/hvac.py](https://codecov.io/gh/peopledoc/vault-cli/pull/47/diff?src=pr&el=tree#diff-dmF1bHRfY2xpL2h2YWMucHk=) | `100% <100%> (ø)` | :arrow_up: |
| [vault\_cli/cli.py](https://codecov.io/gh/peopledoc/vault-cli/pull/47/diff?src=pr&el=tree#diff-dmF1bHRfY2xpL2NsaS5weQ==) | `100% <100%> (ø)` | :arrow_up: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/peopledoc/vault-cli/pull/47?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/peopledoc/vault-cli/pull/47?src=pr&el=footer). Last update [69fc6e4...992332b](https://codecov.io/gh/peopledoc/vault-cli/pull/47?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).

codecov-io: # [Codecov](https://codecov.io/gh/peopledoc/vault-cli/pull/47?src=pr&el=h1) Report
> Merging [#47](https://codecov.io/gh/peopledoc/vault-cli/pull/47?src=pr&el=desc) into [master](https://codecov.io/gh/peopledoc/vault-cli/commit/69fc6e4f361b103a8263bf38cf8a5eaca6c4f161?src=pr&el=desc) will **not change** coverage.
> The diff coverage is `100%`.

[![Impacted file tree graph](https://codecov.io/gh/peopledoc/vault-cli/pull/47/graphs/tree.svg?width=650&token=fjUxXKxpOh&height=150&src=pr)](https://codecov.io/gh/peopledoc/vault-cli/pull/47?src=pr&el=tree)

```diff
@@          Coverage Diff          @@
##           master    #47   +/-   ##
=====================================
  Coverage     100%   100%           
=====================================
  Files           7      8    +1     
  Lines         347    410   +63     
  Branches       41     48    +7     
=====================================
+ Hits          347    410   +63
```

| Flag | Coverage Δ | |
|---|---|---|
| #integration | `62.92% <69.89%> (-3.94%)` | :arrow_down: |
| #unit | `100% <100%> (ø)` | :arrow_up: |

| [Impacted Files](https://codecov.io/gh/peopledoc/vault-cli/pull/47?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [vault\_cli/client.py](https://codecov.io/gh/peopledoc/vault-cli/pull/47/diff?src=pr&el=tree#diff-dmF1bHRfY2xpL2NsaWVudC5weQ==) | `100% <100%> (ø)` | :arrow_up: |
| [vault\_cli/types.py](https://codecov.io/gh/peopledoc/vault-cli/pull/47/diff?src=pr&el=tree#diff-dmF1bHRfY2xpL3R5cGVzLnB5) | `100% <100%> (ø)` | |
| [vault\_cli/requests.py](https://codecov.io/gh/peopledoc/vault-cli/pull/47/diff?src=pr&el=tree#diff-dmF1bHRfY2xpL3JlcXVlc3RzLnB5) | `100% <100%> (ø)` | :arrow_up: |
| [vault\_cli/settings.py](https://codecov.io/gh/peopledoc/vault-cli/pull/47/diff?src=pr&el=tree#diff-dmF1bHRfY2xpL3NldHRpbmdzLnB5) | `100% <100%> (ø)` | :arrow_up: |
| [vault\_cli/\_\_main\_\_.py](https://codecov.io/gh/peopledoc/vault-cli/pull/47/diff?src=pr&el=tree#diff-dmF1bHRfY2xpL19fbWFpbl9fLnB5) | `100% <100%> (ø)` | :arrow_up: |
| [vault\_cli/\_\_init\_\_.py](https://codecov.io/gh/peopledoc/vault-cli/pull/47/diff?src=pr&el=tree#diff-dmF1bHRfY2xpL19faW5pdF9fLnB5) | `100% <100%> (ø)` | :arrow_up: |
| [vault\_cli/hvac.py](https://codecov.io/gh/peopledoc/vault-cli/pull/47/diff?src=pr&el=tree#diff-dmF1bHRfY2xpL2h2YWMucHk=) | `100% <100%> (ø)` | :arrow_up: |
| [vault\_cli/cli.py](https://codecov.io/gh/peopledoc/vault-cli/pull/47/diff?src=pr&el=tree#diff-dmF1bHRfY2xpL2NsaS5weQ==) | `100% <100%> (ø)` | :arrow_up: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/peopledoc/vault-cli/pull/47?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/peopledoc/vault-cli/pull/47?src=pr&el=footer). Last update [69fc6e4...992332b](https://codecov.io/gh/peopledoc/vault-cli/pull/47?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).


Patch:
diff --git a/README.md b/README.md
index 0cee8fc..cca31b1 100644
--- a/README.md
+++ b/README.md
@@ -49,6 +49,7 @@ Options:
   -h, --help                   Show this message and exit.
 
 Commands:
+  env      Launches the given command with all secrets from --path...
   delete   Deletes a single secret.
   get      Return a single secret value.
   get-all  Return multiple secrets.
@@ -114,6 +115,12 @@ test:
 # Delete a secret
 $ vault delete my_other_secret
 Done
+
+# Launch a process with all secrets from folder blob_secret as environment variables
+$ vault bootstrap-env --path blob_secret -- env
+...
+code=supercode
+...
 ```
 
 ## Configuration
diff --git a/vault_cli/cli.py b/vault_cli/cli.py
index d67be80..5d5a944 100644
--- a/vault_cli/cli.py
+++ b/vault_cli/cli.py
@@ -17,7 +17,7 @@ See the License for the specific language governing permissions and
 limitations under the License.
 """
 import os
-from typing import Any, Dict, Mapping, Sequence
+from typing import Any, Dict, Mapping, NoReturn, Sequence
 
 import click
 import yaml
@@ -146,13 +146,13 @@ def list_(client_obj: client.VaultClientBase, path: str):
 @cli.command(name="get-all")
 @click.argument("path", required=False, nargs=-1)
 @click.pass_obj
-def get_all(client_obj: client.VaultClientBase, path: str):
+def get_all(client_obj: client.VaultClientBase, path: Sequence[str]):
     """
     Return multiple secrets. Return a single yaml with all the secrets located
     at the given paths. Folders are recursively explored. Without a path,
     explores all the vault.
     """
-    paths = path or [""]
+    paths = list(path) or [""]
 
     result = client_obj.get_all(paths)
 
@@ -205,7 +205,6 @@ def set_(
     Value can be either passed as argument (several arguments will be
     interpreted as a list) or via stdin with the --stdin flag.
     """
-
     if stdin and value:
         raise click.UsageError("Can't set both --stdin and a value")
 
@@ -238,6 +237,48 @@ def delete(client_obj: client.VaultClientBase, name: str) -> None:
     click.echo("Done")
 
 
+@cli.command("env")
+@click.option(
+    "-p",
+    "--path",
+    multiple=True,
+    required=True,
+    help="Folder or single item. Pass several times to load multiple values",
+)
+@click.argument("command", nargs=-1)
+@click.pass_obj
+def env(
+    client_obj: client.VaultClientBase, path: Sequence[str], command: Sequence[str]
+) -> NoReturn:
+    """
+    Launches the given command with all secrets from --path
+    loaded in environment.
+    """
+    paths = list(path) or [""]
+
+    secrets = client_obj.get_all(paths, merged=True)
+    secrets_str = {key: str(value) for key, value in secrets.items()}
+
+    environ = os.environ.copy()
+    environ.update(secrets_str)
+
+    exec_command(command=command, environ=environ)
+
+
+def read_yaml(filepath) -> Dict[str, str]:
+    with open(os.path.expanduser(filepath), "r") as f:
+        return yaml.safe_load(f)
+
+
+def write_yaml(filepath, content) -> None:
+    with open(os.path.expanduser(filepath), "w") as f:
+        f.write(yaml.safe_dump(content, default_flow_style=False))
+
+
+def exec_command(command: Sequence[str], environ: Dict[str, str]) -> NoReturn:
+    os.execvpe(command[0], tuple(command), environ)
+
+
 def main():
     # https://click.palletsprojects.com/en/7.x/python3/
     os.environ.setdefault("LC_ALL", "C.UTF-8")
diff --git a/vault_cli/client.py b/vault_cli/client.py
index 0e6349c..d1f43d4 100644
--- a/vault_cli/client.py
+++ b/vault_cli/client.py
@@ -146,10 +146,15 @@ class VaultClientBase:
         else:
             raise ValueError("No authentication method supplied")
 
-    def _get_recursive_secrets(self, path: str) -> types.JSONDict:
+    def _get_recursive_secrets(self, path: str) -> types.JSONValue:
         result: types.JSONDict = {}
         path = path.rstrip("/")
-        for key in self.list_secrets(path=path):
+        sub_secrets = self.list_secrets(path=path)
+
+        if not sub_secrets:
+            return self.get_secret(path=path)
+
+        for key in sub_secrets:
             key_url = "/".join([path, key]) if path else key
 
             folder = key_url.endswith("/")
@@ -163,7 +168,7 @@ class VaultClientBase:
 
         return result
 
-    def get_all(self, paths: Iterable[str]) -> types.JSONDict:
+    def get_all(self, paths: Iterable[str], merged: bool = False) -> types.JSONDict:
         result: types.JSONDict = {}
 
         for path in paths:
@@ -175,8 +180,40 @@ class VaultClientBase:
             assert isinstance(root_val, dict)
             result.update(root_val)
 
+        if merged:
+            result = self._merge_secrets(result)
+
         return result
 
+    def _merge_secrets(self, secrets: types.JSONDict) -> types.JSONDict:
+        """
+        From a dict containing both individual values and folders of
+        individual values, create a dict with all the secrets at the same
+        level. Mainly meant for when all values are strings or dicts of strings.
+
+        Imagine you've constructed a dict with 3 paths: 2 folders and a secret
+        >>> d = {
+            "django": {"b": {"m": "n"}, "d": "e"},
+            "conf": {"g": "h", "i": "j", "b": {"x": "y"}},
+            "some_secret": "l",
+        }
+        >>> _merge_secrets(d)
+        {
+            "b": {"x": "y"},
+            "d": "e",
+            "g": "h",
+            "i": "j",
+            "some_secret": "l",
+        }
+        """
+
+        for key, value in list(secrets.items()):
+            if isinstance(value, dict):
+                secrets.pop(key)
+                secrets.update(value)
+
+        return secrets
+
     def _init_session(self, url: str, verify: types.VerifyOrCABundle) -> None:
         raise NotImplementedError
 


Test patch:
diff --git a/tests/integration/test_integration.py b/tests/integration/test_integration.py
index ba1b4de..0da3aea 100644
--- a/tests/integration/test_integration.py
+++ b/tests/integration/test_integration.py
@@ -1,4 +1,5 @@
 import os
+import subprocess
 
 import pytest
 import vault_cli
@@ -68,8 +69,36 @@ def test_integration_lib():
     client.delete_secret("c/d")
 
 
-def test_env_var_config():
+@pytest.fixture
+def environ():
+    yield os.environ
+    os.environ.pop("VAULT_CLI_TOKEN")
+
+
+def test_env_var_config(environ):
     # Test env var config
-    os.environ["VAULT_CLI_TOKEN"] = "some-other-token"
+    environ["VAULT_CLI_TOKEN"] = "some-other-token"
     with pytest.raises(vault_cli.VaultAPIException):
         vault_cli.get_client().set_secret("a", "b")
+
+
+def check_call(command):
+    subprocess.check_call(
+        command.split(), stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL
+    )
+
+
+@pytest.fixture
+def set_ACD(cli_runner):
+    call(cli_runner, ["set", "A", "B"])
+    call(cli_runner, ["set", "C/D", "E"])
+    yield
+    call(cli_runner, ["delete", "A"])
+    call(cli_runner, ["delete", "C/D"])
+
+
+def test_boostrap_env(set_ACD):
+    env = subprocess.check_output("vault env -p A -p C -- env".split())
+
+    assert b"A=B\n" in env
+    assert b"D=E\n" in env
diff --git a/tests/unit/test_cli.py b/tests/unit/test_cli.py
index c768429..adcac72 100644
--- a/tests/unit/test_cli.py
+++ b/tests/unit/test_cli.py
@@ -175,6 +175,41 @@ def test_delete(cli_runner, backend):
     assert backend.deleted == "a"
 
 
+def test_env(cli_runner, backend, mocker):
+    exec_command = mocker.patch("vault_cli.cli.exec_command")
+
+    cli_runner.invoke(
+        cli.cli, ["env", "--path", "foo", "--", "echo", "yay"], catch_exceptions=False
+    )
+
+    _, kwargs = exec_command.call_args
+    assert kwargs["command"] == ("echo", "yay")
+    assert kwargs["environ"]["foo"] == "bar"
+    assert kwargs["environ"]["baz"] == "bar"
+
+
+def test_read_yaml(tmpdir):
+    path = str(tmpdir.join("test.yml"))
+    open(path, "w").write('{"yay": 1}')
+
+    assert cli.read_yaml(path) == {"yay": 1}
+
+
+def test_write_yaml(tmpdir):
+    path = str(tmpdir.join("test.yml"))
+    cli.write_yaml(path, {"yay": 1})
+
+    assert open(path, "r").read() == "yay: 1\n"
+
+
+def test_exec_command(mocker):
+    execvpe = mocker.patch("os.execvpe")
+
+    cli.exec_command(["a", "b"], {"c": "d"})
+
+    execvpe.assert_called_with("a", ("a", "b"), {"c": "d"})
+
+
 def test_main(mocker):
     mock_cli = mocker.patch("vault_cli.cli.cli")
     environ = mocker.patch("os.environ", {})
diff --git a/tests/unit/test_client.py b/tests/unit/test_client.py
index a1b690d..22c27ee 100644
--- a/tests/unit/test_client.py
+++ b/tests/unit/test_client.py
@@ -261,6 +261,34 @@ def test_vault_client_base_get_recursive_secrets():
     assert result == {"a": "secret-a", "b": {"c": "secret-bc"}}
 
 
+def test_vault_client_base_get_recursive_secrets_single_secret():
+    class TestVaultClient(client.VaultClientBase):
+        def __init__(self):
+            pass
+
+        def list_secrets(self, path):
+            return {"a": []}[path]
+
+        def get_secret(self, path):
+            return "secret-a"
+
+    result = TestVaultClient()._get_recursive_secrets("a")
+
+    assert result == "secret-a"
+
+
+def test_vault_client_base__merge_secrets():
+    class TestVaultClient(client.VaultClientBase):
+        def __init__(self):
+            pass
+
+    assert TestVaultClient()._merge_secrets({"a": "b", "c": {"d": "e", "f": "g"}}) == {
+        "a": "b",
+        "d": "e",
+        "f": "g",
+    }
+
+
 def test_vault_client_base_get_all():
     class TestVaultClient(client.VaultClientBase):
         def __init__(self):
@@ -279,3 +307,7 @@ def test_vault_client_base_get_all():
     result = TestVaultClient().get_all(["a"])
 
     assert result == {"a": {"c": "secret-ac"}}
+
+    result = TestVaultClient().get_all(["a", ""], merged=True)
+
+    assert result == {"c": "secret-ac", "b": "secret-b"}