Repository: peopledoc/vault-cli

Issue:
TemporaryFilesystem strategy from SystemD howto doesn't work
Hello, in your systemd tutorial, you suggest the usage of `TemporaryFilesystem` to create a temporary tmpfs directory and store the secrets in a file inside it so the executable in `ExecStart=` can read it.

Example:
``` shell
[Service]
TemporaryFileSystem=/private
ExecStartPre=vault-cli get mysecret --output=/private/path/to/secret/file
```

In my case, I want to use this with Postgres, so I can store the TLS certificate and private key inside Vault and save it inside that directory so Postgres can read it from there.

The issue with that strategy is that the private directory created by `TemporaryFilesystem` will be removed after the `ExecStartPre=` call and will not be accessible from `ExecStart=`.

 Do you know if there is some way around it? I know this is more a systemd specific issue, but since it is referenced in your howto, I'm asking here so we can improve the howto by specifying what more is needed to be done for this strategy to work or simply remove it from the howto since it will not work in the end.

More about the systemd issue [here](https://github.com/systemd/systemd/issues/8695).

Discussion:
codecov[bot]: # [Codecov](https://codecov.io/gh/peopledoc/vault-cli/pull/186?src=pr&el=h1) Report
> Merging [#186](https://codecov.io/gh/peopledoc/vault-cli/pull/186?src=pr&el=desc) (da3476c) into [master](https://codecov.io/gh/peopledoc/vault-cli/commit/3d8871a3c0eb416a0288b53fbe7fc04ba3bc2d1a?el=desc) (3d8871a) will **decrease** coverage by `1.97%`.
> The diff coverage is `36.66%`.

[![Impacted file tree graph](https://codecov.io/gh/peopledoc/vault-cli/pull/186/graphs/tree.svg?width=650&height=150&src=pr&token=fjUxXKxpOh)](https://codecov.io/gh/peopledoc/vault-cli/pull/186?src=pr&el=tree)

```diff
@@             Coverage Diff             @@
##            master     #186      +/-   ##
===========================================
- Coverage   100.00%   98.02%   -1.98%     
===========================================
  Files           13       13              
  Lines          938      963      +25     
  Branches       120      124       +4     
===========================================
+ Hits           938      944       +6     
- Misses           0       17      +17     
- Partials         0        2       +2     
```


| [Impacted Files](https://codecov.io/gh/peopledoc/vault-cli/pull/186?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [vault\_cli/cli.py](https://codecov.io/gh/peopledoc/vault-cli/pull/186/diff?src=pr&el=tree#diff-dmF1bHRfY2xpL2NsaS5weQ==) | `93.72% <36.66%> (-6.28%)` | :arrow_down: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/peopledoc/vault-cli/pull/186?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/peopledoc/vault-cli/pull/186?src=pr&el=footer). Last update [3d8871a...b4167f0](https://codecov.io/gh/peopledoc/vault-cli/pull/186?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).

ewjoachim: Thanks @sheb 

Only thing left seems:
```
./vault_cli/cli.py:2:1: F401 'json' imported but unused
./tests/unit/test_cli.py:463:36: F541 f-string is missing placeholders
```
ewjoachim: Meanwhile: a mypy update has broken the build :D

Your choice: we can solve this in this PR or in a new PR, it's independant from the modification this PR did bring. I have limited time to take care of this now, but I'll review anything you ping me on :)

Patch:
diff --git a/README.rst b/README.rst
index 1738ee6..b50aee2 100644
--- a/README.rst
+++ b/README.rst
@@ -65,7 +65,7 @@ Here are a few things you might do with ``vault-cli``:
     ohsosecret
 
     $ # Load a secret into the environment variables:
-    $ vault-cli env --path mysecret -- env | grep MYSECRET
+    $ vault-cli env --envvar mysecret -- env | grep MYSECRET
     MYSECRET_MYKEY=ohsosecret
 
     $ # Load an ssh key into your ssh-agent:
diff --git a/docs/howto/environment.rst b/docs/howto/environment.rst
index 7f3cefa..87cf9cf 100644
--- a/docs/howto/environment.rst
+++ b/docs/howto/environment.rst
@@ -16,7 +16,7 @@ works:
     $ vault-cli set test/my_secret value=qwerty
     Done
 
-    $ vault-cli env --path test/my_secret -- bash -c 'echo $MY_SECRET_VALUE'
+    $ vault-cli env --envvar test/my_secret -- bash -c 'echo $MY_SECRET_VALUE'
     qwerty
 
 Environment variable naming
@@ -68,29 +68,29 @@ Let's consider the vault contains only the following secret:
 This table maps input to output. Note that there will always be a single environment
 variable and its value will always be ``mysecret``.
 
-+-------------+-----------------------+---------------------------+
-| ``--path``  | ``--omit-single-key`` | environment variable name |
-+-------------+-----------------------+---------------------------+
-| ``a``       | False                 | ``A_B_C``                 |
-+-------------+-----------------------+---------------------------+
-| ``a``       | True                  | ``A_B``                   |
-+-------------+-----------------------+---------------------------+
-| ``a=D``     | False                 | ``D_B_C``                 |
-+-------------+-----------------------+---------------------------+
-| ``a=D``     | True                  | ``D_B``                   |
-+-------------+-----------------------+---------------------------+
-| ``a/b``     | False                 | ``B_C``                   |
-+-------------+-----------------------+---------------------------+
-| ``a/b``     | True                  | ``B``                     |
-+-------------+-----------------------+---------------------------+
-| ``a/b=D``   | False                 | ``D_C``                   |
-+-------------+-----------------------+---------------------------+
-| ``a/b=D``   | True                  | ``D``                     |
-+-------------+-----------------------+---------------------------+
-| ``a/b:c``   | True or False         | ``C``                     |
-+-------------+-----------------------+---------------------------+
-| ``a/b:c=D`` | True or False         | ``D``                     |
-+-------------+-----------------------+---------------------------+
++---------------+-----------------------+---------------------------+
+| ``--envvar``  | ``--omit-single-key`` | environment variable name |
++---------------+-----------------------+---------------------------+
+| ``a``         | False                 | ``A_B_C``                 |
++---------------+-----------------------+---------------------------+
+| ``a``         | True                  | ``A_B``                   |
++---------------+-----------------------+---------------------------+
+| ``a=D``       | False                 | ``D_B_C``                 |
++---------------+-----------------------+---------------------------+
+| ``a=D``       | True                  | ``D_B``                   |
++---------------+-----------------------+---------------------------+
+| ``a/b``       | False                 | ``B_C``                   |
++---------------+-----------------------+---------------------------+
+| ``a/b``       | True                  | ``B``                     |
++---------------+-----------------------+---------------------------+
+| ``a/b=D``     | False                 | ``D_C``                   |
++---------------+-----------------------+---------------------------+
+| ``a/b=D``     | True                  | ``D``                     |
++---------------+-----------------------+---------------------------+
+| ``a/b:c``     | True or False         | ``C``                     |
++---------------+-----------------------+---------------------------+
+| ``a/b:c=D``   | True or False         | ``D``                     |
++---------------+-----------------------+---------------------------+
 
 Recommended setup
 -----------------
@@ -112,7 +112,7 @@ Your call would look like:
 
 .. code:: console
 
-    $ vault-cli env --omit-single-key --path myapp -- myapp
+    $ vault-cli env --omit-single-key --envvar myapp -- myapp
 
 Ignoring errors
 ---------------
@@ -123,7 +123,7 @@ even if it will be missing some secrets.
 
 .. code:: console
 
-    $ vault-cli env --path myapp --force -- myapp
+    $ vault-cli env --envvar myapp --force -- myapp
 
 .. warning::
 
diff --git a/docs/howto/read.rst b/docs/howto/read.rst
index 67bc231..821169b 100644
--- a/docs/howto/read.rst
+++ b/docs/howto/read.rst
@@ -74,6 +74,11 @@ can write the secret to a specific file:
     with ways to write on ephemeral storage, and check your umask__ and the permissions
     of the created file. See :ref:`SystemD` for safe integration strategies.
 
+.. note::
+
+    ``vault-cli env`` also lets you to write secrets to a file just before launching
+    an arbitrary command.
+
 .. __: https://en.wikipedia.org/wiki/Umask
 
 
diff --git a/docs/howto/ssh.rst b/docs/howto/ssh.rst
index c85b236..ffe2982 100644
--- a/docs/howto/ssh.rst
+++ b/docs/howto/ssh.rst
@@ -38,5 +38,5 @@ If you need to have both ssh access and secrets as environment variables (see
 
     $ # If your key is not passphrase-protected
     $ vault-cli ssh --key path/to/ssh_private_key:value \
-      -- vault-cli env --path myapp \
+      -- vault-cli env --envvar myapp \
       -- myapp_that_needs_secrets_and_ssh
diff --git a/docs/howto/systemd.rst b/docs/howto/systemd.rst
index a4da671..0142a58 100644
--- a/docs/howto/systemd.rst
+++ b/docs/howto/systemd.rst
@@ -58,7 +58,7 @@ launch the program through ``vault-cli env``. Let’s launch it as a one-off:
 
 .. code:: console
 
-   $ vault-cli env --path mysecret:value -- myprogram
+   $ vault-cli env --envvar mysecret:value -- myprogram
 
 This will make a variable named ``VALUE`` available to ``myprogram``.
 See the :ref:`vault-cli env <vault-env>` dedicated page for more details on how you can
@@ -85,7 +85,7 @@ We’ll create an override file that will change ExecStart to wrap it in
    # opens a new file for edition
    [Service]
    ExecStart=
-   ExecStart=vault-cli env --path mysecret:value=MYVAR -- myprogram --options
+   ExecStart=vault-cli env --envvar mysecret:value=MYVAR -- myprogram --options
 
 The empty ``ExecStart=`` tells SystemD to ignore the previous command to
 launch and only launch the following one.
@@ -97,8 +97,8 @@ Save and quit the file. Load you new configuration file with:
    $ sudo systemctl daemon-reload
    $ sudo systemctl restart myprogram.service
 
-Writing a single secret to a file before start
-----------------------------------------------
+Writing secrets to files on the filesystem before start
+-------------------------------------------------------
 
 In some cases, you will need to have a file in the filesystem that
 contains directly the secret. This is often the case with private keys.
@@ -110,8 +110,8 @@ be written on disk.
 
 .. __: https://en.wikipedia.org/wiki/RAM_drive
 
-In this case, we’ll also create a service override file, but this time,
-we will be adding a command that launches before our main command:
+In this case, we’ll also create a service override file. We'll add a wrapper
+arount our program like before.
 
 .. code:: console
 
@@ -119,7 +119,7 @@ we will be adding a command that launches before our main command:
    # opens a new file for edition
    [Service]
    TemporaryFileSystem=/private
-   ExecStartPre=vault-cli get mysecret --output=/private/path/to/secret/file
+   ExecStart=vault-cli env --file mysecret:key=/private/path/to/secret/file -- myprogram --options
 
 Save and quit the file. Load your new configuration file with:
 
@@ -131,7 +131,7 @@ Save and quit the file. Load your new configuration file with:
 You will need to configure ``myprogram`` to look for your
 secret file at ``/private/path/to/secret/file``.
 
-If you need several files, you can repeat the ``ExecStartPre`` line as
+If you need several files, you can add more ``--file`` flags, as
 many times as needed.
 
 .. note::
@@ -143,6 +143,15 @@ many times as needed.
 Bake secrets into a complex configuration file
 ----------------------------------------------
 
+.. warning::
+
+   It's been reported__ that this approach doesn't work as intended. It's left
+   for inspiration, but as of today, ``ExecStartPre`` cannot write to the
+   private filesystem created by ``TemporaryFileSystem`` in  way that ``ExecStart``
+   can later read. Please refer to the ticket for workarounds.
+
+   .. __: https://github.com/peopledoc/vault-cli/issues/185
+
 In some cases, the program you want to launch doesn’t accept
 configuration through environment but only through configuration files.
 You could be tempted to use the method above, but the configuration file
diff --git a/docs/howto/upgrade.rst b/docs/howto/upgrade.rst
index 69232bc..d917998 100644
--- a/docs/howto/upgrade.rst
+++ b/docs/howto/upgrade.rst
@@ -37,10 +37,10 @@ The following list shows how to update your commands:
    (old) vault get path/to/creds
    (new) vault get path/to/creds value
 
-   (old) vault env --path path/to/creds=FOO -- env  # FOO=xxx
-   (new) vault env --path path/to/creds=FOO -- env  # FOO_VALUE=xxx
-   (new) vault env --path path/to/creds:value=FOO -- env  # FOO=xxx
-   (new) vault env --omit-single_key --path path/to/creds=FOO -- env  # FOO=xxx
+   (old) vault env --envvar path/to/creds=FOO -- env  # FOO=xxx
+   (new) vault env --envvar path/to/creds=FOO -- env  # FOO_VALUE=xxx
+   (new) vault env --envvar path/to/creds:value=FOO -- env  # FOO=xxx
+   (new) vault env --omit-single_key --envvar path/to/creds=FOO -- env  # FOO=xxx
 
 The default output of ``vault get-all`` has also changed and is now flat
 by default (this behavior is controlled with the ``--flat/--no-flat``
diff --git a/docs/quickstart.rst b/docs/quickstart.rst
index 361ffdc..4b42d69 100644
--- a/docs/quickstart.rst
+++ b/docs/quickstart.rst
@@ -208,7 +208,7 @@ Let's try it. First we'll launch the command ``env``, which prints the environme
 
 .. code:: console
 
-    $ vault-cli env --path demo -- env | tail -1
+    $ vault-cli env --envvar demo -- env | tail -1
     DEMO_BLAKE2_SECRET_KEY=du9dibieNg3lei0teidal9
 
 As you can see, the secrets (or, here, the secret) under the path ``demo`` have been
@@ -238,7 +238,7 @@ Ok, now for the real thing:
 
 .. code:: console
 
-    $ vault-cli env --path demo -- ./docs/quickstart_demo.py yay
+    $ vault-cli env --envvar demo -- ./docs/quickstart_demo.py yay
     341c93333a9df726c57671891d6bbea1
 
 **Yay!**
diff --git a/vault_cli/cli.py b/vault_cli/cli.py
index 9dc3c54..8a68744 100644
--- a/vault_cli/cli.py
+++ b/vault_cli/cli.py
@@ -465,11 +465,27 @@ def delete(client_obj: client.VaultClientBase, name: str, key: Optional[str]) ->
     formerly deprecated. See command help for details.
     """,
 )
+@click.option(
+    "--file",
+    multiple=True,
+    help="""
+    Write a secret from this path into a file on the filesystem. Expected format is
+    path/in/vault[:key]=/path/in/filesystem . This option is meant to be used when
+    your command can only read its inputs from a file and not from the environment (e.g.
+    secret keys, ...). It's highly recommended to only use the option when provided with
+    a secure private temporary filesystem. Writing to a physical disk should be avoided
+    when possible. If the secret a collection (object or array), it's dumped in YAML
+    format.
+    """,
+)
 @click.option(
     "-o",
     "--omit-single-key/--no-omit-single-key",
     default=False,
-    help="When the secret has only one key, don't use that key to build the name of the environment variable",
+    help="""
+    When the secret has only one key, don't use that key to build the name of the
+    environment variable. This option doesn't affect --file.
+    """,
 )
 @click.option(
     "-f",
@@ -483,6 +499,7 @@ def delete(client_obj: client.VaultClientBase, name: str, key: Optional[str]) ->
 def env(
     client_obj: client.VaultClientBase,
     envvar: Sequence[str],
+    file: Sequence[str],
     omit_single_key: bool,
     force: bool,
     command: Sequence[str],
@@ -501,18 +518,19 @@ def env(
     By default the name is build upon the relative path to the parent of the given path (in parameter) and the name of the keys in the value mapping.
     Let's say that we have stored the mapping `{'username': 'me', 'password': 'xxx'}` at path `a/b/c`
 
-    Using `--path a/b` will inject the following environment variables: B_C_USERNAME and B_C_PASSWORD
-    Using `--path a/b/c` will inject the following environment variables: C_USERNAME and C_PASSWORD
-    Using `--path a/b/c:username` will only inject `USERNAME=me` in the environment.
+    Using `--envvar a/b` will inject the following environment variables: B_C_USERNAME and B_C_PASSWORD
+    Using `--envvar a/b/c` will inject the following environment variables: C_USERNAME and C_PASSWORD
+    Using `--envvar a/b/c:username` will only inject `USERNAME=me` in the environment.
 
     You can customize the variable names generation by appending `=SOME_PREFIX` to the path.
     In this case the part corresponding to the base path is replace by your prefix.
 
-    Using `--path a/b=FOO` will inject the following environment variables: FOO_C_USERNAME and FOO_C_PASSWORD
-    Using `--path a/b/c=FOO` will inject the following environment variables: FOO_USERNAME and FOO_PASSWORD
-    Using `--path a/b/c:username=FOO` will inject `FOO=me` in the environment.
+    Using `--envvar a/b=FOO` will inject the following environment variables: FOO_C_USERNAME and FOO_C_PASSWORD
+    Using `--envvar a/b/c=FOO` will inject the following environment variables: FOO_USERNAME and FOO_PASSWORD
+    Using `--envvar a/b/c:username=FOO` will inject `FOO=me` in the environment.
     """
     envvars = list(envvar) or []
+    files = list(file) or []
 
     env_secrets = {}
 
@@ -534,6 +552,22 @@ def env(
 
         env_secrets.update(env_updates)
 
+    for file in files:
+        path, key, filesystem_path = get_env_parts(file)
+        if not (path and filesystem_path):
+            raise click.BadOptionUsage(
+                "file", "--file expects both a vault path and a filesystem path"
+            )
+        secret_obj = client_obj.get_secret(path=path, key=key or None)
+
+        if not isinstance(secret_obj, (list, dict)):
+            secret = str(secret_obj).strip() + "\n"
+        else:
+            secret = yaml.safe_dump(
+                secret_obj, default_flow_style=False, explicit_start=True
+            )
+        pathlib.Path(filesystem_path).write_text(secret)
+
     if bool(client_obj.errors) and not force:
         raise click.ClickException("There was an error while reading the secrets.")
     environment.exec_command(command=command, environment=env_secrets)


Test patch:
diff --git a/tests/unit/test_cli.py b/tests/unit/test_cli.py
index 8dd2559..9760fce 100644
--- a/tests/unit/test_cli.py
+++ b/tests/unit/test_cli.py
@@ -414,9 +414,9 @@ def test_env_filter_key(cli_runner, vault_with_token, mocker):
         cli.cli,
         [
             "env",
-            "--path",
+            "--envvar",
             "foo/baz:user=MYNAME",
-            "--path",
+            "--envvar",
             "foo/baz:password",
             "--",
             "echo",
@@ -444,6 +444,40 @@ def test_env_omit_single_key(cli_runner, vault_with_token, mocker):
     assert kwargs["environment"]["FOO_BAZ"] == "yo"
 
 
+def test_env_file(cli_runner, vault_with_token, mocker, tmp_path):
+    mocker.patch("vault_cli.environment.exec_command")
+
+    path = tmp_path / "foo"
+    vault_with_token.db = {"foo/bar": {"value": "yay"}}
+    cli_runner.invoke(
+        cli.cli, ["env", "--file", f"foo/bar:value={path}", "--", "echo", "yay"]
+    )
+    assert path.read_text() == "yay\n"
+
+
+def test_env_file_format_error(cli_runner, vault_with_token, mocker, tmp_path):
+    mocker.patch("vault_cli.environment.exec_command")
+
+    vault_with_token.db = {"foo/bar": {"value": "yay"}}
+    result = cli_runner.invoke(
+        cli.cli, ["env", "--file", "foo/bar", "--", "echo", "yay"]
+    )
+    assert result.exit_code != 0
+    assert "expects both a vault path and a filesystem path" in result.output
+
+
+def test_env_file_yaml(cli_runner, vault_with_token, mocker, tmp_path):
+    mocker.patch("vault_cli.environment.exec_command")
+
+    path = tmp_path / "foo"
+    vault_with_token.db = {"foo/bar": {"value": "yay"}}
+    cli_runner.invoke(
+        cli.cli,
+        ["env", "--file", f"foo/bar={path}", "--", "echo", "yay"],
+    )
+    assert path.read_text() == "---\nvalue: yay\n"
+
+
 def test_main(environ, mocker):
     mock_cli = mocker.patch("vault_cli.cli.cli")