Vault Associate Certification (Part 6): Utilize Vault CLI
Tags: vault, hashicorp, security, learning
Published: 2023-09-08 15:17:15

In the previous parts of this series we have seen many examples of how to use the Vault CLI to perform various tasks in Vault. In fact, we have only used the CLI to interact with Vault. In the next part of this series we will see the first non-CLI way of interacting with Vault (spoiler: we will use the Vault UI).

Much of the content in this part will be repetition. But _repetition is the mother of learning_ is what they say! 

Using the CLI make up the sixth objective in the Vault certification journey. This objective covers the following sub-objectives:

- [Configure authentication methods](#configure-authentication-methods)
- [Authenticate to Vault](#authenticate-to-vault)
- [Configure Vault policies](#configure-vault-policies)
- [Enable Secret engines](#enable-secret-engines)
- [Access Vault secrets](#access-vault-secrets)
- [Configure environment variables](#configure-environment-variables)

Before we dig into the sub-objectives let's start with a short CLI-101.

## Vault CLI 101

I begin with some basics around Vault CLI commands that is common for every command you might run.

Each (most) command follows this general form:

```console
$ vault <command> [options] [path] [arguments]
```

- Each command we'll see starts with `vault`, because that is the Vault CLI that we all know and love.
- `<command>` might consist of one or two pieces that determines what operation we want to perform in Vault, e.g. `secrets enable`, `auth enable`, `write`, `read`, `token revoke`, etc.
- The `[options]` include flags (i.e. `-flag1=value`, `-flag2=value`, etc). There are some global flags that are available for all commands, and some specific flags depending on the type of command.
- The `[path]` specifies the path (remember that everything in Vault exists at a path) where you want to perform the action.

Using the help flag `-h` for the command you would like to run is one of the best way of getting information for a given command. There is also a `path-help` command that gives you specific help for a path, for example:

```console
$ vault path-help /sys/capabilities

Request:        capabilities
Matching Route: ^capabilities$

Fetches the capabilities of the given token on the given path.

## PARAMETERS

    path (slice)

        (DEPRECATED) Use 'paths' instead.

    paths (slice)

        Paths on which capabilities are being queried.

    token (string)

        Token for which capabilities are being queried.

## DESCRIPTION

Returns the capabilities of the given token on the path.
The path will be searched for a path match in all the policies
associated with the token.
```

## Configure authentication methods

To enable a new auth method, in this case the `userpass` auth method, run:

```console
$ vault auth enable userpass

Success! Enabled userpass auth method at: userpass/
```

To see what auth methods you currently have enabled run:

```console
$ vault auth list

Path         Type        Accessor                  Description                Version
----         ----        --------                  -----------                -------
token/       token       auth_token_5b8eda92       token based credentials    n/a
userpass/    userpass    auth_userpass_20319701    n/a                        n/a
```

Configuring default settings for an auth method is done using the `tune` subcommand. In the following example I change the default lease TTL for tokens coming from the `userpass` auth method from 32 days (the default value) to 12 hours:

```console
$ vault auth tune -default-lease-ttl=12h userpass/

Success! Tuned the auth method at: userpass/
```

If you want to move an enabled auth method to another path you can do so using the `move` subcommand. In the following example I move the `userpass` auth method from `userpass/` to `up/`

```console
$ vault auth move auth/userpass/ auth/up/

Started moving auth method auth/userpass/ to auth/up/, with migration ID 3548ea1b-a039-99bc-85c9-94a2578d8d65
Success! Finished moving auth method auth/userpass/ to auth/up/, with migration ID 3548ea1b-a039-99bc-85c9-94a2578d8d65
```

Note that you must specify the full path of `auth/userpass/` etc. This is a bit confusing, but all the auth methods are really mounted under `auth/` and not directly at `userpass/` and so on.

When you no longer need an auth method you can disable it:

```console
$ vault auth disable up/

Success! Disabled the auth method (if it existed) at: up/
```

## Authenticate to Vault

In the previous posts we've always had a root token set up with our local Vault CLI so we've not had to authenticate to Vault. If we're a user, or application, we must first authenticate using one of the enabled auth methods.

To illustrate how to authenticate to vault using the `userpass` auth method (to continue from the previous section) first enable the auth method:

```console
$ vault auth enable userpass

Success! Enabled userpass auth method at: userpass/
```

Next create a user named `alice`:

```console
$ vault write auth/userpass/users/alice \
    password=S3cr3t123 \
    policies=my-policy

Success! Data written to: auth/userpass/users/alice
```

To authenticate to Vault using the `userpass` auth method and the user named `alice` run the following command:

```console
$ vault login -method=userpass username=alice

Password (will be hidden): ***

Success! You are now authenticated. The token information displayed below
is already stored in the token helper. You do NOT need to run "vault login"
again. Future Vault requests will automatically use this token.

Key                    Value
---                    -----
token                  hvs.CAESIBZ7YFEM<truncated>PaXJwVFdaUDU
token_accessor         WuKRVGKKGMlcRE3oBdHfnyqP
token_duration         768h
token_renewable        true
token_policies         ["default" "my-policy"]
identity_policies      []
policies               ["default" "my-policy"]
token_meta_username    alice
```

Similar commands (`vault login -method=<auth method>`) would be used for the other available auth methods.

## Configure Vault policies

Do you remember policies from back in part two of this course? Policies are used to allow capabilities (actions) on a path(s) for a token. Policies are written in HashiCorp Configuration Language (HCL) or JSON and consist of one or more `path` blocks.

An example of a basic policy:

```hcl
// policy.hcl

path "kv/database/password" {
    capabilities = [ "read" ]
}

path "kv/api/key" {
    capabilities = [ "read" ]
}
```

This policy allows a given token to read from two paths, `kv/database/password` and `kv/api/key`.

Once we have written our policy document (`policy.hcl` in this case) we can create the actual policy in Vault:

```console
$ vault policy write my-policy policy.hcl

Success! Uploaded policy: my-policy
```

We can list all our available policies using the following command:

```console
$ vault policy list

default
my-policy
root
```

To view a specific policy we can use the following command:

```console
$ vault policy read my-policy

path "kv/database/password" {
    capabilities = [ "read" ]
}

path "kv/api/key" {
    capabilities = [ "read" ]
}
```

If we no longer need to keep a policy we can delete it using the following command:

```console
$ vault policy delete my-policy

Success! Deleted policy: my-policy
```

Apart from that there is not much more to say about working with policies using the Vault CLI. To see all available sub-commands and options use:

```console
$ vault policy -h
```

## Enable Secret engines

In the previous post we saw several examples of how to work with secrets engines using the Vault CLI. This section contains a summary of the common commands. I will illustrate the commands using the key/value (kv) secrets engine, both version 1 and 2.

To enable the kv secrets engine:

```console
$ # enabling version 1 at the path kv/
$ vault secrets enable kv
$
$ # enabling version 1 at the path kvv1/
$ vault secrets enable -path=kvv1 kv
$
$ # enabling version 2 at the path kvv2/
$ vault secrets enable -version=2 -path=kvv2 kv
```

If you have enabled version 1 at the path `kv/` and would like to upgrade it to version 2 you could run the following command:

```console
$ vault kv enable-versioning kv

Success! Tuned the secrets engine at: kv/
```

To see all the secrets engines I have enabled I can run:

```console
$ vault secrets list

Path          Type         Accessor              Description
----          ----         --------              -----------
cubbyhole/    cubbyhole    cubbyhole_e99e381f    per-token private secret storage
identity/     identity     identity_146bba4e     identity store
kv/           kv           kv_cd4c8f19           n/a
secret/       kv           kv_5260e2d2           key/value secret storage
sys/          system       system_55d6ae73       system endpoints
```

If I would like additional details I can add the `-detailed` flag:

```console
$ vault secrets list -detailed

# (output not shown due to size limits)
```

I can move a secrets engine from one path to another. To move my kv secrets engine from `kv/` to `kv-new/` I run:

```console
$ vault secrets move kv/ kv-new/

Started moving secrets engine kv/ to kv-new/, with migration ID a63e75dd-b047-f095-6b67-de1fe27bef88
Success! Finished moving secrets engine kv/ to kv-new/, with migration ID a63e75dd-b047-f095-6b67-de1fe27bef88
```

When I no longer need a given secrets engine I can disable it:

```console
$ vault secrets disable kv-new

Success! Disabled the secrets engine (if it existed) at: kv-new/
```

## Access Vault secrets

We saw many examples of how to access Vault secrets using the Vault CLI in the previous post, but we'll repeat that material a bit here. For these examples I will continue using the key/value secrets engine, version 2. To start I enable the secrets engine:

```console
$ vault secrets enable -version=2 -path=secrets kv

Success! Enabled the kv secrets engine at: secrets/
```

Next I store a secret at the path `database/passwords`:

```console
$ vault kv put secrets/database/password password=s3cr3tz

========= Secret Path =========
secrets/data/database/password

======= Metadata =======
Key                Value
---                -----
created_time       2023-09-08T15:05:57.185984Z
custom_metadata    <nil>
deletion_time      n/a
destroyed          false
version            1
```

Since I am using version 2 of the kv secrets engine I can update the password to a new version if I like:

```console
$ vault kv put secrets/database/password password=s3cr3tzhello

========= Secret Path =========
secrets/data/database/password

======= Metadata =======
Key                Value
---                -----
created_time       2023-09-08T15:07:03.875497Z
custom_metadata    <nil>
deletion_time      n/a
destroyed          false
version            2
```

We can see that `version` increased to `2`. To read my secret back I can run:

```console
$ vault kv get secrets/database/password

========= Secret Path =========
secrets/data/database/password

======= Metadata =======
Key                Value
---                -----
created_time       2023-09-08T20:07:03.875497Z
custom_metadata    <nil>
deletion_time      n/a
destroyed          false
version            2

====== Data ======
Key         Value
---         -----
password    s3cr3tzhello
```

This output is not very convenient for parsing, so let's specify that we want JSON output instead:

```console
$ vault kv get -format=json secrets/database/password
```

The returned data has this format:

```json
{
  "request_id": "d39f28d7-14fc-2f40-5b8d-c7fac2c6cc9f",
  "lease_id": "",
  "lease_duration": 0,
  "renewable": false,
  "data": {
    "data": {
      "password": "s3cr3tzhello"
    },
    "metadata": {
      "created_time": "2023-09-08T20:07:03.875497Z",
      "custom_metadata": null,
      "deletion_time": "",
      "destroyed": false,
      "version": 2
    }
  },
  "warnings": null
}
```

To access a given version of the secret (i.e. an older version) I run:

```console
$ vault kv get -version=1 secrets/database/password

========= Secret Path =========
secrets/data/database/password

======= Metadata =======
Key                Value
---                -----
created_time       2023-09-08T20:05:57.185984Z
custom_metadata    <nil>
deletion_time      n/a
destroyed          false
version            1

====== Data ======
Key         Value
---         -----
password    s3cr3tz
```

If I would like to only access the metadata for a given secret I can do so:

```console
$ vault kv metadata get secrets/database/password

========== Metadata Path ==========
secrets/metadata/database/password

========== Metadata ==========
Key                     Value
---                     -----
cas_required            false
created_time            2023-09-08T20:05:57.185984Z
current_version         2
custom_metadata         <nil>
delete_version_after    0s
max_versions            0
oldest_version          0
updated_time            2023-09-08T20:07:03.875497Z

====== Version 1 ======
Key              Value
---              -----
created_time     2023-09-08T20:05:57.185984Z
deletion_time    n/a
destroyed        false

====== Version 2 ======
Key              Value
---              -----
created_time     2023-09-08T20:07:03.875497Z
deletion_time    n/a
destroyed        false
```

When working with version 2 of the kv secrets engine we can delete versions of our secrets and we can destroy them. Deleting a secret version just places a delete marker on it, but it is recoverable. Destroying a secret version permanently removes it. In the following command I destroy both version 1 and 2 of my secret:

```console
$ vault kv destroy -versions=1,2 secrets/database/password

Success! Data written to: secrets/destroy/database/password
```

## Configure environment variables

There are a number of environment variables that Vault looks for. Most of them are not required at all but could be useful in certain situations. I'll explain a few of the most common environment variables below[^environment]:

- `VAULT_TOKEN` allows you to set the current Vault token which should be used for all the commands you execute.
- `VAULT_ADDR` should contain the URL and port where Vault is running, e.g. `https://my-vault-server.com:8200/`.
- `VAULT_CLIENT_TIMEOUT` is the timeout for commands issued to the Vault server, it defaults to 60 seconds.
- `VAULT_FORMAT` allows you to select the output format for the CLI. The available values are `json`, `yaml`, and `table`.
- `VAULT_LOG_LEVEL` allows you to control the verbosity of the logs that Vault reports. Available values are `trace`, `debug`, `info`, `warn`, and `err`. The default log level is `info`.

[^environment]: For a complete list of environment variables see https://developer.hashicorp.com/vault/docs/commands#environment-variables.