Get started with Hashicorp Vault
By: Phil Xu
Tags: Consul, Hashicorp, Hashicorp Vault, Secrets, Kubernetes

HashiCorp Vault — This product is currently running in many big enterprise companies. I have seen a lot of people complain about the complexity of it and the pain of setting it up. I think this product is great and would like to share my experience and knowledge to help you speed up the initial set up. You will find it’s not as difficult as you think.

In this tutorial, I will walk you through how to set up vault with different types of backends in your local environment, also give you the example code so you can begin with a single command. But first, let’s have a quick look at why this product is currently in high demand.

What is Vault and Why?

Vault is the one-stop-shop for all your sensitive data. It stores and tightly controls access to tokens, passwords, certificates, encryption keys for protecting secrets, and other sensitive data using a UI, CLI, or HTTP API.

Centralized — Vault is a comprehensive solution in one box, no need to have additional products for a specific feature, it contains

Key Management System

Encryption System

PKI System

…

Multi-cloud — Just like all the other HashiCorp products, Vault is multi-cloud friendly. Imagine that your company is running a multi-cloud solution, and your AWS application needs to have access to GCP projects, the AWS IAM doesn’t make sense to GCP, however, with Vault as a broker, it does.

Run Anywhere — Vault is just a piece of software and can be run anywhere. To install it, use homebrew , yum and apt-get , or maybe just download directly from Hashi’s website.

Vault binary can be used in many modes

Server mode — this is the mode that allows Vault to host the API server, interact with your client, and persist your secret data.

this is the mode that allows Vault to host the API server, interact with your client, and persist your secret data. Client mode — the same binary can be used as a client, since vault is just an API server, in the client mode vault can send a request to the server (consider it as a wrapper for curl) you may use curl or any other tool to send requests as well.

the same binary can be used as a client, since vault is just an API server, in the client mode vault can send a request to the server (consider it as a wrapper for curl) you may use curl or any other tool to send requests as well. Agent mode — this is a slightly more advanced way of using vault, it will sit somewhere in your VM or Container, talk to the vault server and retrieve the secret on your behalf, and eventually injects the secrets somewhere in the VM for your application. Read my other post for more details.

Let’s have a quick look at how to run it with different backend storage.

Run the test server in memory

Have vault added in your environment PATH and run the following command, just as simple as that, you can start to use vault in dev mode against your localhost at port 8200 .

vault server -dev

Or do it with docker

docker run --cap-add=IPC_LOCK -e 'VAULT_DEV_ROOT_TOKEN_ID=myroot' -e 'VAULT_DEV_LISTEN_ADDRESS=0.0.0.0:1234' -p 8200:1234 vault

IPC_LOCK memory lock is required to prevent swapping sensitive data to disk

VAULT_DEV_ROOT_TOKEN_ID (optional) the root id can be easily be set in dev with the environment variable, if not, Vault will generate a random root token during initialization and output it in the log. Once up in dev mode, Vault will be initialized and unsealed, i.e. ready to use.

VAULT_DEV_LISTEN_ADDRESS (optional), it defines where the API endpoint is listening to, needs to be 0.0.0.0 if in a container.

Use the same binary as the client to login with the following command, and Vault server will return a token stored at ~/.vault-token , the vault client will use it to send any later commands.

vault login <your-password>

vault login with root token

File storage backend

Vault can the secret data in a local file. However, it does not support High Availability, so it’s not ideal for production, I personally use it as the local test environment, so you can keep the work environment from last time.

In order to make it work, we will need to create a config file and add the following code block. This block is telling vault where to persist the encrypted data and listen to any request at port 8200 , you can also custom the storage settings and have multiple listener blocks.

storage "file" {

path = "vault/data"

} listener "tcp" {

address = "127.0.0.1:8200"

tls_disable = 1

} ui = true

Run the following command to start Vault server.

vault server -config=/path/to/your/config/file

When running Vault in non-dev mode, you will need to init and unseal the Vault before using it.

vault operator init

vault operator init

vault operator unseal <shamir-key1>

vault operator unseal <shamir-key2>

vault operator unseal <shamir-key3>

vault login <initial-root-token>

Of course, you can do it in docker, it’s pretty much what people do these days, to keep things tidy and clean. Start fresh from downloading the latest version of Vault on alpine or simply use the latest Vault Docker image.

Since we have the ui=true specified in the config file, Vault is also available at localhost:8200 , try it in the browser.

Sealed Vault

Use Consul as backend

When talking about Consul we can easily expand this into another whole new article, but for now, all you need to know is we can use Consul as the storage, and it supports HA which is the recommended way of running vault for production by HashiCorp. The ideal structure is having multiple consul servers as the storage cluster and a few vault servers on top. A production-ready cluster diagram would look as follows.

Here I will provide an example of a docker-compose running a single vault and consul together. Don’t worry if the code is not complete here, I have the repo at the very end of the page, with more operational instructions in the README.

docker-compose up -d

Clone the repo and run the above command in the consul-backend folder, after about 10 seconds, Vault and Consul should be alive.

Kubernetes

You may think using Vault is the most complex on k8s, but on the contrary, it’s actually the easiest, everything has been sorted out for you, you can get it installed using helm and it will work without any change to the default chart file.

Install Vault only



helm install vault hashicorp/vault helm repo add hashicorp https://helm.releases.hashicorp.com helm install vault hashicorp/vault

Or use Consul as the backend



helm install consul hashicorp/consul --values helm-consul-values.yml

helm install vault hashicorp/vault --values helm-vault-values.yml helm repo add hashicorp https://helm.releases.hashicorp.com helm install consul hashicorp/consul --values helm-consul-values.ymlhelm install vault hashicorp/vault --values helm-vault-values.yml

That’s pretty much what you need to do to get Vault up and running to play with, see it’s not that hard right?

Here is the repo with all the script you need on this page. Thanks for reading.

I hope you found this page useful so far. Follow me and watch this space as I will be posting more Vault related articles. Here is my personal blog, check it out and click the AD link if you like it. Cheers.