Setup External Secrets with Hashicorp Vault on AWS EKS
Tags: vault, security, aws, secretmanagement
Published: 2023-02-15 11:23:47


[`external-secrets`](https://external-secrets.io/v0.7.2/introduction/getting-started/) is one of the most efficient and secure ways manage Kubernetes Secrets. `external-secrets` integrates with many external secret stores like Hashicorp Vault, AWS Secret Manager, etc. and can be used to manage secret files and variables.

### Key components

The main components of external secrets are as follows:

**External Secrets Operator (ESO)** is a collection of custom API resources - **`ExternalSecret`**, **`SecretStore`**, and **`ClusterSecretStore`** that provide a user-friendly abstraction for the external API that stores and manages the lifecycle of the secrets for you.

**ExternalSecret -** It is a declaration of what data has to be fetched from your external secret manager. It references a **`SecretStore`**, which knows how to access the data. You can do more things like set refresh interval, specify a blueprint for the resulting **`Kind=Secret`**, use inline templates to construct the desired config file containing your secret, set a target that will be created, and create and delete policies.

**SecretStore -** They are namespaced by design and cannot communicate with resources across namespaces. This is the file where you select your ESO controller, and cloud provider, along with the role and access IDs, and retry settings in case of connection failure.

The secrets are fetched from an external vault and made available in SecretStores, these are limited to that particular namespace. For use in multiple namespaces, use the **`ClusterSecretStore`**.

This guide will go through the process of setting up the External Secret Operator for your AWS EKS Kubernetes cluster.

To find out about other approaches to Kubernetes secrets management, check out [this blog](https://www.argonaut.dev/blog/secret-management-in-kubernetes).

### Pre-requisites

1. An existing Kubernetes cluster
2. Kubernetes v1.16.0 or later
3. Connected Apps or databases that require secrets to access
4. Hashicorp Vault as your external secret store
5. `kubectl` and `helm` CLI installed
6. Optional: an Argonaut account and ([`art cli`](https://www.argonaut.dev/docs/setup/art-cli))

Section I of the installation can be done using both Helm (1.a.) or Argonaut’s UI (1.b.). Choose one of the below section.

### 1.a. Install using Helm

To set this up, you will need your **kubeconfig** file. You can use a file created from your existing cluster or obtain this from Argonaut’s CLI.

1: Run `art configure generate-aws-credentials`

2: Run `aws eks update-kubeconfig --name <clustername> --region <region>`

Once this is ready, run the following commands from your Terminal.

For Mac and Linux:

```bash
export KUBECONFIG=/home/argo/.kube/clustername-kubeconfig.yaml
```

For Windows:

```bash
$env:KUBECONFIG = "C:\Users\argo\.kube\clustername-kubeconfig.yaml"
```

Confirm your current context by running the command:

```bash
kubectl config current-context
```

You will see an output with the name of your cluster `argonaut-cluster`. Once the context is set up, you can add `external-secrets` repo to Helm.

```bash
helm repo add external-secrets https://charts.external-secrets.io
```

You can run a helm update to make sure you’re on the latest version

```bash
helm repo update
```

You should see this output: `"external-secrets" has been added to your repositories`

You can now install the external-secrets repository

```bash
helm install external-secrets \
    external-secrets/external-secrets \
    -n tools \
    --create-namespace \
    --set installCRDs=true
```

You should now see an install successful message with the name and namespace specified.

### 1.b. Install with Argonaut

1. Open Argonaut, and navigate to your desired AWS EKS cluster.
1. Click on the `Application +` and the `From Library` option.
1. Choose `Custom-Apps` under configuration.
1. Ensure the selected environment is correct, then select the cluster you want to deploy the agent to.
1. Set the namespace you want to deploy the external secrets to be `tools` (or where you plan to use the secrets).
1. Set the release name of the application as `external-secrets`.
1. Set the chart name as `external-secrets`.
1. Set the chart repository as `https://charts.external-secrets.io`.
1. Leave the chart version blank. It will be automatically populated to the latest version.
1. Load the values.yaml file and make any changes (if required).
1. Then click `Install`. In just a minute, the external-secrets operator will be added to your cluster, and you will be able to see the outputs as follows.

Now your external-secrets operator is installed in the cluster as an Add-on application. You can view the logs, status, and update the configs by going clicking on external-secrets under Add-ons.

### 2. Connecting k8s CLI

External Secrets Operator (ESO) is now installed in your cluster. You can start to use it by connecting to your cluster from your Terminal and running the following kubectl commands.

To access your Kubernetes cluster, you will have to generate AWS credentials through art CLI and connect it to your EKS cluster. This is a quick two step process.

1: Run `art configure generate-aws-credentials`

2: Run `aws eks update-kubeconfig --name <clustername> --region <region>`

This connects and gives you access of your cluster in the specified region.

> Note: The generated access tokens are valid for 12 hours. These inherit the same privileges granted to the Argonaut account for accessing the aws account and infra. To set up more apps such as Mirantis Lens and k9s, follow this [docs page](https://www.argonaut.dev/docs/Configs/access-k8s-cluster).

### 3. secret-store.yaml

The `ClusterSecretStore` is a file having the definition of how the `external-secret` operator can find the secrets from an external secret store. This includes two main things, an existing secret to provide authentication to the external secret store (AWS secret manager linked account) and the AWS project identifier of that particular secret that you wish to access. Make sure you have `kubectl` installed before proceeding further.

```yaml
apiVersion: external-secrets.io/v1beta1
kind: SecretStore
metadata:
  name: "vault-backend"
spec:
  provider: # provider field contains the configuration to access the provider
    vault:
      server: "https://your-domain:8200"
      path: "kv" # Path is the mount path of the Vault KV backend endpoint
      version: "v1"
      auth:
        tokenSecretRef: # static token: https://www.vaultproject.io/docs/auth/token
          name: "vault-token"
          key: "token"
```

> Note: A `SecretStore` file is also similar but is namespaced and maps to exactly one instance of an external API.

```bash
kubectl apply --filename secret-store.yaml
```

### 4. external-secret.yaml

This file tells the `external-secret` operator what specific data is to be fetched from your external secret store. It has three sections:

1.  Secret Store reference - This references the previous resource `ClusterSecretStore` we created. It tells external-secrets on how to access the resources.
1.  Target - This is the target secret that should be created. This defines the name and type of secret that is created in Kubernetes secret. For example PostgreSQL.
1.  DataFrom - This defines the name of the secret as it is stored in your external AWS secret store.

```yaml
apiVersion: external-secrets.io/v1alpha1
kind: ExternalSecret
metadata:
  name: "my-first-secret"
spec:
	secretStoreRef:
    name: vault-backend
    kind: SecretStore  # or ClusterSecretStore
  refreshInterval: "15s" # How often this secret is synchronized
  target: # Our target Kubernetes Secret
    name: my-first-aws-secret # If not present, then the secretKey field under data will be used
    creationPolicy: "Owner" # This will create the secret if it doesn't exist. Options are 'Owner', 'Merge', or 'None'
		deletionPolicy: "Retain"
  data:
    - secretKey: my-first-aws-secret-key
      remoteRef:
        key:message # This is the remote key in the secret provider (might change in meaning based on your provider)
        property:value # The property inside of the secret inside your secret provider
```

```yaml
kubectl apply --namespace argonaut --filename external-secret.yaml
```

Once this is done, the secrets from your external secret store are brought securely to your cluster and are available to access as Kubernetes secrets. This can be checked by executing the following command:

```yaml
kubectl --namespace argonaut get ExternalSecret my-first-secret
```

**Output:**

```
NAME              STORE           REFRESH INTERVAL   STATUS
my-first-secret   vault-backend   15s                SecretSynced
```

> Note: The secrets are now in Kubernetes and available to anyone with access to the cluster.

> Whenever you update a secret directly on your external secret store, it takes a few moments for it to reflect in your Kubernetes cluster. This refresh interval can be set as a part of your configuration.

### 5. Conclusion

You have now successfully set up [external-secrets](http://external-secrets.io). This powerful open source tool allows you to manage secrets in a more secure way. You can also choose from various other [secret manager tools](https://external-secrets.io/v0.7.2/provider/aws-secrets-manager/) like the AWS Secrets Manager or GCP Secrets Manager.

The `external-secrets` approach allows you to securely manage secrets and bring them to your cluster as needed. There are also other approaches to the same that we discuss in [this blog](https://www.argonaut.dev/blog/secret-management-in-kubernetes). `external-secrets` can do so much more like generating secrets, CRDs, controller classes, and multi-tenancy. And many more exciting things [in their roadmap](https://github.com/orgs/external-secrets/projects/2/views/1).

---

Argonaut has a native secret management solution for scaling teams. Argonaut integrations with third party secret providers is coming soon Q1CY23.

With Argonaut’s modern deployment platform, you can get up and running on AWS or GCP in minutes, not weeks. Our intuitive UI and quick integrations with GitHub, GitLab, AWS, and GCP make managing your infra and applications easy - all in one place.

[![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/pv9wlp2s7y0ne04w0xi8.png)](https://www.argonaut.dev/)