How to Add KMS to an Azure Kubernetes Service cluster
Tags: aks, kubernetes, azure, kms
Published: 2022-05-25 19:26:29





Azure Kubernetes Service now helps Key Management System (KMS) plugin integration which permits encryption at the rest of our Kubernetes data in etcd using Azure Key Vault. We can now store secrets in carry your own key (BYOK) encrypted etcd using KMS.

KMS Plugin for Key Vault is the advocated preference for the usage of a third-party tool for key management. KMS plugin simplifies key rotation, with a new data encryption key (DEK) generated for each encryption, and key encryption key (KEK) rotation controlled by way of the user.

## Features

- Use a key in Key Vault.
- Bring your own keys.
- Provide encryption at rest for secrets stored in etcd.

## Prerequisites

- An Azure subscription. We can create a [free account](https://azure.microsoft.com/en-us/free/) if we don't have a subscription.
- [Azure CLI installed](https://docs.microsoft.com/en-us/cli/azure/install-azure-cli).

## Install the `aks-preview` Azure CLI

We also want the _aks-preview_ Azure CLI extension version 0.5.58 or later. Install the _aks-preview_ Azure CLI extension by using the [az extension add](https://docs.microsoft.com/en-us/cli/azure/extension?view=azure-cli-latest#az_extension_add) command or install any available updates by using the [az extension update](https://docs.microsoft.com/en-us/cli/azure/extension?view=azure-cli-latest#az-extension-update) command.

```
# Install the aks-preview extension
az extension add --name aks-preview
# Update the extension to make sure you have the latest version installed
az extension update --name aks-preview
```

## Register the `AzureKeyVaultKmsPreview` preview feature

To use the feature, we must also enable the `AzureKeyVaultKmsPreview` feature flag on our subscription.

Register the `AzureKeyVaultKmsPreview` feature flag by using the [az feature register](https://docs.microsoft.com/en-us/cli/azure/feature?view=azure-cli-latest#az-feature-register) command, as shown in the following example:

```
az feature register --namespace "Microsoft.ContainerService" --name "AzureKeyVaultKmsPreview"
```

It takes a few minutes for the status to show Registered. Verify the registration status by using the [az feature list](https://docs.microsoft.com/en-us/cli/azure/feature?view=azure-cli-latest#az-feature-list) command:

```
az feature list -o table --query "[?contains(name, 'Microsoft.ContainerService/AzureKeyVaultKmsPreview')].{Name:name,State:properties.state}"
```

When ready, refresh the registration of the Microsoft.ContainerService resource provider by using the [az provider register](https://docs.microsoft.com/en-us/cli/azure/provider?view=azure-cli-latest#az_provider_register) command:

```
az provider register --namespace Microsoft.ContainerService
```

## Limitations

The following obstacles observe when we integrate KMS etcd encryption with AKS:

- The KMS etcd encryption feature is disabled.

- Changing of key ID, inclusive of key name and key version.
- The Key Vault or the associated identity can be deleted.
- System-Assigned Managed Identity doesn't work with KMS etcd encryption. Before the feature is enabled, the keyvault access-policy needs to be set. System-Assigned Managed Identity isn't available until cluster creation.
- PrivateLink is enabled when using the azure key vault.
- There are more than 2000 secrets in the cluster.
- There is managed support for hsm. 
- You can bring your own key vault from another tenant.

## Create a KeyVault and key

Use `az keyvault create` to create a KeyVault.

```
az keyvault create --name kcdchennaikv --resource-group kcdchennairg
```
Use `az keyvault key create` to create a key.

```
az keyvault key create --name kcdchennaikey --vault-name kcdchennaikv
```

Use `az keyvault key show` to export the Key ID.

```
export KEY_ID=$(az keyvault key show --name kcdchennaikey --vault-name kcdchennaikv --query 'key.kid' -o tsv)
echo $KEY_ID
```



The above example stores the Key ID in _KEY_ID_.

## Create a user-assigned managed identity

Use `az identity create` to create a User-assigned managed identity.

```
az identity create --name kcdidentity --resource-group kcdchennairg
```

Use `az identity show `to get Identity Object ID.

```
IDENTITY_OBJECT_ID=$(az identity show --name kcdidentity --resource-group kcdchennairg --query 'principalId' -o tsv)
echo $IDENTITY_OBJECT_ID
```

The above example stores the value of the Identity Object ID in _IDENTITY_OBJECT_ID_.

Use `az identity show` to get Identity Resource ID.


```
IDENTITY_RESOURCE_ID=$(az identity show --name kcdidentity --resource-group kcdchennairg --query 'id' -o tsv)
echo $IDENTITY_RESOURCE_ID

```
The above example stores the value of the Identity Resource ID in _IDENTITY_RESOURCE_ID_.

## Assign permissions (decrypt and encrypt) to access key vault

Use `az keyvault set-policy` to create an Azure KeyVault policy.


```
az keyvault set-policy -n kcdchennaikv --key-permissions decrypt encrypt --object-id $IDENTITY_OBJECT_ID
```

## Create an AKS cluster with KMS etcd encryption enabled

Create an AKS cluster using the [az aks create](https://docs.microsoft.com/en-us/cli/azure/aks?view=azure-cli-latest#az-aks-create) command with the `--enable-azure-keyvault-kms` and `--azure-keyvault-kms-key-id` parameters to enable KMS etcd encryption.


```
az aks create --name kcdchnakscluster --resource-group kcdchennairg --assign-identity $IDENTITY_RESOURCE_ID --enable-azure-keyvault-kms --azure-keyvault-kms-key-id $KEY_ID
```

## Update an exiting AKS cluster to enable KMS etcd encryption

Use [az aks update](https://docs.microsoft.com/en-us/cli/azure/aks?view=azure-cli-latest#az_aks_update) with the `--enable-azure-keyvault-kms` and `--azure-keyvault-kms-key-id` parameters to enable KMS etcd encryption on an existing cluster.


```
az aks update --name kcdchnakscluster --resource-group kcdchennairg --enable-azure-keyvault-kms --azure-keyvault-kms-key-id $KEY_ID
```

Use beneath command to update all secrets. Otherwise, the old secrets aren't encrypted.

```
kubectl get secrets --all-namespaces -o json | kubectl replace -f -
```


For greater records on the use of the KMS plugin, see [Encrypting Secret Data at Rest](https://kubernetes.io/docs/tasks/administer-cluster/kms-provider/).

## Note



This feature is in a preview. On a self-service basis, preview features are available in AKS. Previews are not included in the service-level agreements and limited warranty. On a best-effort basis, AKS previews are partially covered by customer support. These features are not meant for production use.


Thanks for reading my article till end. I hope you learned some thing extraordinary today. If you loved this article then please share to your friends and if you have pointers or thoughts to share with me then please write in the comment box.


Above blog is submitted as part of 'Devtron Blogathon 2022' - https://devtron.ai/
Check out Devtron's GitHub repo - https://github.com/devtron-labs/devtron/ and give a ⭐ to show your love & support.
Follow Devtron on LinkedIn - https://www.linkedin.com/company/devtron-labs/ and Twitter - https://twitter.com/DevtronL/, to keep yourself updated on this Open Source project.