Project: Rook (Runtime / Cloud Native Storage)
File: key-management-system.md

title: Key Management System <!-- markdownlint-disable MD024 --> <!-- allow duplicate headers in this file --> Rook has the ability to encrypt OSDs of clusters running on PVC via the flag (`encrypted: true`) in your `storageClassDeviceSets` . Rook also has the ability to rotate encryption keys of OSDs using a cron job per OSD. By default, the Key Encryption Keys (also known as Data Encryption Keys) are stored in a Kubernetes Secret. However, if a Key Management System exists Rook is capable of using it. The `security` section contains settings related to encryption of the cluster. `security`: `kms`: Key Management System settings `connectionDetails`: the list of parameters representing kms connection details `tokenSecretName`: the name of the Kubernetes Secret containing the kms authentication token `keyRotation`: Key Rotation settings `enabled`: whether key rotation is enabled or not, default is `false` `schedule`: the schedule, written in , with which key rotation is created, default value is `"@weekly"`. !!! note Currently key rotation is only supported for the default type, where the Key Encryption Keys are stored in a Kubernetes Secret. Supported KMS providers: - - - - - - Rook supports storing OSD encryption keys in . Rook support two authentication methods: : a token is provided by the user and is stored in a Kubernetes Secret. It's used to authenticate the KMS by the Rook operator. This has several pitfalls such as: when the token expires it must be renewed, so the secret holding it must be updated no token automatic rotation uses [Vault Kubernetes native authentication](https://www.vaultproject.io/docs/auth/kubernetes) mechanism and alleviate some of the limitations from the token authentication such as token automatic renewal. This method is generally recommended over the token-based authentication. When using the token-based authentication, a Kubernetes Secret must be created to hold the token. This is governed by the `tokenSecretName` parameter. Note: Rook supports all the Vault . The Kubernetes Secret `rook-vault-token` should contain: ```yaml apiVersion: v1 kind: Secret metadata: name: rook-vault-token namespace: rook-ceph data: token: <TOKEN> # base64 of a token to connect to Vault, for example: cy5GWXpsbzAyY2duVGVoRjhkWG5Bb3EyWjkK ``` You can create a token in Vault by running the following command: ```console vault token create -policy=rook ``` Refer to the official vault document for more details on [how to create a token](https://www.vaultproject.io/docs/commands/token/create). For which policy to apply see the next section. In order for Rook to connect to Vault, you must configure the following in your `CephCluster` template: ```yaml security: kms: connectionDetails: KMS_PROVIDER: vault VAULT_ADDR:

VAULTBACKENDPATH: rook VAULTSECRETENGINE: kv VAULTAUTHMETHOD: token tokenSecretName: rook-vault-token ``` In order to use the Kubernetes Service Account authentication method, the following must be run to properly configure Vault: ```console ROOK_NAMESPACE=rook-ceph ROOKVAULTSA=rook-vault-auth ROOKSYSTEMSA=rook-ceph-system ROOKOSDSA=rook-ceph-osd VAULTPOLICYNAME=rook kubectl -n "$ROOKNAMESPACE" create serviceaccount "$ROOKVAULT_SA" kubectl -n "$ROOKNAMESPACE" create clusterrolebinding vault-tokenreview-binding --clusterrole=system:auth-delegator --serviceaccount="$ROOKNAMESPACE":"$ROOKVAULTSA" VAULTSASECRETNAME=$(kubectl -n "$ROOKNAMESPACE" get sa "$ROOKVAULTSA" -o jsonpath="{.secrets}") SAJWTTOKEN=$(kubectl -n "$ROOKNAMESPACE" get secret "$VAULTSASECRETNAME" -o jsonpath="{.data.token}" | base64 --decode) SACACRT=$(kubectl -n "$ROOKNAMESPACE" get secret "$VAULTSASECRETNAME" -o jsonpath="{.data['ca\.crt']}" | base64 --decode) K8S_HOST=$(kubectl config view --minify --flatten -o jsonpath="{.clusters[0].cluster.server}") vault auth enable kubernetes kubectl proxy & proxy_pid=$! vault write auth/kubernetes/config \ tokenreviewerjwt="$SAJWTTOKEN" \ kuberneteshost="$K8SHOST" \ kubernetescacert="$SACACRT" \ issuer="$(curl --silent http://127.0.0.1:8001/.well-known/openid-configuration | jq -r .issuer)" kill $proxy_pid vault write auth/kubernetes/role/"$ROOK_NAMESPACE" \ boundserviceaccountnames="$ROOKSYSTEMSA","$ROOKOSD_SA" \ boundserviceaccountnamespaces="$ROOKNAMESPACE" \ policies="$VAULTPOLICYNAME" \ ttl=1440h ``` Once done, your `CephCluster` CR should look like: ```yaml security: kms: connectionDetails: KMS_PROVIDER: vault VAULT_ADDR: https://vault.default.svc.cluster.local:8200 VAULTBACKENDPATH: rook VAULTSECRETENGINE: kv VAULTAUTHMETHOD: kubernetes VAULTAUTHKUBERNETES_ROLE: rook-ceph ``` !!! note The `VAULT_ADDR` value above assumes that Vault is accessible within the cluster itself on the default port (8200). If running elsewhere, please update the URL accordingly. As part of the token, here is an example of a policy that can be used: ```hcl path "rook/*" { capabilities = ["create", "read", "update", "delete", "list"] } path "sys/mounts" { capabilities = ["read"] } ``` You can write the policy like so and then create a token: ```console $ vault policy write rook /tmp/rook.hcl $ vault token create -policy=rook Key Value -- token s.FYzlo02cgnTehF8dXnAoq2Z9 token_accessor oMo7sAXQKbYtxU4HtO8k3pko token_duration 768h token_renewable true token_policies ["default" "rook"] identity_policies [] policies ["default" "rook"] ``` In the above example, Vault's secret backend path name is `rook`. It must be enabled with the following: ```console vault secrets enable -path=rook kv ``` If a different path is used, the `VAULTBACKENDPATH` key in `connectionDetails` must be changed. This is an advanced but recommended configuration for production deployments, in this case the `vault-connection-details` will look like: ```yaml security: kms: connectionDetails: KMS_PROVIDER: vault VAULT_ADDR: https://vault.default.svc.cluster.local:8200 VAULT_CACERT: <name of the k8s secret containing the PEM-encoded CA certificate> VAULTCLIENTCERT: <name of the k8s secret containing the PEM-encoded client certificate> VAULTCLIENTKEY: <name of the k8s secret containing the PEM-encoded private key> tokenSecretName: rook-vault-token ``` Each secret keys are expected to be: VAULT_CACERT: `cert` VAULTCLIENTCERT: `cert` VAULTCLIENTKEY: `key` For instance `VAULT_CACERT` Secret named `vault-tls-ca-certificate` will look like: ```yaml apiVersion: v1 kind: Secret metadata: name: vault-tls-ca-certificate namespace: rook-ceph data: cert: <PEM base64 encoded CA certificate> ``` Note: if you are using self-signed certificates (not known/approved by a proper CA) you must pass `VAULTSKIPVERIFY: true`. Communications will remain encrypted but the validity of the certificate will not be verified. Rook supports storing OSD encryption keys in [IBM Key Protect](https://www.ibm.com/cloud/key-protect). The current implementation stores OSD encryption keys as using the [Bring Your Own Key](https://cloud.ibm.com/docs/key-protect?topic=key-protect-importing-keys) (BYOK) method. This means that the Key Protect instance policy must have Standard Imported Key enabled. First, you need to on the IBM Cloud. Once completed,

Make a record of it; we need it in the CRD. On the IBM Cloud, the user must create a Service ID, then assign an Access Policy to this service. Ultimately, a Service API Key needs to be generated. All the steps are summarized in the [official documentation](https://www.ibm.com/docs/en/cloud-private/3.2.0?topic=dg-creating-service-id-by-using-cloud-private-management-console). The Service ID must be granted access to the Key Protect Service. Once the Service API Key is generated, store it in a Kubernetes Secret. ```yaml apiVersion: v1 kind: Secret metadata: name: ibm-kp-svc-api-key namespace: rook-ceph data: IBMKPSERVICEAPIKEY: <service API Key> ``` In order for Rook to connect to IBM Key Protect, you must configure the following in your `CephCluster` template: ```yaml security: kms: connectionDetails: KMS_PROVIDER: ibmkeyprotect IBMKPSERVICEINSTANCEID: <instance ID that was retrieved in the first paragraph> tokenSecretName: ibm-kp-svc-api-key ``` More options are supported such as: `IBMBASEURL`: the base URL of the Key Protect instance, depending on your . Defaults to `https://us-south.kms.cloud.ibm.com`. `IBMTOKENURL`: the URL of the Key Protect instance to retrieve the token. Defaults to `https://iam.cloud.ibm.com/oidc/token`. Only needed for private instances. Rook supports storing OSD encryption keys in supported KMS. The current implementation stores OSD encryption keys using the operation. Key is fetched and deleted using and operations respectively. The Secret with credentials for the KMIP KMS is expected to contain the following. ```yaml apiVersion: v1 kind: Secret metadata: name: kmip-credentials namespace: rook-ceph stringData: CA_CERT: <ca certificate> CLIENT_CERT: <client certificate> CLIENT_KEY: <client key> ``` In order for Rook to connect to KMIP, you must configure the following in your `CephCluster` template: ```yaml security: kms: connectionDetails: KMS_PROVIDER: kmip KMIP_ENDPOINT: <KMIP endpoint address> TLSSERVERNAME: <tls server name> READ_TIMEOUT: <read timeout> WRITE_TIMEOUT: <write timeout> tokenSecretName: kmip-credentials ``` Rook supports storing OSD encryption keys in Different methods are available in Azure to authenticate a client. Rook supports Azure recommended method of authentication with Service Principal and a certificate. Refer the following Azure documentation to set up key vault and authenticate it via service principal and certtificate * `AZUREVAULTURL` can be retrieved at this step * `AZURECLIENTID` and `AZURETENANTID` can be obtained after creating the service principal Ensure that the service principal is authenticated with a certificate and not with a client secret. * Ensure that the role assigned to the key vault should be able to create, retrieve and delete secrets in the key vault. Provide the following KMS connection details in order to connect with Azure Key Vault. ```yaml security: kms: connectionDetails: KMS_PROVIDER: azure-kv AZUREVAULTURL: https://<key-vault name>.vault.azure.net AZURECLIENTID: Application ID of an Azure service principal AZURETENANTID: ID of the application's Microsoft Entra tenant AZURECERTSECRET_NAME: <name of the k8s secret containing the certificate along with the private key (without password protection)> ``` `AZURECERTSECRET_NAME` should hold the name of the k8s secret. The secret data should be base64 encoded certificate along with private key (without password protection)