Project: Longhorn (Runtime / Cloud Native Storage)
File: 20221024-pv-encryption.md

This enhancement adds support for user configured (storage class, secrets) encrypted volumes, this in return means that backups of that volume end up also being encrypted. // TODO: create issue // TODO: create issue // TODO: create issue user is able to create & use an encrypted volume with cipher customization options user is able to configure the keys that are used for encryption user is able to take backups from an encrypted volume user is able to restore an encrypted backup to a new encrypted volume external key management support, currently keys utilize kubernetes secrets rotating key support, user can do this manually though as a workaround securing of secrets, user is responsible for cluster setup and security of the secrets All regular longhorn operations should also be supported for encrypted volumes, therefore the only user story that is mentioned is how to create and use an encrypted volume. create a storage class with (encrypted=true) and either a global secret or a per volume secret create the secret for that volume in the configured namespace with customization options of the cipher for instance `cipher`, `key-size` and `hash` create a pvc that references the created storage class volume will be created then encrypted during first use afterwards a regular filesystem that lives on top of the encrypted volume will be exposed to the pod Creation and usage of an encrypted volume requires 2 things: the storage class needs to specify `encrypted: "true"` as part of its parameters. secrets need to be created and reference for the csi operations need to be setup. see below examples for different types of secret usage. The kubernetes sidecars are responsible for retrieval of the secret and passing it to the csi driver. If the secret hasn't been created the PVC will remain in the Pending State. And the side cars will retry secret retrieval periodically, once it's available the sidecar container will call `Controller::CreateVolume` and pass the secret after which longhorn will create a volume. The below storage class uses a global secret named `longhorn-crypto` in the `longhorn-system` namespace. ```yaml kind: StorageClass apiVersion: storage.k8s.io/v1 metadata: name: longhorn-crypto-global provisioner: driver.longhorn.io allowVolumeExpansion: true parameters: numberOfReplicas: "3" staleReplicaTimeout: "2880" # 48 hours in minutes fromBackup: "" encrypted: "true" csi.storage.k8s.io/provisioner-secret-name: "longhorn-crypto" csi.storage.k8s.io/provisioner-secret-namespace: "longhorn-system" csi.storage.k8s.io/node-publish-secret-name: "longhorn-crypto" csi.storage.k8s.io/node-publish-secret-namespace: "longhorn-system" csi.storage.k8s.io/node-stage-secret-name: "longhorn-crypto" csi.storage.k8s.io/node-stage-secret-namespace: "longhorn-system" ``` The global secret reference by the `longhorn-crypto-global` storage class. This type of setup means that all volumes share the same encryption key. ```yaml apiVersion: v1 kind: Secret metadata: name: longhorn-crypto namespace: longhorn-system stringData: CRYPTOKEYVALUE: "Simple passphrase" CRYPTOKEYPROVIDER: "secret" # this is optional we currently only support direct keys via secrets CRYPTOKEYCIPHER: "aes-xts-plain64" # this is optional CRYPTOKEYHASH: "sha256" # this is optional CRYPTOKEYSIZE: "256" # this is optional ``` The below storage class uses a per volume secret, the name and namespace of the secret is based on the pvc values. These templates will be resolved by the external sidecars and the resolved values end up as Secret refs on the PV. ```yaml kind: StorageClass apiVersion: storage.k8s.io/v1 metadata: name: longhorn-crypto-per-volume provisioner: driver.longhorn.io allowVolumeExpansion: true parameters: numberOfReplicas: "3" staleReplicaTimeout: "2880" # 48 hours in minutes fromBackup: "" encrypted: "true" csi.storage.k8s.io/provisioner-secret-name: ${pvc.name} csi.storage.k8s.io/provisioner-secret-namespace: ${pvc.namespace} csi.storage.k8s.io/node-publish-secret-name: ${pvc.name} csi.storage.k8s.io/node-publish-secret-namespace: ${pvc.namespace} csi.storage.k8s.io/node-stage-secret-name: ${pvc.name} csi.storage.k8s.io/node-stage-secret-namespace: ${pvc.namespace} ``` add a `Encrypted` boolean to the `Volume` struct utilized by the http client, this ends up being stored in `Volume.Spec.encrypted` of the volume cr. Storing the `Encrypted` value is necessary to support encryption for RWX

Host requires `dm_crypt` kernel module as well as `cryptsetup` installed. We utilize the below parameters from a secret, `CRYPTOKEYPROVIDER` allows us in the future to add other key management systems `CRYPTOKEYCIPHER` allow users to choose the cipher algorithm when creating an encrypted volume by `cryptsetup` `CRYPTOKEYHASH` specifies the hash used in the LUKS key setup scheme and volume key digest `CRYPTOKEYSIZE` sets the key size in bits. The argument has to be a multiple of 8 and the maximum interactive passphrase length is 512 (characters) ```yaml CRYPTOKEYVALUE: "Simple passphrase" CRYPTOKEYPROVIDER: "secret" # this is optional we currently only support direct keys via secrets CRYPTOKEYCIPHER: "aes-xts-plain64" # this is optional CRYPTOKEYHASH: "sha256" # this is optional CRYPTOKEYSIZE: "256" # this is optional ``` utilize host `dm_crypt` kernel module for device encryption utilize host installed `cryptsetup` for configuration of the crypto device add csi driver `NodeStageVolume` support to handle device global per node mounting, we skip mounting for volumes that are being used via `VolumeMode: Block` refactor csi driver NodePublishVolume to bind mount the `stagingpath` into the `targetpath` we utilize a bind mount for `VolumeMode: Mount` we do a regular device file creation for `VolumeMode: Block` during csi `NodeStageVolume` encrypt (first time use) / open regular longhorn device this exposes a crypto mapped device (/dev/mapper/<volume-name>) mount crypto device into `staging_path` during csi `NodeUnstageVolume` unmount `staging_path` close crypto device create a storage class with (encrypted=true) and either a global secret or a per volume secret create the secret for that volume in the configured namespace create a pvc that references the created storage class create a pod that uses that pvc for a volume mount wait for pod up and healthy create a storage class with (encrypted=true) and either a global secret or a per volume secret create the secret with customized options of the cipher for that volume in the configured namespace create a pvc that references the created storage class create a pod that uses that pvc for a volume mount wait for pod up and healthy check if the customized options of the cipher are correct create a storage class with (encrypted=true) and either a global secret or a per volume secret create a pvc that references the created storage class create a pod that uses that pvc for a volume mount verify pvc remains in pending state verify pod remains in creation state create a storage class with (encrypted=true) and either a global secret or a per volume secret create the secret for that volume in the configured namespace create a pvc that references the created storage class create a pod that uses that pvc for a volume mount wait for pod up and healthy write known test pattern into fs verify absence (grep) of known test pattern after reading block device content `/dev/longhorn/<volume-name>` create a storage class with (encrypted=true) and either a global secret or a per volume secret create the secret for that volume in the configured namespace create a pvc that references the created storage class create a pod that uses that pvc for a volume mount wait for pod up and healthy scale down pod change `CRYPTOKEYVALUE` of secret scale up pod verify pod remains in pending state (failure to mount volume) requires new pvc's since encryption would overwrite the previously created filesystem. (csi driver prevents this) Host requires `dm_crypt` kernel module as well as `cryptsetup` installed. supporting external key vaults is possible in the future with some additional implementation support rotating keys is possible in the future with some additional implementation