Chapter 7. Working with volume snapshots Cluster administrators can use volume snapshots to help protect against data loss by using the supported MicroShift logical volume manager storage (LVMS) Container Storage Interface (CSI) provider. Familiarity with persistent volumes is required. A snapshot represents the state of the storage volume in a cluster at a particular point in time. Volume snapshots can also be used to provision new volumes. Snapshots are created as read-only logical volumes (LVs) located on the same device as the original data. A cluster administrator can complete the following tasks using CSI volume snapshots: Create a snapshot of an existing persistent volume claim (PVC). Back up a volume snapshot to a secure location. Restore a volume snapshot as a different PVC. Delete an existing volume snapshot. Important Only the logical volume manager storage (LVMS) plugin CSI driver is supported by MicroShift. Additional resources Understanding persistent volumes Managing LVM volume groups (RHEL documentation) CSI snapshots: VolumeSnapshot APIs (OpenShift Container Platform documentation) VolumeSnapshot API specification (OpenShift Container Platform documentation) 7.1. About LVM thin volumes To use advanced storage features such as creating volume snapshots or volume cloning, you must perform the following actions: Configure both the logical volume manager storage (LVMS) provider and the cluster. Provision a logical volume manager (LVM) thin-pool on the RHEL for Edge host. Attach LVM thin-pools to a volume group. Important To create Container Storage Interface (CSI) snapshots, you must configure thin volumes on the RHEL for Edge host. The CSI does not support volume shrinking. Important When using thin provisioning, it is important that you monitor the storage pool and add more capacity as the available physical space runs out. You can configure the storage pool to auto expand when there is available space within the volume group (VG). See Creating thinly provisioned logical volumes . For LVMS to manage thin logical volumes (LVs), a thin-pool device-class array must be specified in the etc/lvmd.yaml configuration file. Multiple thin-pool device classes are permitted. If additional storage pools are configured with device classes, then additional storage classes must also exist to expose the storage pools to users and workloads. To enable dynamic provisioning on a thin-pool, a StorageClass resource must be present on the cluster. The StorageClass resource specifies the source device-class array in the topolvm.io/device-class parameter. Example lvmd.yaml file that specifies a single device class for a thin-pool socket-name: 1 device-classes: 2 - name: thin 3 default: true spare-gb: 0 4 thin-pool: name: thin overprovision-ratio: 1 5 type: thin 6 volume-group: ssd 7 1 String. The UNIX domain socket endpoint of gRPC. Defaults to /run/lvmd/lvmd.socket . 2 A list of maps for the settings for each device-class . 3 String. The unique name of the device-class . 4 Unsigned 64-bit integer. Storage capacity in GB to be left unallocated in the volume group. Defaults to 0 . 5 Specify a float factor by which you can provision additional storage based on the available storage in the thin pool. For example, if this field is set to 10, you can provision up to 10 times the amount of available storage in the thin pool. To disable over-provisioning, set this field to 1 . 6 Thin provisioning is required to create volume snapshots. 7 String. The group where the device-class creates the logical volumes. Important When multiple PVCs are created simultaneously, a race condition prevents LVMS from accurately tracking the allocated space and preserving the storage capacity for a device class. Use separate volume groups and device classes to protect the storage of highly dynamic workloads from each other. Additional resources To create a thin pool on the host, see Creating a thin logical volume (RHEL documentation) Configuring and managing thin provisioned volumes Storage classes Storage device classes 7.1.1. Storage classes Storage classes provide the workload layer interface for selecting a device class. The following storage class parameters are supported in MicroShift: The csi.storage.k8s.io/fstype parameter selects the file system types. Both xfs and ext4 file system types are supported. The topolvm.io/device-class parameter is the name of the device class. If a device class is not provided, the default device class is assumed. Multiple storage classes can refer to the same device class. You can provide varying sets of parameters for the same backing device class, such as xfs and ext4 variants. Example MicroShift default storage class resource apiVersion: storage.k8s.io/v1 kind: StorageClass metadata: annotations: storageclass.kubernetes.io/is-default-class: "true" 1 name: topolvm-provisioner parameters: "csi.storage.k8s.io/fstype": "xfs" 2 provisioner: topolvm.io 3 reclaimPolicy: Delete volumeBindingMode: WaitForFirstConsumer 4 allowVolumeExpansion: 5 1 An example of the default storage class. If a PVC does not specify a storage class, this class is assumed. There can only be one default storage class in a cluster. Having no value assigned to this annotation is also supported. 2 Specifies which file system to provision on the volume. Options are "xfs" and "ext4". 3 Identifies which provisioner should manage this class. 4 Specifies whether to provision the volume before a client pod is present or immediately. Options are WaitForFirstConsumer and Immediate . WaitForFirstConsumer is recommended to ensure that storage is only provisioned for pods that can be scheduled. 5 Specifies if PVCs provisioned from the StorageClass permit expansion. The MicroShift LVMS CSI plugin does support volume expansion, but if this value is set to false , expansion is blocked. Additional resources Defining storage classes Storage device classes 7.2. Volume snapshot classes Snapshotting is a CSI storage feature supported by LVMS. To enable dynamic snapshotting, at least one VolumeSnapshotClass configuration file must be present on the cluster. Important You must enable thin logical volumes to take logical volume snapshots. Example VolumeSnapshotClass configuration file apiVersion: snapshot.storage.k8s.io/v1 kind: VolumeSnapshotClass metadata: name: topolvm-snapclass annotations: snapshot.storage.kubernetes.io/is-default-class: "true" 1 driver: topolvm.io 2 deletionPolicy: Delete 3 1 Determines which VolumeSnapshotClass configuration file to use when none is specified by VolumeSnapshot , which is a request for snapshot of a volume by a user. 2 Identifies which snapshot provisioner should manage the requests for snapshots of a volume by a user for this class. 3 Determines whether VolumeSnapshotContent objects and the backing snapshots are kept or deleted when a bound VolumeSnapshot is deleted. Valid values are Retain or Delete . Additional resources OpenShift CSI volume snapshots 7.3. About volume snapshots You can use volume snapshots with logical volume manager (LVM) thin volumes to help protect against data loss from applications running in a MicroShift cluster. MicroShift only supports the logical volume manager storage (LVMS) Container Storage Interface (CSI) provider. Note LVMS only supports the volumeBindingMode of the storage class being set to WaitForFirstConsumer . This setting means the storage volume is not provisioned until a pod is ready to mount it. Example workload that deploys a single pod and PVC USD oc apply -f - <<EOF apiVersion: v1 kind: PersistentVolumeClaim metadata: name: test-claim-thin spec: accessModes: - ReadWriteOnce resources: requests: storage: 1Gi storageClassName: topolvm-provisioner-thin --- apiVersion: v1 kind: Pod metadata: name: base spec: containers: - command: - nginx - -g - 'daemon off;' image: registry.redhat.io/rhel8/nginx-122@sha256:908ebb0dec0d669caaf4145a8a21e04fdf9ebffbba5fd4562ce5ab388bf41ab2 name: test-container securityContext: allowPrivilegeEscalation: false capabilities: drop: - ALL volumeMounts: - mountPath: /vol name: test-vol securityContext: runAsNonRoot: true seccompProfile: type: RuntimeDefault volumes: - name: test-vol persistentVolumeClaim: claimName: test-claim-thin EOF 7.3.1. Creating a volume snapshot To create a snapshot of a MicroShift storage volume, you must first configure RHEL for Edge and the cluster. In the following example procedure, the pod that the source volume is mounted to is deleted. Deleting the pod prevents data from being written to it during snapshot creation. Ensuring that no data is being written during a snapshot is crucial to creating a viable snapshot. Prerequisites User has root access to a MicroShift cluster. A MicroShift cluster is running. A device class defines an LVM thin-pool. A volumeSnapshotClass specifies driver: topolvm.io . Any workload attached to the source PVC is paused or deleted. This helps avoid data corruption. Important All writes to the volume must be halted while you are creating the snapshot. If you do not halt writes, your data might be corrupted. Procedure Prevent data from being written to the volume during snapshotting by using one of the two following steps: Delete the pod to ensure that no data is written to the volume during snapshotting by running the following command: USD oc delete my-pod Scale the replica count to zero on a pod that is managed with a replication controller. Setting the count to zero prevents the instant creation of a new pod when one is deleted. After all writes to the volume are halted, run a command similar to the example that follows. Insert your own configuration details. Example snapshot configuration # oc apply -f <<EOF apiVersion: snapshot.storage.k8s.io/v1 kind: VolumeSnapshot 1 metadata: name: <snapshot_name> 2 spec: volumeSnapshotClassName: topolvm-snapclass 3 source: persistentVolumeClaimName: test-claim-thin 4 EOF 1 Create a VolumeSnapshot object. 2 The name that you specify for the snapshot. 3 Specify the desired name of the VolumeSnapshotClass object. 4 Specify either persistentVolumeClaimName or volumeSnapshotContentName . In this example, a snapshot is created from a PVC named test-claim-thin . Wait for the storage driver to finish creating the snapshot by running the following command: USD oc wait volumesnapshot/<snapshot_name> --for=jsonpath\='{.status.readyToUse}=true' steps When the volumeSnapshot object is in a ReadyToUse state, you can restore it as a volume for future PVCs. Restart the pod or scale the replica count back up to the desired number. After you have created the volume snapshot, you can remount the source PVC to a new pod. Important Volume snapshots are located on the same devices as the original data. To use the volume snapshots as backups, move the snapshots to a secure location. 7.3.2. Backing up a volume snapshot Snapshots of data from applications running on a MicroShift cluster are created as read-only logical volumes (LVs) located on the same devices as the original data. You must manually mount local volumes before they can be copied as persistent volumes (PVs) and used as backup copies. To use a snapshot of a MicroShift storage volume as a backup, find it on the local host and then move it to a secure location. To find specific snapshots and copy them, use the following procedure. Prerequisites You have root access to the host machine. You have an existing volume snapshot. Procedure Get the name of the volume snapshot by running the following command: USD oc get volumesnapshot -n <namespace> <snapshot_name> -o 'jsonpath={.status.volumeSnapshotContentName}' 1 1 Replace <namespace> and <snapshot_name> with the namespace and snapshot name you used. Get the unique identity of the volume created on the storage backend by using the following command and inserting the name retrieved in the step: USD oc get volumesnapshotcontent snapcontent-<retrieved_volume_identity> -o 'jsonpath={.status.snapshotHandle}' 1 1 Replace <retrieved_volume_identity> with the volume identity. Display the snapshots by using the unique identity of the volume you retrieved in the step to determine which one you want to backup by running the following command: USD sudo lvdisplay <retrieved_volume_identity> 1 1 Replace <retrieved_volume_identity> with the volume identity. Example output --- Logical volume --- LV Path /dev/rhel/732e45ff-f220-49ce-859e-87ccca26b14c LV Name 732e45ff-f220-49ce-859e-87ccca26b14c VG Name rhel LV UUID 6Ojwc0-YTfp-nKJ3-F9FO-PvMR-Ic7b-LzNGSx LV Write Access read only LV Creation host, time rhel-92.lab.local, 2023-08-07 14:45:26 -0500 LV Pool name thinpool LV Thin origin name a2d2dcdc-747e-4572-8c83-56cd873d3b07 LV Status available # open 0 LV Size 1.00 GiB Mapped size 1.04% Current LE 256 Segments 1 Allocation inherit Read ahead sectors auto - currently set to 256 Block device 253:11 Create a directory to use for mounting the LV by running the following command: USD sudo mkdir /mnt/snapshot Mount the LV using the device name for the retrieved snapshot handle by running the following command: USD sudo mount /dev/<retrieved_snapshot_handle> /mnt/snapshot 1 1 Replace <retrieved_snapshot_handle> with the device name. Copy the files from the mounted location and store them in a secure location by running the following command: USD sudo cp -r /mnt/snapshot <destination> 1 1 Replace <destination> with the path to the secure location. 7.3.3. Restoring a volume snapshot The following workflow demonstrates snapshot restoration. In this example, the verification steps are also given to ensure that data written to a source persistent volume claim (PVC) is preserved and restored on a new PVC. Important A snapshot must be restored to a PVC of exactly the same size as the source volume of the snapshot. You can resize the PVC after the snapshot is restored successfully if a larger PVC is needed. Procedure Restore a snapshot by specifying the VolumeSnapshot object as the data source in a persistent volume claim by entering the following command: USD oc apply -f <<EOF apiVersion: v1 kind: PersistentVolumeClaim metadata: name: snapshot-restore spec: accessModes: - ReadWriteOnce dataSource: apiGroup: snapshot.storage.k8s.io kind: VolumeSnapshot name: my-snap resources: requests: storage: 1Gi storageClassName: topolvm-provisioner-thin --- apiVersion: v1 kind: Pod metadata: name: base spec: containers: - command: - nginx - -g - 'daemon off;' image: registry.redhat.io/rhel8/nginx-122@sha256:908ebb0dec0d669caaf4145a8a21e04fdf9ebffbba5fd4562ce5ab388bf41ab2 name: test-container securityContext: allowPrivilegeEscalation: false capabilities: drop: - ALL volumeMounts: - mountPath: /vol name: test-vol securityContext: runAsNonRoot: true seccompProfile: type: RuntimeDefault volumes: - name: test-vol persistentVolumeClaim: claimName: snapshot-restore EOF Verification Wait for the pod to reach the Ready state: USD oc wait --for=condition=Ready pod/base When the new pod is ready, verify that the data from your application is correct in the snapshot. Additional resources Restoring a volume snapshot 7.3.4. Deleting a volume snapshot You can configure how Red Hat build of MicroShift deletes volume snapshots. Procedure Specify the deletion policy that you require in the VolumeSnapshotClass object, as shown in the following example: volumesnapshotclass.yaml apiVersion: snapshot.storage.k8s.io/v1 kind: VolumeSnapshotClass metadata: name: csi-hostpath-snap driver: hostpath.csi.k8s.io deletionPolicy: Delete 1 1 When deleting the volume snapshot, if the Delete value is set, the underlying snapshot is deleted along with the VolumeSnapshotContent object. If the Retain value is set, both the underlying snapshot and VolumeSnapshotContent object remain. If the Retain value is set and the VolumeSnapshot object is deleted without deleting the corresponding VolumeSnapshotContent object, the content remains. The snapshot itself is also retained in the storage back end. Delete the volume snapshot by entering the following command: USD oc delete volumesnapshot <volumesnapshot_name> 1 1 Replace <volumesnapshot_name> with the name of the volume snapshot you want to delete. Example output volumesnapshot.snapshot.storage.k8s.io "mysnapshot" deleted If the deletion policy is set to Retain , delete the volume snapshot content by entering the following command: USD oc delete volumesnapshotcontent <volumesnapshotcontent_name> 1 1 Replace <volumesnapshotcontent_name> with the content you want to delete. Optional: If the VolumeSnapshot object is not successfully deleted, enter the following command to remove any finalizers for the leftover resource so that the delete operation can continue: Important Only remove the finalizers if you are confident that there are no existing references from either persistent volume claims or volume snapshot contents to the VolumeSnapshot object. Even with the --force option, the delete operation does not delete snapshot objects until all finalizers are removed. USD oc patch -n USDPROJECT volumesnapshot/USDNAME --type=merge -p '{"metadata": {"finalizers":null}}' Example output volumesnapshotclass.snapshot.storage.k8s.io "csi-ocs-rbd-snapclass" deleted The finalizers are removed and the volume snapshot is deleted. 7.4. About LVM volume cloning The logical volume manager storage (LVMS) supports persistent volume claim (PVC) cloning for logical volume manager (LVM) thin volumes. A clone is a duplicate of an existing volume that can be used like any other volume. When provisioned, an exact duplicate of the original volume is created if the data source references a source PVC in the same namespace. After a cloned PVC is created, it is considered a new object and completely separate from the source PVC. The clone represents the data from the source at the moment in time it was created. Note Cloning is only possible when the source and destination PVCs are in the same namespace. To create PVC clones, you must configure thin volumes on the RHEL for Edge host. Additional resources CSI volume cloning To configure the host to enable cloning, see About LVM thin volumes

Commands:
socket-name: 1 device-classes: 2 - name: thin 3 default: true spare-gb: 0 4 thin-pool: name: thin overprovision-ratio: 1 5 type: thin 6 volume-group: ssd 7
apiVersion: storage.k8s.io/v1 kind: StorageClass metadata: annotations: storageclass.kubernetes.io/is-default-class: "true" 1 name: topolvm-provisioner parameters: "csi.storage.k8s.io/fstype": "xfs" 2 provisioner: topolvm.io 3 reclaimPolicy: Delete volumeBindingMode: WaitForFirstConsumer 4 allowVolumeExpansion: 5
apiVersion: snapshot.storage.k8s.io/v1 kind: VolumeSnapshotClass metadata: name: topolvm-snapclass annotations: snapshot.storage.kubernetes.io/is-default-class: "true" 1 driver: topolvm.io 2 deletionPolicy: Delete 3
oc apply -f - <<EOF apiVersion: v1 kind: PersistentVolumeClaim metadata: name: test-claim-thin spec: accessModes: - ReadWriteOnce resources: requests: storage: 1Gi storageClassName: topolvm-provisioner-thin --- apiVersion: v1 kind: Pod metadata: name: base spec: containers: - command: - nginx - -g - 'daemon off;' image: registry.redhat.io/rhel8/nginx-122@sha256:908ebb0dec0d669caaf4145a8a21e04fdf9ebffbba5fd4562ce5ab388bf41ab2 name: test-container securityContext: allowPrivilegeEscalation: false capabilities: drop: - ALL volumeMounts: - mountPath: /vol name: test-vol securityContext: runAsNonRoot: true seccompProfile: type: RuntimeDefault volumes: - name: test-vol persistentVolumeClaim: claimName: test-claim-thin EOF
oc delete my-pod
oc apply -f <<EOF apiVersion: snapshot.storage.k8s.io/v1 kind: VolumeSnapshot 1 metadata: name: <snapshot_name> 2 spec: volumeSnapshotClassName: topolvm-snapclass 3 source: persistentVolumeClaimName: test-claim-thin 4 EOF
oc wait volumesnapshot/<snapshot_name> --for=jsonpath\='{.status.readyToUse}=true'
oc get volumesnapshot -n <namespace> <snapshot_name> -o 'jsonpath={.status.volumeSnapshotContentName}' 1
oc get volumesnapshotcontent snapcontent-<retrieved_volume_identity> -o 'jsonpath={.status.snapshotHandle}' 1
sudo lvdisplay <retrieved_volume_identity> 1
--- Logical volume --- LV Path /dev/rhel/732e45ff-f220-49ce-859e-87ccca26b14c LV Name 732e45ff-f220-49ce-859e-87ccca26b14c VG Name rhel LV UUID 6Ojwc0-YTfp-nKJ3-F9FO-PvMR-Ic7b-LzNGSx LV Write Access read only LV Creation host, time rhel-92.lab.local, 2023-08-07 14:45:26 -0500 LV Pool name thinpool LV Thin origin name a2d2dcdc-747e-4572-8c83-56cd873d3b07 LV Status available open 0 LV Size 1.00 GiB Mapped size 1.04% Current LE 256 Segments 1 Allocation inherit Read ahead sectors auto - currently set to 256 Block device 253:11
sudo mkdir /mnt/snapshot
sudo mount /dev/<retrieved_snapshot_handle> /mnt/snapshot 1
sudo cp -r /mnt/snapshot <destination> 1
oc apply -f <<EOF apiVersion: v1 kind: PersistentVolumeClaim metadata: name: snapshot-restore spec: accessModes: - ReadWriteOnce dataSource: apiGroup: snapshot.storage.k8s.io kind: VolumeSnapshot name: my-snap resources: requests: storage: 1Gi storageClassName: topolvm-provisioner-thin --- apiVersion: v1 kind: Pod metadata: name: base spec: containers: - command: - nginx - -g - 'daemon off;' image: registry.redhat.io/rhel8/nginx-122@sha256:908ebb0dec0d669caaf4145a8a21e04fdf9ebffbba5fd4562ce5ab388bf41ab2 name: test-container securityContext: allowPrivilegeEscalation: false capabilities: drop: - ALL volumeMounts: - mountPath: /vol name: test-vol securityContext: runAsNonRoot: true seccompProfile: type: RuntimeDefault volumes: - name: test-vol persistentVolumeClaim: claimName: snapshot-restore EOF
oc wait --for=condition=Ready pod/base
apiVersion: snapshot.storage.k8s.io/v1 kind: VolumeSnapshotClass metadata: name: csi-hostpath-snap driver: hostpath.csi.k8s.io deletionPolicy: Delete 1
oc delete volumesnapshot <volumesnapshot_name> 1
volumesnapshot.snapshot.storage.k8s.io "mysnapshot" deleted
oc delete volumesnapshotcontent <volumesnapshotcontent_name> 1
oc patch -n USDPROJECT volumesnapshot/USDNAME --type=merge -p '{"metadata": {"finalizers":null}}'
volumesnapshotclass.snapshot.storage.k8s.io "csi-ocs-rbd-snapclass" deleted