Chapter 7. Advanced VM creation 7.1. Creating VMs in the web console 7.1.1. Creating virtual machines from Red Hat images overview Red Hat images are golden images . They are published as container disks in a secure registry. The Containerized Data Importer (CDI) polls and imports the container disks into your cluster and stores them in the openshift-virtualization-os-images project as snapshots or persistent volume claims (PVCs). You can optionally use a custom namespace for golden images. Red Hat images are automatically updated. You can disable and re-enable automatic updates for these images. See Managing Red Hat boot source updates . Cluster administrators can enable automatic subscription for Red Hat Enterprise Linux (RHEL) virtual machines in the OpenShift Virtualization web console . You can create virtual machines (VMs) from operating system images provided by Red Hat by using one of the following methods: Creating a VM from a template by using the web console Creating a VM from an instance type by using the web console Creating a VM from a VirtualMachine manifest by using the command line Important Do not create VMs in the default openshift-* namespaces. Instead, create a new namespace or use an existing namespace without the openshift prefix. 7.1.1.1. About golden images A golden image is a preconfigured snapshot of a virtual machine (VM) that you can use as a resource to deploy new VMs. For example, you can use golden images to provision the same system environment consistently and deploy systems more quickly and efficiently. 7.1.1.1.1. How do golden images work? Golden images are created by installing and configuring an operating system and software applications on a reference machine or virtual machine. This includes setting up the system, installing required drivers, applying patches and updates, and configuring specific options and preferences. After the golden image is created, it is saved as a template or image file that can be replicated and deployed across multiple clusters. The golden image can be updated by its maintainer periodically to incorporate necessary software updates and patches, ensuring that the image remains up to date and secure, and newly created VMs are based on this updated image. 7.1.1.1.2. Red Hat implementation of golden images Red Hat publishes golden images as container disks in the registry for versions of Red Hat Enterprise Linux (RHEL). Container disks are virtual machine images that are stored as a container image in a container image registry. Any published image will automatically be made available in connected clusters after the installation of OpenShift Virtualization. After the images are available in a cluster, they are ready to use to create VMs. 7.1.1.2. About VM boot sources Virtual machines (VMs) consist of a VM definition and one or more disks that are backed by data volumes. VM templates enable you to create VMs using predefined specifications. Every template requires a boot source, which is a fully configured disk image including configured drivers. Each template contains a VM definition with a pointer to the boot source. Each boot source has a predefined name and namespace. For some operating systems, a boot source is automatically provided. If it is not provided, then an administrator must prepare a custom boot source. Provided boot sources are updated automatically to the latest version of the operating system. For auto-updated boot sources, persistent volume claims (PVCs) and volume snapshots are created with the cluster's default storage class. If you select a different default storage class after configuration, you must delete the existing boot sources in the cluster namespace that are configured with the default storage class. 7.1.1.3. Configuring a custom namespace for golden images The default namespace for golden images is openshift-virtualization-os-images , but you can configure a custom namespace to restrict user access to the default boot sources. 7.1.1.3.1. Configuring a custom namespace for golden images by using the web console You can configure a custom namespace for golden images in your cluster by using the Red Hat OpenShift Service on AWS web console. Procedure In the web console, select Virtualization Overview . Select the Settings tab. On the Cluster tab, select General settings Bootable volumes project . Select a namespace to use for golden images. If you already created a namespace, select it from the Project list. If you did not create a namespace, scroll to the bottom of the list and click Create project . Enter a name for your new namespace in the Name field of the Create project dialog. Click Create . 7.1.1.3.2. Configuring a custom namespace for golden images by using the CLI You can configure a custom namespace for golden images in your cluster by setting the spec.commonBootImageNamespace field in the HyperConverged custom resource (CR). Prerequisites You installed the OpenShift CLI ( oc ). You created a namespace to use for golden images. Procedure Open the HyperConverged CR in your default editor by running the following command: USD oc edit hyperconverged kubevirt-hyperconverged -n openshift-cnv Configure the custom namespace by updating the value of the spec.commonBootImageNamespace field: Example configuration file apiVersion: hco.kubevirt.io/v1 kind: HyperConverged metadata: name: kubevirt-hyperconverged namespace: openshift-cnv spec: commonBootImageNamespace: <custom_namespace> 1 # ... 1 The namespace to use for golden images. Save your changes and exit the editor. 7.1.2. Creating VMs by importing images from web pages You can create virtual machines (VMs) by importing operating system images from web pages. Important You must install the QEMU guest agent on VMs created from operating system images that are not provided by Red Hat. 7.1.2.1. Creating a VM from an image on a web page by using the web console You can create a virtual machine (VM) by importing an image from a web page by using the Red Hat OpenShift Service on AWS web console. Prerequisites You must have access to the web page that contains the image. Procedure Navigate to Virtualization Catalog in the web console. Click a template tile without an available boot source. Click Customize VirtualMachine . On the Customize template parameters page, expand Storage and select URL (creates PVC) from the Disk source list. Enter the image URL. Example: https://access.redhat.com/downloads/content/69/ver=/rhel---7/7.9/x86_64/product-software Set the disk size. Click . Click Create VirtualMachine . 7.1.2.2. Creating a VM from an image on a web page by using the command line You can create a virtual machine (VM) from an image on a web page by using the command line. When the virtual machine (VM) is created, the data volume with the image is imported into persistent storage. Prerequisites You must have access credentials for the web page that contains the image. Procedure Edit the VirtualMachine manifest and save it as a vm-rhel-datavolume.yaml file: apiVersion: kubevirt.io/v1 kind: VirtualMachine metadata: creationTimestamp: null name: vm-rhel-datavolume 1 labels: kubevirt.io/vm: vm-rhel-datavolume spec: dataVolumeTemplates: - metadata: creationTimestamp: null name: rhel-dv 2 spec: sourceRef: kind: DataSource name: rhel9 namespace: openshift-virtualization-os-images storage: resources: requests: storage: 10Gi 3 instancetype: name: u1.small 4 preference: inferFromVolume: datavolumedisk1 runStrategy: Always template: metadata: creationTimestamp: null labels: kubevirt.io/vm: vm-rhel-datavolume spec: domain: devices: {} resources: {} terminationGracePeriodSeconds: 180 volumes: - dataVolume: name: rhel-dv name: datavolumedisk1 status: {} 1 Specify the name of the VM. 2 Specify the name of the data volume. 3 Specify the size of the storage requested for the data volume. 4 Optional: Specify the instance type to use to control resource sizing of the VM. Create the VM by running the following command: USD oc create -f vm-rhel-datavolume.yaml The oc create command creates the data volume and the VM. The CDI controller creates an underlying PVC with the correct annotation and the import process begins. When the import is complete, the data volume status changes to Succeeded . You can start the VM. Data volume provisioning happens in the background, so there is no need to monitor the process. Verification The importer pod downloads the image from the specified URL and stores it on the provisioned persistent volume. View the status of the importer pod by running the following command: USD oc get pods Monitor the data volume until its status is Succeeded by running the following command: USD oc describe dv rhel-dv 1 1 Specify the data volume name that you defined in the VirtualMachine manifest. Verify that provisioning is complete and that the VM has started by accessing its serial console: USD virtctl console vm-rhel-datavolume 7.1.3. Creating VMs by uploading images You can create virtual machines (VMs) by uploading operating system images from your local machine. You can create a Windows VM by uploading a Windows image to a PVC. Then you clone the PVC when you create the VM. Important You must install the QEMU guest agent on VMs created from operating system images that are not provided by Red Hat. You must also install VirtIO drivers on Windows VMs. 7.1.3.1. Creating a VM from an uploaded image by using the web console You can create a virtual machine (VM) from an uploaded operating system image by using the Red Hat OpenShift Service on AWS web console. Prerequisites You must have an IMG , ISO , or QCOW2 image file. Procedure Navigate to Virtualization Catalog in the web console. Click a template tile without an available boot source. Click Customize VirtualMachine . On the Customize template parameters page, expand Storage and select Upload (Upload a new file to a PVC) from the Disk source list. Browse to the image on your local machine and set the disk size. Click Customize VirtualMachine . Click Create VirtualMachine . 7.1.3.1.1. Generalizing a VM image You can generalize a Red Hat Enterprise Linux (RHEL) image to remove all system-specific configuration data before you use the image to create a golden image, a preconfigured snapshot of a virtual machine (VM). You can use a golden image to deploy new VMs. You can generalize a RHEL VM by using the virtctl , guestfs , and virt-sysprep tools. Prerequisites You have a RHEL virtual machine (VM) to use as a base VM. You have installed the OpenShift CLI ( oc ). You have installed the virtctl tool. Procedure Stop the RHEL VM if it is running, by entering the following command: USD virtctl stop <my_vm_name> Optional: Clone the virtual machine to avoid losing the data from your original VM. You can then generalize the cloned VM. Retrieve the dataVolume that stores the root filesystem for the VM by running the following command: USD oc get vm <my_vm_name> -o jsonpath="{.spec.template.spec.volumes}{'\n'}" Example output [{"dataVolume":{"name":"<my_vm_volume>"},"name":"rootdisk"},{"cloudInitNoCloud":{...}] Retrieve the persistent volume claim (PVC) that matches the listed dataVolume by running the followimg command: USD oc get pvc Example output NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE <my_vm_volume> Bound ... Note If your cluster configuration does not enable you to clone a VM, to avoid losing the data from your original VM, you can clone the VM PVC to a data volume instead. You can then use the cloned PVC to create a golden image. If you are creating a golden image by cloning a PVC, continue with the steps, using the cloned PVC. Deploy a new interactive container with libguestfs-tools and attach the PVC to it by running the following command: USD virtctl guestfs <my-vm-volume> --uid 107 This command opens a shell for you to run the command. Remove all configurations specific to your system by running the following command: USD virt-sysprep -a disk.img In the Red Hat OpenShift Service on AWS console, click Virtualization Catalog . Click Add volume . In the Add volume window: From the Source type list, select Use existing Volume . From the Volume project list, select your project. From the Volume name list, select the correct PVC. In the Volume name field, enter a name for the new golden image. From the Preference list, select the RHEL version you are using. From the Default Instance Type list, select the instance type with the correct CPU and memory requirements for the version of RHEL you selected previously. Click Save . The new volume appears in the Select volume to boot from list. This is your new golden image. You can use this volume to create new VMs. Additional resources for generalizing VMs Cloning VMs Cloning a PVC to a data volume 7.1.3.2. Creating a Windows VM You can create a Windows virtual machine (VM) by uploading a Windows image to a persistent volume claim (PVC) and then cloning the PVC when you create a VM by using the Red Hat OpenShift Service on AWS web console. Prerequisites You created a Windows installation DVD or USB with the Windows Media Creation Tool. See Create Windows 10 installation media in the Microsoft documentation. You created an autounattend.xml answer file. See Answer files (unattend.xml) in the Microsoft documentation. Procedure Upload the Windows image as a new PVC: Navigate to Storage PersistentVolumeClaims in the web console. Click Create PersistentVolumeClaim With Data upload form . Browse to the Windows image and select it. Enter the PVC name, select the storage class and size and then click Upload . The Windows image is uploaded to a PVC. Configure a new VM by cloning the uploaded PVC: Navigate to Virtualization Catalog . Select a Windows template tile and click Customize VirtualMachine . Select Clone (clone PVC) from the Disk source list. Select the PVC project, the Windows image PVC, and the disk size. Apply the answer file to the VM: Click Customize VirtualMachine parameters . On the Sysprep section of the Scripts tab, click Edit . Browse to the autounattend.xml answer file and click Save . Set the run strategy of the VM: Clear Start this VirtualMachine after creation so that the VM does not start immediately. Click Create VirtualMachine . On the YAML tab, replace running:false with runStrategy: RerunOnFailure and click Save . Click the Options menu and select Start . The VM boots from the sysprep disk containing the autounattend.xml answer file. 7.1.3.2.1. Generalizing a Windows VM image You can generalize a Windows operating system image to remove all system-specific configuration data before you use the image to create a new virtual machine (VM). Before generalizing the VM, you must ensure the sysprep tool cannot detect an answer file after the unattended Windows installation. Prerequisites A running Windows VM with the QEMU guest agent installed. Procedure In the Red Hat OpenShift Service on AWS console, click Virtualization VirtualMachines . Select a Windows VM to open the VirtualMachine details page. Click Configuration Disks . Click the Options menu beside the sysprep disk and select Detach . Click Detach . Rename C:\Windows\Panther\unattend.xml to avoid detection by the sysprep tool. Start the sysprep program by running the following command: %WINDIR%\System32\Sysprep\sysprep.exe /generalize /shutdown /oobe /mode:vm After the sysprep tool completes, the Windows VM shuts down. The disk image of the VM is now available to use as an installation image for Windows VMs. You can now specialize the VM. 7.1.3.2.2. Specializing a Windows VM image Specializing a Windows virtual machine (VM) configures the computer-specific information from a generalized Windows image onto the VM. Prerequisites You must have a generalized Windows disk image. You must create an unattend.xml answer file. See the Microsoft documentation for details. Procedure In the Red Hat OpenShift Service on AWS console, click Virtualization Catalog . Select a Windows template and click Customize VirtualMachine . Select PVC (clone PVC) from the Disk source list. Select the PVC project and PVC name of the generalized Windows image. Click Customize VirtualMachine parameters . Click the Scripts tab. In the Sysprep section, click Edit , browse to the unattend.xml answer file, and click Save . Click Create VirtualMachine . During the initial boot, Windows uses the unattend.xml answer file to specialize the VM. The VM is now ready to use. Additional resources for creating Windows VMs Microsoft, Sysprep (Generalize) a Windows installation Microsoft, generalize Microsoft, specialize 7.1.3.3. Creating a VM from an uploaded image by using the command line You can upload an operating system image by using the virtctl command line tool. You can use an existing data volume or create a new data volume for the image. Prerequisites You must have an ISO , IMG , or QCOW2 operating system image file. For best performance, compress the image file by using the virt-sparsify tool or the xz or gzip utilities. You must have virtctl installed. The client machine must be configured to trust the Red Hat OpenShift Service on AWS router's certificate. Procedure Upload the image by running the virtctl image-upload command: USD virtctl image-upload dv <datavolume_name> \ 1 --size=<datavolume_size> \ 2 --image-path=</path/to/image> \ 3 1 The name of the data volume. 2 The size of the data volume. For example: --size=500Mi , --size=1G 3 The file path of the image. Note If you do not want to create a new data volume, omit the --size parameter and include the --no-create flag. When uploading a disk image to a PVC, the PVC size must be larger than the size of the uncompressed virtual disk. To allow insecure server connections when using HTTPS, use the --insecure parameter. When you use the --insecure flag, the authenticity of the upload endpoint is not verified. Optional. To verify that a data volume was created, view all data volumes by running the following command: USD oc get dvs 7.1.4. Cloning VMs You can clone virtual machines (VMs) or create new VMs from snapshots. Important Cloning a VM with a vTPM device attached to it or creating a new VM from its snapshot is not supported. 7.1.4.1. Cloning a VM by using the web console You can clone an existing VM by using the web console. Procedure Navigate to Virtualization VirtualMachines in the web console. Select a VM to open the VirtualMachine details page. Click Actions . Select Clone . On the Clone VirtualMachine page, enter the name of the new VM. (Optional) Select the Start cloned VM checkbox to start the cloned VM. Click Clone . 7.1.4.2. Creating a VM from an existing snapshot by using the web console You can create a new VM by copying an existing snapshot. Procedure Navigate to Virtualization VirtualMachines in the web console. Select a VM to open the VirtualMachine details page. Click the Snapshots tab. Click the Options menu for the snapshot you want to copy. Select Create VirtualMachine . Enter the name of the virtual machine. (Optional) Select the Start this VirtualMachine after creation checkbox to start the new virtual machine. Click Create . 7.1.4.3. Additional resources Creating VMs by cloning PVCs 7.2. Creating VMs using the CLI 7.2.1. Creating virtual machines from the command line You can create virtual machines (VMs) from the command line by editing or creating a VirtualMachine manifest. You can simplify VM configuration by using an instance type in your VM manifest. Note You can also create VMs from instance types by using the web console . 7.2.1.1. Creating manifests by using the virtctl tool You can use the virtctl CLI utility to simplify creating manifests for VMs, VM instance types, and VM preferences. For more information, see VM manifest creation commands . 7.2.1.2. Creating a VM from a VirtualMachine manifest You can create a virtual machine (VM) from a VirtualMachine manifest. Procedure Edit the VirtualMachine manifest for your VM. The following example configures a Red Hat Enterprise Linux (RHEL) VM: Note This example manifest does not configure VM authentication. Example manifest for a RHEL VM apiVersion: kubevirt.io/v1 kind: VirtualMachine metadata: name: rhel-9-minimal spec: dataVolumeTemplates: - metadata: name: rhel-9-minimal-volume spec: sourceRef: kind: DataSource name: rhel9 1 namespace: openshift-virtualization-os-images 2 storage: {} instancetype: name: u1.medium 3 preference: name: rhel.9 4 runStrategy: Always template: spec: domain: devices: {} volumes: - dataVolume: name: rhel-9-minimal-volume name: rootdisk 1 The rhel9 golden image is used to install RHEL 9 as the guest operating system. 2 Golden images are stored in the openshift-virtualization-os-images namespace. 3 The u1.medium instance type requests 1 vCPU and 4Gi memory for the VM. These resource values cannot be overridden within the VM. 4 The rhel.9 preference specifies additional attributes that support the RHEL 9 guest operating system. Create a virtual machine by using the manifest file: USD oc create -f <vm_manifest_file>.yaml Optional: Start the virtual machine: USD virtctl start <vm_name> -n <namespace> steps Configuring SSH access to virtual machines 7.2.2. Creating VMs by using container disks You can create virtual machines (VMs) by using container disks built from operating system images. You can enable auto updates for your container disks. See Managing automatic boot source updates for details. Important If the container disks are large, the I/O traffic might increase and cause worker nodes to be unavailable. You can prune DeploymentConfig objects to resolve this issue: You create a VM from a container disk by performing the following steps: Build an operating system image into a container disk and upload it to your container registry . If your container registry does not have TLS, configure your environment to disable TLS for your registry . Create a VM with the container disk as the disk source by using the web console or the command line . Important You must install the QEMU guest agent on VMs created from operating system images that are not provided by Red Hat. 7.2.2.1. Building and uploading a container disk You can build a virtual machine (VM) image into a container disk and upload it to a registry. The size of a container disk is limited by the maximum layer size of the registry where the container disk is hosted. Note For Red Hat Quay , you can change the maximum layer size by editing the YAML configuration file that is created when Red Hat Quay is first deployed. Prerequisites You must have podman installed. You must have a QCOW2 or RAW image file. Procedure Create a Dockerfile to build the VM image into a container image. The VM image must be owned by QEMU, which has a UID of 107 , and placed in the /disk/ directory inside the container. Permissions for the /disk/ directory must then be set to 0440 . The following example uses the Red Hat Universal Base Image (UBI) to handle these configuration changes in the first stage, and uses the minimal scratch image in the second stage to store the result: USD cat > Dockerfile << EOF FROM registry.access.redhat.com/ubi8/ubi:latest AS builder ADD --chown=107:107 <vm_image>.qcow2 /disk/ 1 RUN chmod 0440 /disk/* FROM scratch COPY --from=builder /disk/* /disk/ EOF 1 Where <vm_image> is the image in either QCOW2 or RAW format. If you use a remote image, replace <vm_image>.qcow2 with the complete URL. Build and tag the container: USD podman build -t <registry>/<container_disk_name>:latest . Push the container image to the registry: USD podman push <registry>/<container_disk_name>:latest 7.2.2.2. Disabling TLS for a container registry You can disable TLS (transport layer security) for one or more container registries by editing the insecureRegistries field of the HyperConverged custom resource. Prerequisites Open the HyperConverged CR in your default editor by running the following command: USD oc edit hyperconverged kubevirt-hyperconverged -n openshift-cnv Add a list of insecure registries to the spec.storageImport.insecureRegistries field. Example HyperConverged custom resource apiVersion: hco.kubevirt.io/v1beta1 kind: HyperConverged metadata: name: kubevirt-hyperconverged namespace: openshift-cnv spec: storageImport: insecureRegistries: 1 - "private-registry-example-1:5000" - "private-registry-example-2:5000" 1 Replace the examples in this list with valid registry hostnames. 7.2.2.3. Creating a VM from a container disk by using the web console You can create a virtual machine (VM) by importing a container disk from a container registry by using the Red Hat OpenShift Service on AWS web console. Prerequisites You must have access to the container registry that contains the container disk. Procedure Navigate to Virtualization Catalog in the web console. Click a template tile without an available boot source. Click Customize VirtualMachine . On the Customize template parameters page, expand Storage and select Registry (creates PVC) from the Disk source list. Enter the image URL. Example: https://access.redhat.com/downloads/content/69/ver=/rhel---7/7.9/x86_64/product-software Enter the container image URL. Example: https://mirror.arizona.edu/fedora/linux/releases/38/Cloud/x86_64/images/Fedora-Cloud-Base-38-1.6.x86_64.qcow2 Set the disk size. Click . Click Create VirtualMachine . 7.2.2.4. Creating a VM from a container disk by using the command line You can create a virtual machine (VM) from a container disk by using the command line. When the virtual machine (VM) is created, the data volume with the container disk is imported into persistent storage. Prerequisites You must have access credentials for the container registry that contains the container disk. Procedure Edit the VirtualMachine manifest and save it as a vm-rhel-datavolume.yaml file: apiVersion: kubevirt.io/v1 kind: VirtualMachine metadata: creationTimestamp: null name: vm-rhel-datavolume 1 labels: kubevirt.io/vm: vm-rhel-datavolume spec: dataVolumeTemplates: - metadata: creationTimestamp: null name: rhel-dv 2 spec: sourceRef: kind: DataSource name: rhel9 namespace: openshift-virtualization-os-images storage: resources: requests: storage: 10Gi 3 instancetype: name: u1.small 4 preference: inferFromVolume: datavolumedisk1 runStrategy: Always template: metadata: creationTimestamp: null labels: kubevirt.io/vm: vm-rhel-datavolume spec: domain: devices: {} resources: {} terminationGracePeriodSeconds: 180 volumes: - dataVolume: name: rhel-dv name: datavolumedisk1 status: {} 1 Specify the name of the VM. 2 Specify the name of the data volume. 3 Specify the size of the storage requested for the data volume. 4 Optional: Specify the instance type to use to control resource sizing of the VM. Create the VM by running the following command: USD oc create -f vm-rhel-datavolume.yaml The oc create command creates the data volume and the VM. The CDI controller creates an underlying PVC with the correct annotation and the import process begins. When the import is complete, the data volume status changes to Succeeded . You can start the VM. Data volume provisioning happens in the background, so there is no need to monitor the process. Verification The importer pod downloads the container disk from the specified URL and stores it on the provisioned persistent volume. View the status of the importer pod by running the following command: USD oc get pods Monitor the data volume until its status is Succeeded by running the following command: USD oc describe dv rhel-dv 1 1 Specify the data volume name that you defined in the VirtualMachine manifest. Verify that provisioning is complete and that the VM has started by accessing its serial console: USD virtctl console vm-rhel-datavolume 7.2.3. Creating VMs by cloning PVCs You can create virtual machines (VMs) by cloning existing persistent volume claims (PVCs) with custom images. You must install the QEMU guest agent on VMs created from operating system images that are not provided by Red Hat. You clone a PVC by creating a data volume that references a source PVC. 7.2.3.1. About cloning When cloning a data volume, the Containerized Data Importer (CDI) chooses one of the following Container Storage Interface (CSI) clone methods: CSI volume cloning Smart cloning Both CSI volume cloning and smart cloning methods are efficient, but they have certain requirements for use. If the requirements are not met, the CDI uses host-assisted cloning. Host-assisted cloning is the slowest and least efficient method of cloning, but it has fewer requirements than either of the other two cloning methods. 7.2.3.1.1. CSI volume cloning Container Storage Interface (CSI) cloning uses CSI driver features to more efficiently clone a source data volume. CSI volume cloning has the following requirements: The CSI driver that backs the storage class of the persistent volume claim (PVC) must support volume cloning. For provisioners not recognized by the CDI, the corresponding storage profile must have the cloneStrategy set to CSI Volume Cloning. The source and target PVCs must have the same storage class and volume mode. If you create the data volume, you must have permission to create the datavolumes/source resource in the source namespace. The source volume must not be in use. 7.2.3.1.2. Smart cloning When a Container Storage Interface (CSI) plugin with snapshot capabilities is available, the Containerized Data Importer (CDI) creates a persistent volume claim (PVC) from a snapshot, which then allows efficient cloning of additional PVCs. Smart cloning has the following requirements: A snapshot class associated with the storage class must exist. The source and target PVCs must have the same storage class and volume mode. If you create the data volume, you must have permission to create the datavolumes/source resource in the source namespace. The source volume must not be in use. 7.2.3.1.3. Host-assisted cloning When the requirements for neither Container Storage Interface (CSI) volume cloning nor smart cloning have been met, host-assisted cloning is used as a fallback method. Host-assisted cloning is less efficient than either of the two other cloning methods. Host-assisted cloning uses a source pod and a target pod to copy data from the source volume to the target volume. The target persistent volume claim (PVC) is annotated with the fallback reason that explains why host-assisted cloning has been used, and an event is created. Example PVC target annotation apiVersion: v1 kind: PersistentVolumeClaim metadata: annotations: cdi.kubevirt.io/cloneFallbackReason: The volume modes of source and target are incompatible cdi.kubevirt.io/clonePhase: Succeeded cdi.kubevirt.io/cloneType: copy Example event NAMESPACE LAST SEEN TYPE REASON OBJECT MESSAGE test-ns 0s Warning IncompatibleVolumeModes persistentvolumeclaim/test-target The volume modes of source and target are incompatible 7.2.3.2. Creating a VM from a PVC by using the web console You can create a virtual machine (VM) by importing a container disk from a container registry by using the Red Hat OpenShift Service on AWS web console. You can create a virtual machine (VM) by cloning a persistent volume claim (PVC) by using the Red Hat OpenShift Service on AWS web console. Prerequisites You must have access to the container registry that contains the container disk. You must have access to the namespace that contains the source PVC. Procedure Navigate to Virtualization Catalog in the web console. Click a template tile without an available boot source. Click Customize VirtualMachine . On the Customize template parameters page, expand Storage and select PVC (clone PVC) from the Disk source list. Enter the image URL. Example: https://access.redhat.com/downloads/content/69/ver=/rhel---7/7.9/x86_64/product-software Enter the container image URL. Example: https://mirror.arizona.edu/fedora/linux/releases/38/Cloud/x86_64/images/Fedora-Cloud-Base-38-1.6.x86_64.qcow2 Select the PVC project and the PVC name. Set the disk size. Click . Click Create VirtualMachine . 7.2.3.3. Creating a VM from a PVC by using the command line You can create a virtual machine (VM) by cloning the persistent volume claim (PVC) of an existing VM by using the command line. You can clone a PVC by using one of the following options: Cloning a PVC to a new data volume. This method creates a data volume whose lifecycle is independent of the original VM. Deleting the original VM does not affect the new data volume or its associated PVC. Cloning a PVC by creating a VirtualMachine manifest with a dataVolumeTemplates stanza. This method creates a data volume whose lifecycle is dependent on the original VM. Deleting the original VM deletes the cloned data volume and its associated PVC. 7.2.3.3.1. Cloning a PVC to a data volume You can clone the persistent volume claim (PVC) of an existing virtual machine (VM) disk to a data volume by using the command line. You create a data volume that references the original source PVC. The lifecycle of the new data volume is independent of the original VM. Deleting the original VM does not affect the new data volume or its associated PVC. Cloning between different volume modes is supported for host-assisted cloning, such as cloning from a block persistent volume (PV) to a file system PV, as long as the source and target PVs belong to the kubevirt content type. Prerequisites The VM with the source PVC must be powered down. If you clone a PVC to a different namespace, you must have permissions to create resources in the target namespace. Additional prerequisites for smart-cloning: Your storage provider must support snapshots. The source and target PVCs must have the same storage provider and volume mode. The value of the driver key of the VolumeSnapshotClass object must match the value of the provisioner key of the StorageClass object as shown in the following example: Example VolumeSnapshotClass object kind: VolumeSnapshotClass apiVersion: snapshot.storage.k8s.io/v1 driver: openshift-storage.rbd.csi.ceph.com # ... Example StorageClass object kind: StorageClass apiVersion: storage.k8s.io/v1 # ... provisioner: openshift-storage.rbd.csi.ceph.com Procedure Create a DataVolume manifest as shown in the following example: apiVersion: cdi.kubevirt.io/v1beta1 kind: DataVolume metadata: name: <datavolume> 1 spec: source: pvc: namespace: "<source_namespace>" 2 name: "<my_vm_disk>" 3 storage: {} 1 Specify the name of the new data volume. 2 Specify the namespace of the source PVC. 3 Specify the name of the source PVC. Create the data volume by running the following command: USD oc create -f <datavolume>.yaml Note Data volumes prevent a VM from starting before the PVC is prepared. You can create a VM that references the new data volume while the PVC is being cloned. 7.2.3.3.2. Creating a VM from a cloned PVC by using a data volume template You can create a virtual machine (VM) that clones the persistent volume claim (PVC) of an existing VM by using a data volume template. This method creates a data volume whose lifecycle is dependent on the original VM. Deleting the original VM deletes the cloned data volume and its associated PVC. Prerequisites The VM with the source PVC must be powered down. Procedure Create a VirtualMachine manifest as shown in the following example: apiVersion: kubevirt.io/v1 kind: VirtualMachine metadata: labels: kubevirt.io/vm: vm-dv-clone name: vm-dv-clone 1 spec: runStrategy: Halted template: metadata: labels: kubevirt.io/vm: vm-dv-clone spec: domain: devices: disks: - disk: bus: virtio name: root-disk resources: requests: memory: 64M volumes: - dataVolume: name: favorite-clone name: root-disk dataVolumeTemplates: - metadata: name: favorite-clone spec: storage: accessModes: - ReadWriteOnce resources: requests: storage: 2Gi source: pvc: namespace: <source_namespace> 2 name: "<source_pvc>" 3 1 Specify the name of the VM. 2 Specify the namespace of the source PVC. 3 Specify the name of the source PVC. Create the virtual machine with the PVC-cloned data volume: USD oc create -f <vm-clone-datavolumetemplate>.yaml

Commands:
oc edit hyperconverged kubevirt-hyperconverged -n openshift-cnv
apiVersion: hco.kubevirt.io/v1 kind: HyperConverged metadata: name: kubevirt-hyperconverged namespace: openshift-cnv spec: commonBootImageNamespace: <custom_namespace> 1
apiVersion: kubevirt.io/v1 kind: VirtualMachine metadata: creationTimestamp: null name: vm-rhel-datavolume 1 labels: kubevirt.io/vm: vm-rhel-datavolume spec: dataVolumeTemplates: - metadata: creationTimestamp: null name: rhel-dv 2 spec: sourceRef: kind: DataSource name: rhel9 namespace: openshift-virtualization-os-images storage: resources: requests: storage: 10Gi 3 instancetype: name: u1.small 4 preference: inferFromVolume: datavolumedisk1 runStrategy: Always template: metadata: creationTimestamp: null labels: kubevirt.io/vm: vm-rhel-datavolume spec: domain: devices: {} resources: {} terminationGracePeriodSeconds: 180 volumes: - dataVolume: name: rhel-dv name: datavolumedisk1 status: {}
oc create -f vm-rhel-datavolume.yaml
oc get pods
oc describe dv rhel-dv 1
virtctl console vm-rhel-datavolume
virtctl stop <my_vm_name>
oc get vm <my_vm_name> -o jsonpath="{.spec.template.spec.volumes}{'\n'}"
[{"dataVolume":{"name":"<my_vm_volume>"},"name":"rootdisk"},{"cloudInitNoCloud":{...}]
oc get pvc
NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE <my_vm_volume> Bound ...
virtctl guestfs <my-vm-volume> --uid 107
virt-sysprep -a disk.img
%WINDIR%\System32\Sysprep\sysprep.exe /generalize /shutdown /oobe /mode:vm
virtctl image-upload dv <datavolume_name> \ 1 --size=<datavolume_size> \ 2 --image-path=</path/to/image> \ 3
oc get dvs
apiVersion: kubevirt.io/v1 kind: VirtualMachine metadata: name: rhel-9-minimal spec: dataVolumeTemplates: - metadata: name: rhel-9-minimal-volume spec: sourceRef: kind: DataSource name: rhel9 1 namespace: openshift-virtualization-os-images 2 storage: {} instancetype: name: u1.medium 3 preference: name: rhel.9 4 runStrategy: Always template: spec: domain: devices: {} volumes: - dataVolume: name: rhel-9-minimal-volume name: rootdisk
oc create -f <vm_manifest_file>.yaml
virtctl start <vm_name> -n <namespace>
cat > Dockerfile << EOF FROM registry.access.redhat.com/ubi8/ubi:latest AS builder ADD --chown=107:107 <vm_image>.qcow2 /disk/ 1 RUN chmod 0440 /disk/* FROM scratch COPY --from=builder /disk/* /disk/ EOF
podman build -t <registry>/<container_disk_name>:latest .
podman push <registry>/<container_disk_name>:latest
oc edit hyperconverged kubevirt-hyperconverged -n openshift-cnv
apiVersion: hco.kubevirt.io/v1beta1 kind: HyperConverged metadata: name: kubevirt-hyperconverged namespace: openshift-cnv spec: storageImport: insecureRegistries: 1 - "private-registry-example-1:5000" - "private-registry-example-2:5000"
apiVersion: kubevirt.io/v1 kind: VirtualMachine metadata: creationTimestamp: null name: vm-rhel-datavolume 1 labels: kubevirt.io/vm: vm-rhel-datavolume spec: dataVolumeTemplates: - metadata: creationTimestamp: null name: rhel-dv 2 spec: sourceRef: kind: DataSource name: rhel9 namespace: openshift-virtualization-os-images storage: resources: requests: storage: 10Gi 3 instancetype: name: u1.small 4 preference: inferFromVolume: datavolumedisk1 runStrategy: Always template: metadata: creationTimestamp: null labels: kubevirt.io/vm: vm-rhel-datavolume spec: domain: devices: {} resources: {} terminationGracePeriodSeconds: 180 volumes: - dataVolume: name: rhel-dv name: datavolumedisk1 status: {}
oc create -f vm-rhel-datavolume.yaml
oc get pods
oc describe dv rhel-dv 1
virtctl console vm-rhel-datavolume
apiVersion: v1 kind: PersistentVolumeClaim metadata: annotations: cdi.kubevirt.io/cloneFallbackReason: The volume modes of source and target are incompatible cdi.kubevirt.io/clonePhase: Succeeded cdi.kubevirt.io/cloneType: copy
NAMESPACE LAST SEEN TYPE REASON OBJECT MESSAGE test-ns 0s Warning IncompatibleVolumeModes persistentvolumeclaim/test-target The volume modes of source and target are incompatible
kind: VolumeSnapshotClass apiVersion: snapshot.storage.k8s.io/v1 driver: openshift-storage.rbd.csi.ceph.com
kind: StorageClass apiVersion: storage.k8s.io/v1 provisioner: openshift-storage.rbd.csi.ceph.com
apiVersion: cdi.kubevirt.io/v1beta1 kind: DataVolume metadata: name: <datavolume> 1 spec: source: pvc: namespace: "<source_namespace>" 2 name: "<my_vm_disk>" 3 storage: {}
oc create -f <datavolume>.yaml
apiVersion: kubevirt.io/v1 kind: VirtualMachine metadata: labels: kubevirt.io/vm: vm-dv-clone name: vm-dv-clone 1 spec: runStrategy: Halted template: metadata: labels: kubevirt.io/vm: vm-dv-clone spec: domain: devices: disks: - disk: bus: virtio name: root-disk resources: requests: memory: 64M volumes: - dataVolume: name: favorite-clone name: root-disk dataVolumeTemplates: - metadata: name: favorite-clone spec: storage: accessModes: - ReadWriteOnce resources: requests: storage: 2Gi source: pvc: namespace: <source_namespace> 2 name: "<source_pvc>" 3
oc create -f <vm-clone-datavolumetemplate>.yaml