Chapter 1. About 1.1. About OpenShift Virtualization Learn about OpenShift Virtualization's capabilities and support scope. 1.1.1. What you can do with OpenShift Virtualization OpenShift Virtualization provides the scalable, enterprise-grade virtualization functionality in Red Hat OpenShift. You can use it to manage virtual machines (VMs) exclusively or alongside container workloads. Note If you have a Red Hat OpenShift Virtualization Engine subscription, you can run unlimited VMs on subscribed hosts, but you cannot run application instances in containers. For more information, see the subscription guide section about Red Hat OpenShift Virtualization Engine and related products . OpenShift Virtualization adds new objects into your OpenShift Container Platform cluster by using Kubernetes custom resources to enable virtualization tasks. These tasks include: Creating and managing Linux and Windows VMs Running pod and VM workloads alongside each other in a cluster Connecting to VMs through a variety of consoles and CLI tools Importing and cloning existing VMs Managing network interface controllers and storage disks attached to VMs Live migrating VMs between nodes You can manage your cluster and virtualization resources by using the Virtualization perspective of the OpenShift Container Platform web console, and by using the OpenShift CLI ( oc ). OpenShift Virtualization is designed and tested to work well with Red Hat OpenShift Data Foundation features. Important When you deploy OpenShift Virtualization with OpenShift Data Foundation, you must create a dedicated storage class for Windows virtual machine disks. See Optimizing ODF PersistentVolumes for Windows VMs for details. You can use OpenShift Virtualization with OVN-Kubernetes or one of the other certified network plugins listed in Certified OpenShift CNI Plug-ins . You can check your OpenShift Virtualization cluster for compliance issues by installing the Compliance Operator and running a scan with the ocp4-moderate and ocp4-moderate-node profiles . The Compliance Operator uses OpenSCAP, a NIST-certified tool , to scan and enforce security policies. 1.1.2. Comparing OpenShift Virtualization to VMware vSphere If you are familiar with VMware vSphere, the following table lists OpenShift Virtualization components that you can use to accomplish similar tasks. However, because OpenShift Virtualization is conceptually different from vSphere, and much of its functionality comes from the underlying OpenShift Container Platform, OpenShift Virtualization does not have direct alternatives for all vSphere concepts or components. Table 1.1. Mapping of vSphere concepts to their closest OpenShift Virtualization counterparts vSphere concept OpenShift Virtualization Explanation Datastore Persistent volume (PV) + Persistent volume claim (PVC) Stores VM disks. A PV represents existing storage and is attached to a VM through a PVC. When created with the ReadWriteMany (RWX) access mode, PVCs can be mounted by multiple VMs simultaneously. Dynamic Resource Scheduling (DRS) Pod eviction policy + Descheduler Provides active resource balancing. A combination of pod eviction policies and a descheduler allows VMs to be live migrated to more appropriate nodes to keep node resource utilization manageable. NSX Multus + OVN-Kubernetes + Third-party container network interface (CNI) plug-ins Provides an overlay network configuration. There is no direct equivalent for NSX in OpenShift Virtualization, but you can use the OVN-Kubernetes network provider or install certified third-party CNI plug-ins. Storage Policy Based Management (SPBM) Storage class Provides policy-based storage selection. Storage classes represent various storage types and describe storage capabilities, such as quality of service, backup policy, reclaim policy, and whether volume expansion is allowed. A PVC can request a specific storage class to satisfy application requirements. vCenter vRealize Operations OpenShift Metrics and Monitoring Provides host and VM metrics. You can view metrics and monitor the overall health of the cluster and VMs by using the OpenShift Container Platform web console. vMotion Live migration Moves a running VM to another node without interruption. For live migration to be available, the PVC attached to the VM must have the ReadWriteMany (RWX) access mode. vSwitch DvSwitch NMState Operator + Multus Provides a physical network configuration. You can use the NMState Operator to apply state-driven network configuration and manage various network interface types, including Linux bridges and network bonds. With Multus, you can attach multiple network interfaces and connect VMs to external networks. 1.1.3. Supported cluster versions for OpenShift Virtualization OpenShift Virtualization 4.17 is supported for use on OpenShift Container Platform 4.17 clusters. To use the latest z-stream release of OpenShift Virtualization, you must first upgrade to the latest version of OpenShift Container Platform. 1.1.4. About volume and access modes for virtual machine disks If you use the storage API with known storage providers, the volume and access modes are selected automatically. However, if you use a storage class that does not have a storage profile, you must configure the volume and access mode. For best results, use the ReadWriteMany (RWX) access mode and the Block volume mode. This is important for the following reasons: ReadWriteMany (RWX) access mode is required for live migration. The Block volume mode performs significantly better than the Filesystem volume mode. This is because the Filesystem volume mode uses more storage layers, including a file system layer and a disk image file. These layers are not necessary for VM disk storage. For example, if you use Red Hat OpenShift Data Foundation, Ceph RBD volumes are preferable to CephFS volumes. Important You cannot live migrate virtual machines with the following configurations: Storage volume with ReadWriteOnce (RWO) access mode Passthrough features such as GPUs Set the evictionStrategy field to None for these virtual machines. The None strategy powers down VMs during node reboots. 1.1.5. Single-node OpenShift differences You can install OpenShift Virtualization on single-node OpenShift. However, you should be aware that Single-node OpenShift does not support the following features: High availability Pod disruption Live migration Virtual machines or templates that have an eviction strategy configured 1.1.6. Additional resources Glossary of common terms for OpenShift Container Platform storage About single-node OpenShift Assisted installer Pod disruption budgets About live migration Eviction strategies Tuning & Scaling Guide Supported limits for OpenShift Virtualization 4.x 1.2. Security policies Learn about OpenShift Virtualization security and authorization. Key points OpenShift Virtualization adheres to the restricted Kubernetes pod security standards profile, which aims to enforce the current best practices for pod security. Virtual machine (VM) workloads run as unprivileged pods. Security context constraints (SCCs) are defined for the kubevirt-controller service account. TLS certificates for OpenShift Virtualization components are renewed and rotated automatically. 1.2.1. About workload security By default, virtual machine (VM) workloads do not run with root privileges in OpenShift Virtualization, and there are no supported OpenShift Virtualization features that require root privileges. For each VM, a virt-launcher pod runs an instance of libvirt in session mode to manage the VM process. In session mode, the libvirt daemon runs as a non-root user account and only permits connections from clients that are running under the same user identifier (UID). Therefore, VMs run as unprivileged pods, adhering to the security principle of least privilege. 1.2.2. TLS certificates TLS certificates for OpenShift Virtualization components are renewed and rotated automatically. You are not required to refresh them manually. Automatic renewal schedules TLS certificates are automatically deleted and replaced according to the following schedule: KubeVirt certificates are renewed daily. Containerized Data Importer controller (CDI) certificates are renewed every 15 days. MAC pool certificates are renewed every year. Automatic TLS certificate rotation does not disrupt any operations. For example, the following operations continue to function without any disruption: Migrations Image uploads VNC and console connections 1.2.3. Authorization OpenShift Virtualization uses role-based access control (RBAC) to define permissions for human users and service accounts. The permissions defined for service accounts control the actions that OpenShift Virtualization components can perform. You can also use RBAC roles to manage user access to virtualization features. For example, an administrator can create an RBAC role that provides the permissions required to launch a virtual machine. The administrator can then restrict access by binding the role to specific users. 1.2.3.1. Default cluster roles for OpenShift Virtualization By using cluster role aggregation, OpenShift Virtualization extends the default OpenShift Container Platform cluster roles to include permissions for accessing virtualization objects. Table 1.2. OpenShift Virtualization cluster roles Default cluster role OpenShift Virtualization cluster role OpenShift Virtualization cluster role description view kubevirt.io:view A user that can view all OpenShift Virtualization resources in the cluster but cannot create, delete, modify, or access them. For example, the user can see that a virtual machine (VM) is running but cannot shut it down or gain access to its console. edit kubevirt.io:edit A user that can modify all OpenShift Virtualization resources in the cluster. For example, the user can create VMs, access VM consoles, and delete VMs. admin kubevirt.io:admin A user that has full permissions to all OpenShift Virtualization resources, including the ability to delete collections of resources. The user can also view and modify the OpenShift Virtualization runtime configuration, which is located in the HyperConverged custom resource in the openshift-cnv namespace. 1.2.3.2. RBAC roles for storage features in OpenShift Virtualization The following permissions are granted to the Containerized Data Importer (CDI), including the cdi-operator and cdi-controller service accounts. 1.2.3.2.1. Cluster-wide RBAC roles Table 1.3. Aggregated cluster roles for the cdi.kubevirt.io API group CDI cluster role Resources Verbs cdi.kubevirt.io:admin datavolumes , uploadtokenrequests * (all) datavolumes/source create cdi.kubevirt.io:edit datavolumes , uploadtokenrequests * datavolumes/source create cdi.kubevirt.io:view cdiconfigs , dataimportcrons , datasources , datavolumes , objecttransfers , storageprofiles , volumeimportsources , volumeuploadsources , volumeclonesources get , list , watch datavolumes/source create cdi.kubevirt.io:config-reader cdiconfigs , storageprofiles get , list , watch Table 1.4. Cluster-wide roles for the cdi-operator service account API group Resources Verbs rbac.authorization.k8s.io clusterrolebindings , clusterroles get , list , watch , create , update , delete security.openshift.io securitycontextconstraints get , list , watch , update , create apiextensions.k8s.io customresourcedefinitions , customresourcedefinitions/status get , list , watch , create , update , delete cdi.kubevirt.io * * upload.cdi.kubevirt.io * * admissionregistration.k8s.io validatingwebhookconfigurations , mutatingwebhookconfigurations create , list , watch admissionregistration.k8s.io validatingwebhookconfigurations Allow list: cdi-api-dataimportcron-validate, cdi-api-populator-validate, cdi-api-datavolume-validate, cdi-api-validate, objecttransfer-api-validate get , update , delete admissionregistration.k8s.io mutatingwebhookconfigurations Allow list: cdi-api-datavolume-mutate get , update , delete apiregistration.k8s.io apiservices get , list , watch , create , update , delete Table 1.5. Cluster-wide roles for the cdi-controller service account API group Resources Verbs "" (core) events create , patch "" (core) persistentvolumeclaims get , list , watch , create , update , delete , deletecollection , patch "" (core) persistentvolumes get , list , watch , update "" (core) persistentvolumeclaims/finalizers , pods/finalizers update "" (core) pods , services get , list , watch , create , delete "" (core) configmaps get , create storage.k8s.io storageclasses , csidrivers get , list , watch config.openshift.io proxies get , list , watch cdi.kubevirt.io * * snapshot.storage.k8s.io volumesnapshots , volumesnapshotclasses , volumesnapshotcontents get , list , watch , create , delete snapshot.storage.k8s.io volumesnapshots update , deletecollection apiextensions.k8s.io customresourcedefinitions get , list , watch scheduling.k8s.io priorityclasses get , list , watch image.openshift.io imagestreams get , list , watch "" (core) secrets create kubevirt.io virtualmachines/finalizers update 1.2.3.2.2. Namespaced RBAC roles Table 1.6. Namespaced roles for the cdi-operator service account API group Resources Verbs rbac.authorization.k8s.io rolebindings , roles get , list , watch , create , update , delete "" (core) serviceaccounts , configmaps , events , secrets , services get , list , watch , create , update , patch , delete apps deployments , deployments/finalizers get , list , watch , create , update , delete route.openshift.io routes , routes/custom-host get , list , watch , create , update config.openshift.io proxies get , list , watch monitoring.coreos.com servicemonitors , prometheusrules get , list , watch , create , delete , update , patch coordination.k8s.io leases get , create , update Table 1.7. Namespaced roles for the cdi-controller service account API group Resources Verbs "" (core) configmaps get , list , watch , create , update , delete "" (core) secrets get , list , watch batch cronjobs get , list , watch , create , update , delete batch jobs create , delete , list , watch coordination.k8s.io leases get , create , update networking.k8s.io ingresses get , list , watch route.openshift.io routes get , list , watch 1.2.3.3. Additional SCCs and permissions for the kubevirt-controller service account Security context constraints (SCCs) control permissions for pods. These permissions include actions that a pod, a collection of containers, can perform and what resources it can access. You can use SCCs to define a set of conditions that a pod must run with to be accepted into the system. The virt-controller is a cluster controller that creates the virt-launcher pods for virtual machines in the cluster. These pods are granted permissions by the kubevirt-controller service account. The kubevirt-controller service account is granted additional SCCs and Linux capabilities so that it can create virt-launcher pods with the appropriate permissions. These extended permissions allow virtual machines to use OpenShift Virtualization features that are beyond the scope of typical pods. The kubevirt-controller service account is granted the following SCCs: scc.AllowHostDirVolumePlugin = true This allows virtual machines to use the hostpath volume plugin. scc.AllowPrivilegedContainer = false This ensures the virt-launcher pod is not run as a privileged container. scc.AllowedCapabilities = []corev1.Capability{"SYS_NICE", "NET_BIND_SERVICE"} SYS_NICE allows setting the CPU affinity. NET_BIND_SERVICE allows DHCP and Slirp operations. Viewing the SCC and RBAC definitions for the kubevirt-controller You can view the SecurityContextConstraints definition for the kubevirt-controller by using the oc tool: USD oc get scc kubevirt-controller -o yaml You can view the RBAC definition for the kubevirt-controller clusterrole by using the oc tool: USD oc get clusterrole kubevirt-controller -o yaml 1.2.4. Additional resources Managing security context constraints Using RBAC to define and apply permissions Creating a cluster role Cluster role binding commands Enabling user permissions to clone data volumes across namespaces 1.3. OpenShift Virtualization Architecture The Operator Lifecycle Manager (OLM) deploys operator pods for each component of OpenShift Virtualization: Compute: virt-operator Storage: cdi-operator Network: cluster-network-addons-operator Scaling: ssp-operator OLM also deploys the hyperconverged-cluster-operator pod, which is responsible for the deployment, configuration, and life cycle of other components, and several helper pods: hco-webhook , and hyperconverged-cluster-cli-download . After all operator pods are successfully deployed, you should create the HyperConverged custom resource (CR). The configurations set in the HyperConverged CR serve as the single source of truth and the entrypoint for OpenShift Virtualization, and guide the behavior of the CRs. The HyperConverged CR creates corresponding CRs for the operators of all other components within its reconciliation loop. Each operator then creates resources such as daemon sets, config maps, and additional components for the OpenShift Virtualization control plane. For example, when the HyperConverged Operator (HCO) creates the KubeVirt CR, the OpenShift Virtualization Operator reconciles it and creates additional resources such as virt-controller , virt-handler , and virt-api . The OLM deploys the Hostpath Provisioner (HPP) Operator, but it is not functional until you create a hostpath-provisioner CR. Virtctl client commands 1.3.1. About the HyperConverged Operator (HCO) The HCO, hco-operator , provides a single entry point for deploying and managing OpenShift Virtualization and several helper operators with opinionated defaults. It also creates custom resources (CRs) for those operators. Table 1.8. HyperConverged Operator components Component Description deployment/hco-webhook Validates the HyperConverged custom resource contents. deployment/hyperconverged-cluster-cli-download Provides the virtctl tool binaries to the cluster so that you can download them directly from the cluster. KubeVirt/kubevirt-kubevirt-hyperconverged Contains all operators, CRs, and objects needed by OpenShift Virtualization. SSP/ssp-kubevirt-hyperconverged A Scheduling, Scale, and Performance (SSP) CR. This is automatically created by the HCO. CDI/cdi-kubevirt-hyperconverged A Containerized Data Importer (CDI) CR. This is automatically created by the HCO. NetworkAddonsConfig/cluster A CR that instructs and is managed by the cluster-network-addons-operator . 1.3.2. About the Containerized Data Importer (CDI) Operator The CDI Operator, cdi-operator , manages CDI and its related resources, which imports a virtual machine (VM) image into a persistent volume claim (PVC) by using a data volume. Table 1.9. CDI Operator components Component Description deployment/cdi-apiserver Manages the authorization to upload VM disks into PVCs by issuing secure upload tokens. deployment/cdi-uploadproxy Directs external disk upload traffic to the appropriate upload server pod so that it can be written to the correct PVC. Requires a valid upload token. pod/cdi-importer Helper pod that imports a virtual machine image into a PVC when creating a data volume. 1.3.3. About the Cluster Network Addons Operator The Cluster Network Addons Operator, cluster-network-addons-operator , deploys networking components on a cluster and manages the related resources for extended network functionality. Table 1.10. Cluster Network Addons Operator components Component Description deployment/kubemacpool-cert-manager Manages TLS certificates of Kubemacpool's webhooks. deployment/kubemacpool-mac-controller-manager Provides a MAC address pooling service for virtual machine (VM) network interface cards (NICs). daemonset/bridge-marker Marks network bridges available on nodes as node resources. daemonset/kube-cni-linux-bridge-plugin Installs Container Network Interface (CNI) plugins on cluster nodes, enabling the attachment of VMs to Linux bridges through network attachment definitions. 1.3.4. About the Hostpath Provisioner (HPP) Operator The HPP Operator, hostpath-provisioner-operator , deploys and manages the multi-node HPP and related resources. Table 1.11. HPP Operator components Component Description deployment/hpp-pool-hpp-csi-pvc-block-<worker_node_name> Provides a worker for each node where the HPP is designated to run. The pods mount the specified backing storage on the node. daemonset/hostpath-provisioner-csi Implements the Container Storage Interface (CSI) driver interface of the HPP. daemonset/hostpath-provisioner Implements the legacy driver interface of the HPP. 1.3.5. About the Scheduling, Scale, and Performance (SSP) Operator The SSP Operator, ssp-operator , deploys the common templates, the related default boot sources, the pipeline tasks, and the template validator. 1.3.6. About the OpenShift Virtualization Operator The OpenShift Virtualization Operator, virt-operator , deploys, upgrades, and manages OpenShift Virtualization without disrupting current virtual machine (VM) workloads. In addition, the OpenShift Virtualization Operator deploys the common instance types and common preferences. Table 1.12. virt-operator components Component Description deployment/virt-api HTTP API server that serves as the entry point for all virtualization-related flows. deployment/virt-controller Observes the creation of a new VM instance object and creates a corresponding pod. When the pod is scheduled on a node, virt-controller updates the VM with the node name. daemonset/virt-handler Monitors any changes to a VM and instructs virt-launcher to perform the required operations. This component is node-specific. pod/virt-launcher Contains the VM that was created by the user as implemented by libvirt and qemu .

Commands:
oc get scc kubevirt-controller -o yaml
oc get clusterrole kubevirt-controller -o yaml