Chapter 12. Configuring AMD SEV Compute nodes to provide memory encryption for instances As a cloud administrator, you can provide cloud users the ability to create instances that run on SEV-capable Compute nodes with memory encryption enabled. This feature is available to use from the 2nd Gen AMD EPYCTM 7002 Series ("Rome"). To enable your cloud users to create instances that use memory encryption, you must perform the following tasks: Designate the AMD SEV Compute nodes for memory encryption. Configure the Compute nodes for memory encryption. Deploy the overcloud. Create a flavor or image for launching instances with memory encryption. Tip If the AMD SEV hardware is limited, you can also configure a host aggregate to optimize scheduling on the AMD SEV Compute nodes. To schedule only instances that request memory encryption on the AMD SEV Compute nodes, create a host aggregate of the Compute nodes that have the AMD SEV hardware, and configure the Compute scheduler to place only instances that request memory encryption on the host aggregate. For more information, see Creating and managing host aggregates and Filtering by isolating host aggregates . 12.1. Secure Encrypted Virtualization (SEV) Secure Encrypted Virtualization (SEV), provided by AMD, protects the data in DRAM that a running virtual machine instance is using. SEV encrypts the memory of each instance with a unique key. SEV increases security when you use non-volatile memory technology (NVDIMM), because an NVDIMM chip can be physically removed from a system with the data intact, similar to a hard drive. Without encryption, any stored information such as sensitive data, passwords, or secret keys can be compromised. For more information, see the AMD Secure Encrypted Virtualization (SEV) documentation. Limitations of instances with memory encryption You cannot live migrate, or suspend and resume instances with memory encryption. You cannot use PCI passthrough to directly access devices on instances with memory encryption. You cannot use virtio-blk as the boot disk of instances with memory encryption with Red Hat Enterprise Linux (RHEL) kernels earlier than kernel-4.18.0-115.el8 (RHEL-8.1.0). Note You can use virtio-scsi or SATA as the boot disk, or virtio-blk for non-boot disks. The operating system that runs in an encrypted instance must provide SEV support. For more information, see the Red Hat Knowledgebase solution Enabling AMD Secure Encrypted Virtualization in RHEL 8 . Machines that support SEV have a limited number of slots in their memory controller for storing encryption keys. Each running instance with encrypted memory consumes one of these slots. Therefore, the number of instances with memory encryption that can run concurrently is limited to the number of slots in the memory controller. For example, on 1st Gen AMD EPYCTM 7001 Series ("Naples") the limit is 16, and on 2nd Gen AMD EPYCTM 7002 Series ("Rome") the limit is 255. Instances with memory encryption pin pages in RAM. The Compute service cannot swap these pages, therefore you cannot overcommit memory on a Compute node that hosts instances with memory encryption. You cannot use memory encryption with instances that have multiple NUMA nodes. 12.2. Designating AMD SEV Compute nodes for memory encryption To designate AMD SEV Compute nodes for instances that use memory encryption, you must create a new role file to configure the AMD SEV role, and configure a new overcloud flavor and AMD SEV resource class to use to tag the Compute nodes for memory encryption. Procedure Log in to the undercloud as the stack user. Source the stackrc file: Generate a new roles data file that includes the ComputeAMDSEV role, along with any other roles that you need for the overcloud. The following example generates the roles data file roles_data_amd_sev.yaml , which includes the roles Controller and ComputeAMDSEV : Open roles_data_amd_sev.yaml and edit or add the following parameters and sections: Section/Parameter Current value New value Role comment Role: Compute Role: ComputeAMDSEV Role name name: Compute name: ComputeAMDSEV description Basic Compute Node role AMD SEV Compute Node role HostnameFormatDefault %stackname%-novacompute-%index% %stackname%-novacomputeamdsev-%index% deprecated_nic_config_name compute.yaml compute-amd-sev.yaml Register the AMD SEV Compute nodes for the overcloud by adding them to your node definition template, node.json or node.yaml . For more information, see Registering nodes for the overcloud in the Director Installation and Usage guide. Inspect the node hardware: For more information, see Creating an inventory of the bare-metal node hardware in the Director Installation and Usage guide. Create the compute-amd-sev overcloud flavor for AMD SEV Compute nodes: Replace <ram_size_mb> with the RAM of the bare metal node, in MB. Replace <disk_size_gb> with the size of the disk on the bare metal node, in GB. Replace <no_vcpus> with the number of CPUs on the bare metal node. Note These properties are not used for scheduling instances. However, the Compute scheduler does use the disk size to determine the root partition size. Retrieve a list of your nodes to identify their UUIDs: Tag each bare metal node that you want to designate for memory encryption with a custom AMD SEV resource class: Replace <node> with the ID of the bare metal node. Associate the compute-amd-sev flavor with the custom AMD SEV resource class: To determine the name of a custom resource class that corresponds to a resource class of a Bare Metal service node, convert the resource class to uppercase, replace each punctuation mark with an underscore, and prefix with CUSTOM_ . Note A flavor can request only one instance of a bare metal resource class. Set the following flavor properties to prevent the Compute scheduler from using the bare metal flavor properties to schedule instances: Optional: If the network topology of the ComputeAMDSEV role is different from the network topology of your Compute role, then create a custom network interface template. For more information, see Custom network interface templates in the Advanced Overcloud Customization guide. If the network topology of the ComputeAMDSEV role is the same as the Compute role, then you can use the default network topology defined in compute.yaml . Register the Net::SoftwareConfig of the ComputeAMDSEV role in your network-environment.yaml file: Replace <amd_sev_net_top> with the name of the file that contains the network topology of the ComputeAMDSEV role, for example, compute.yaml to use the default network topology. Add the following parameters to the node-info.yaml file to specify the number of AMD SEV Compute nodes, and the flavor that you want to use for the AMD SEV designated Compute nodes: To verify that the role was created, enter the following command: Example output: 12.3. Configuring AMD SEV Compute nodes for memory encryption To enable your cloud users to create instances that use memory encryption, you must configure the Compute nodes that have the AMD SEV hardware. Prerequisites Your deployment must include a Compute node that runs on AMD hardware capable of supporting SEV, such as an AMD EPYC CPU. You can use the following command to determine if your deployment is SEV-capable: Procedure Open your Compute environment file. Optional: Add the following configuration to your Compute environment file to specify the maximum number of memory-encrypted instances that the AMD SEV Compute nodes can host concurrently: Note The default value of the libvirt/num_memory_encrypted_guests parameter is none . If you do not set a custom value, the AMD SEV Compute nodes do not impose a limit on the number of memory-encrypted instances that the nodes can host concurrently. Instead, the hardware determines the maximum number of memory-encrypted instances that the AMD SEV Compute nodes can host concurrently, which might cause some memory-encrypted instances to fail to launch. Optional: To specify that all x86_64 images use the q35 machine type by default, add the following configuration to your Compute environment file: If you specify this parameter value, you do not need to set the hw_machine_type property to q35 on every AMD SEV instance image. To ensure that the AMD SEV Compute nodes reserve enough memory for host-level services to function, add 16MB for each potential AMD SEV instance: Configure the kernel parameters for the AMD SEV Compute nodes: Note When you first add the KernelArgs parameter to the configuration of a role, the overcloud nodes are automatically rebooted. If required, you can disable the automatic rebooting of nodes and instead perform node reboots manually after each overcloud deployment. For more information, see Configuring manual node reboot to define KernelArgs . Save the updates to your Compute environment file. Add your Compute environment file to the stack with your other environment files and deploy the overcloud: 12.4. Creating an image for memory encryption When the overcloud contains AMD SEV Compute nodes, you can create an AMD SEV instance image that your cloud users can use to launch instances that have memory encryption. Procedure Create a new image for memory encryption: Note If you use an existing image, the image must have the hw_firmware_type property set to uefi . Optional: Add the property hw_mem_encryption=True to the image to enable AMD SEV memory encryption on the image: Tip You can enable memory encryption on the flavor. For more information, see Creating a flavor for memory encryption . Optional: Set the machine type to q35 , if not already set in the Compute node configuration: Optional: To schedule memory-encrypted instances on a SEV-capable host aggregate, add the following trait to the image extra specs: Tip You can also specify this trait on the flavor. For more information, see Creating a flavor for memory encryption . 12.5. Creating a flavor for memory encryption When the overcloud contains AMD SEV Compute nodes, you can create one or more AMD SEV flavors that your cloud users can use to launch instances that have memory encryption. Note An AMD SEV flavor is necessary only when the hw_mem_encryption property is not set on an image. Procedure Create a flavor for memory encryption: To schedule memory-encrypted instances on a SEV-capable host aggregate, add the following trait to the flavor extra specs: 12.6. Launching an instance with memory encryption To verify that you can launch instances on an AMD SEV Compute node with memory encryption enabled, use a memory encryption flavor or image to create an instance. Procedure Create an instance by using an AMD SEV flavor or image. The following example creates an instance by using the flavor created in Creating a flavor for memory encryption and the image created in Creating an image for memory encryption : Log in to the instance as a cloud user. To verify that the instance uses memory encryption, enter the following command from the instance:

Commands:
[stack@director ~]USD source ~/stackrc
(undercloud)USD openstack overcloud roles generate -o /home/stack/templates/roles_data_amd_sev.yaml Compute:ComputeAMDSEV Controller
(undercloud)USD openstack overcloud node introspect --all-manageable --provide
(undercloud)USD openstack flavor create --id auto --ram <ram_size_mb> --disk <disk_size_gb> --vcpus <no_vcpus> compute-amd-sev
(undercloud)USD openstack baremetal node list
(undercloud)USD openstack baremetal node set --resource-class baremetal.AMD-SEV <node>
(undercloud)USD openstack flavor set --property resources:CUSTOM_BAREMETAL_AMD_SEV=1 compute-amd-sev
(undercloud)USD openstack flavor set --property resources:VCPU=0 --property resources:MEMORY_MB=0 --property resources:DISK_GB=0 compute-amd-sev
resource_registry: OS::TripleO::Compute::Net::SoftwareConfig: /home/stack/templates/nic-configs/compute.yaml OS::TripleO::ComputeCPUPinning::Net::SoftwareConfig: /home/stack/templates/nic-configs/<amd_sev_net_top>.yaml OS::TripleO::Controller::Net::SoftwareConfig: /home/stack/templates/nic-configs/controller.yaml
parameter_defaults: OvercloudComputeAMDSEVFlavor: compute-amd-sev ComputeAMDSEVCount: 3
(undercloud)USD openstack baremetal node list --long -c "UUID" -c "Instance UUID" -c "Resource Class" -c "Provisioning State" -c "Power State" -c "Last Error" -c "Fault" -c "Name" -f json
Instance UUID: e8e60d37-d7c7-4210-acf7-f04b245582ea
Name: compute-0
Power State: power on
Provisioning State: active
Resource Class: baremetal.AMD-SEV
UUID: b5a9ac58-63a7-49ba-b4ad-33d84000ccb4
Instance UUID: 3ec34c0b-c4f5-4535-9bd3-8a1649d2e1bd
Name: compute-1
Power State: power on
Provisioning State: active
Resource Class: compute
UUID: 432e7f86-8da2-44a6-9b14-dfacdf611366
Instance UUID: 4992c2da-adde-41b3-bef1-3a5b8e356fc0
Name: controller-0
Power State: power on
Provisioning State: active
Resource Class: controller
UUID: 474c2fc8-b884-4377-b6d7-781082a3a9c0
lscpu | grep sev
parameter_defaults: ComputeAMDSEVExtraConfig: nova::config::nova_config: libvirt/num_memory_encrypted_guests: value: 15
parameter_defaults: ComputeAMDSEVParameters: NovaHWMachineType: x86_64=q35
parameter_defaults: ComputeAMDSEVParameters: NovaReservedHostMemory: <libvirt/num_memory_encrypted_guests * 16>
parameter_defaults: ComputeAMDSEVParameters: KernelArgs: "hugepagesz=1GB hugepages=32 default_hugepagesz=1GB mem_encrypt=on kvm_amd.sev=1"
(undercloud)USD openstack overcloud deploy --templates -e [your environment files] -e /home/stack/templates/<compute_environment_file>.yaml
(overcloud)USD openstack image create ... --property hw_firmware_type=uefi amd-sev-image
(overcloud)USD openstack image set --property hw_mem_encryption=True amd-sev-image
(overcloud)USD openstack image set --property hw_machine_type=q35 amd-sev-image
(overcloud)USD openstack image set --property trait:HW_CPU_X86_AMD_SEV=required amd-sev-image
(overcloud)USD openstack flavor create --vcpus 1 --ram 512 --disk 2 --property hw:mem_encryption=True m1.small-amd-sev
(overcloud)USD openstack flavor set --property trait:HW_CPU_X86_AMD_SEV=required m1.small-amd-sev
(overcloud)USD openstack server create --flavor m1.small-amd-sev --image amd-sev-image amd-sev-instance
dmesg | grep -i sev AMD Secure Encrypted Virtualization (SEV) active