Chapter 5. Creating and managing resources for bare-metal instances As a cloud operator you can create and manage resources for bare-metal workloads and enable your cloud users to create bare-metal instances. You can create the following resources for bare-metal workloads: Bare-metal instances Images for bare-metal instances Virtual network interfaces (VIFs) for bare-metal nodes Port groups You can perform the following resource management tasks: Manual node cleaning Attach a virtual network interface (VIF) to a bare-metal instance 5.1. Prerequisites The RHOSO environment includes the Bare Metal Provisioning service. For more information, see Enabling the Bare Metal Provisioning service (ironic) . You are logged on to a workstation that has access to the RHOCP cluster, as a user with cluster-admin privileges. The oc command line tool is installed on the workstation. 5.2. Launching bare-metal instances You can launch a bare-metal instance by using the OpenStack Client CLI. Procedure Access the remote shell for the OpenStackClient pod from your workstation: Create the bare-metal instance: Replace <network_uuid> with the unique identifier for the network that you created to use with the Bare Metal Provisioning service. Replace <image_uuid> with the unique identifier for the image that has the software profile that your instance requires. Check the status of the instance: Exit the openstackclient pod: 5.3. Images for launching bare-metal instances A Red Hat OpenStack Services on OpenShift (RHOSO) environment that includes the Bare Metal Provisioning service (ironic) requires two sets of images: Deploy images: The deploy images are the agent.ramdisk and agent.kernel images that the Bare Metal Provisioning agent ( ironic-python-agent ) requires to boot the RAM disk over the network and copy the user image to the disk. User images: The images the cloud user uses to provision their bare-metal instances. The user image consists of a kernel image, a ramdisk image, and a main image. The main image is either a root partition, or a whole-disk image: Whole-disk image: An image that contains the partition table and boot loader. Root partition image: Contains only the root partition of the operating system. Compatible whole-disk RHEL guest images should work without modification. To create your own custom disk image, see Creating RHEL KVM or RHOSP-compatible images in Creating and managing images . 5.4. Booting an ISO image directly for use as a RAM disk You can boot a bare-metal instance from a RAM disk or an ISO image if you want to boot an instance with PXE, iPXE, or Virtual Media, and use the instance memory for local storage. This is useful for advanced scientific and ephemeral workloads where writing an image to the local storage is not required or desired. Procedure Access the remote shell for the OpenStackClient pod from your workstation: Specify ramdisk as the deploy interface for the bare-metal node that boots from an ISO image: Tip You can configure the deploy interface when you create the bare-metal node by adding --deploy-interface ramdisk to the openstack baremetal node create command. For information on how to create a bare-metal node, see Enrolling a bare-metal node manually . Update the bare-metal node to boot an ISO image: Replace <node_UUID> with the UUID of the bare-metal node that you want to boot from an ISO image. Replace <boot_iso_url> with the URL of the boot ISO file. You can specify the boot ISO file URL by using one of the following methods: HTTP or HTTPS URL File path URL Image service (glance) object UUID Deploy the bare-metal node as an ISO image: Exit the openstackclient pod: 5.5. Creating the virtual network interfaces (VIFs) for bare-metal instances Cloud users can attach their bare-metal instances to the network interfaces you create for the bare-metal workloads. You must create the virtual network interfaces (VIFs) for the cloud user to select for attachment. 5.5.1. Bare Metal Provisioning service virtual network interfaces (VIFs) The Bare Metal Provisioning service (ironic) uses the Networking service (neutron) to manage the attachment state of the virtual network interfaces (VIFs). A VIF is a Networking service port, referred to by the port ID, which is a UUID value. A VIF can be available across a limited number of physical networks, dependent upon the cloud's operating configuration and operating constraints. The Bare Metal Provisioning service can also attach the bare-metal instance to a separate provider network to improve the overall operational security. Each VIF must be attached to a port or port group, therefore the maximum number of VIFs is determined by the number of configured and available ports represented in the Bare Metal Provisioning service. The network interface is one of the driver interfaces that manages the network switching for bare-metal instances. The type of network interface you create influences the operation of your bare-metal workloads. The following network interfaces are available to use with the Bare Metal Provisioning service: noop : Used for standalone deployments, and does not perform any network switching. flat : Places all nodes into a single provider network that is pre-configured on the Networking service and physical equipment. Nodes remain physically connected to this network during their entire life cycle. The supplied VIF attachment record is updated with new DHCP records as needed. When using this network interface, the VIF needs to be created on the same network that the bare-metal node is physically attached to. neutron : Provides tenant-defined networking through the Networking service, separating tenant networks from each other and from the provisioning and cleaning provider networks. Nodes move between these networks during their life cycle. This interface requires Networking service support for the switches attached to the bare-metal instances so they can be programmed. This interface requires the ML2 plugin OVN mechanism driver or other SDN integrations to facilitate port configuration on the network. Use the neutron interface when your environment uses IPv6. 5.5.2. How the Bare Metal Provisioning service manages VIFs when provisioning a bare-metal node When provisioning, by default the Bare Metal Provisioning service (ironic) attempts to attach all PXE-enabled ports to the provisioning network. If you have neutron.add_all_ports enabled, then the Bare Metal Provisioning service attempts to bind all ports to the required service network beyond the Bare Metal Provisioning service ports with pxe_enabled set to True . After the bare-metal nodes are provisioned, and before the bare-metal nodes are moved to the ACTIVE provisioning state, the previously attached ports are unbound. The process for unbinding is dependent on the network interface: flat : All the requested VIFs with all binding configurations in all states are unbound. neutron : The VIFs requested by the cloud user are attached to the bare-metal node for the first time, because the VIFs that the Bare Metal Provisioning service created were being deleted during the provisioning process. The same flow and logic applies to the cleaning, service, and rescue processes. 5.5.3. Creating a virtual network interface (VIF) for bare-metal nodes Use the Networking service (neutron) to create the port that serves as the virtual network interface (VIF). If you are using the neutron network interface, then you must also create a physical connection to the underlying physical network by creating a Bare Metal Provisioning service (ironic) port with a binding profile. The binding profile is required by the Networking service's ML2 mechanism driver when a VIF is attached to a bare-metal instance. The binding profile includes the VNIC_BAREMETAL port type, the bare-metal node UUID, and local link connection information that identifies the tenant network that the ML2 mechanism driver must attach to the physical bare-metal port. The binding profile information is populated through the introspection process by using LLDP data that is broadcast from the switches, therefore the switches must have LLDP enabled. You need to manually set or update the binding profile when there is a physical networking change, for example, when a bare-metal port's cable has been moved to a different port on a switch, or the switch has been replaced. Note Decoding LLDP data is performed as a best effort action. Some switch vendors, or changes in switch vendor firmware might impact field decoding. Procedure Access the remote shell for the OpenStackClient pod from your workstation: Create the virtual network interface (VIF): If you are using the neutron network interface, then create a Bare Metal Provisioning service port with the binding profile information: Replace <switch_mac_address> with the MAC address or OpenFlow-based datapath_id of the switch. Replace <switch_hostname> with the name of the bare-metal node that hosts the switch. Replace <switch_port_for_connection> with the port ID on the switch, for example, Gig0/1 , or rep0-0 . Replace <phys_net> with the name of the physical network you want to associate with the bare-metal port. The Bare Metal Provisioning service uses the physical network to map the Networking service virtual ports to physical ports and port groups. If not set then any VIF is mapped to that port when there no bare-metal port with a suitable physical network assignment exists. Exit the openstackclient pod: 5.6. Configuring port groups in the Bare Metal Provisioning service Note Port group functionality for bare-metal nodes is available in this release as a Technology Preview , and therefore is not fully supported by Red Hat. It should be used only for testing, and should not be deployed in a production environment. For more information about Technology Preview features, see Scope of Coverage Details . Port groups (bonds) provide a method to aggregate multiple network interfaces into a single "bonded" interface. Port group configuration always takes precedence over an individual port configuration. During interface attachment, port groups have a higher priority than the ports, so they are used first. Currently, it is not possible to specify preference for port or port group in an interface attachment request. If a port group is available, the interface attachment will use it. Port groups that do not have any ports are ignored. If a port group has a physical network, then all the ports in that port group must have the same physical network. The Bare Metal Provisioning service uses configdrive to support configuration of port groups in the instances. Note Bare Metal Provisioning service API version 1.26 supports port group configuration. To configure port groups in a bare metal deployment, you must configure the port groups on the switches manually. You must ensure that the mode and properties on the switch correspond to the mode and properties on the bare metal side as the naming can vary on the switch. Note You cannot use port groups for provisioning and cleaning if you need to boot a deployment using iPXE. With port group fallback, all the ports in a port group can fallback to individual switch ports when a connection fails. Based on whether a switch supports port group fallback or not, you can use the --support-standalone-ports and --unsupport-standalone-ports options. 5.6.1. Prerequisites The RHOSO environment includes the Bare Metal Provisioning service. For more information, see Enabling the Bare Metal Provisioning service (ironic) . You are logged on to a workstation that has access to the RHOCP cluster, as a user with cluster-admin privileges. The oc command line tool is installed on the workstation. 5.6.2. Configuring port groups in the Bare Metal Provisioning service Create a port group to aggregate multiple network interfaces into a single bonded interface. Procedure Access the remote shell for the OpenStackClient pod from your workstation: Create a port group: Replace <node_uuid> with the UUID of the node that this port group belongs to. Replace <group_name> with the name for this port group. Optional: Replace <mac_address> with the MAC address for the port group. If you do not specify an address, the deployed instance port group address is the same as the Networking service port. If you do not attach the Networking service port, the port group configuration fails. Optional: Replace <mode> with mode of the port group. Specify if the group supports fallback to standalone ports. Note You must configure port groups manually in standalone mode either in the image or by generating the configdrive and adding it to the node's instance_info . Ensure that you have cloud-init version 0.7.7 or later for the port group configuration to work. Associate a port with a port group: During port creation: During port update: Boot an instance by providing an image that has cloud-init or supports bonding. To check if the port group is configured properly, run the following command: Here, X is a number that cloud-init generates automatically for each configured port group, starting with a 0 and incremented by one for each configured port group. Exit the openstackclient pod: 5.7. Cleaning nodes manually The Bare Metal Provisioning service (ironic) cleans nodes automatically when they are unprovisioned to prepare them for provisioning. You can perform manual cleaning on specific nodes as required. Node cleaning has two modes: Metadata only clean: Removes partitions from all disks on the node. The metadata only mode of cleaning is faster than a full clean, but less secure because it erases only partition tables. Use this mode only on trusted tenant environments. Full clean: Removes all data from all disks, using either ATA secure erase or by shredding. A full clean can take several hours to complete. Procedure Access the remote shell for the OpenStackClient pod from your workstation: Check the current state of the node: Replace <node> with the name or UUID of the node to clean. If the node is not in the manageable state, then set it to manageable : Clean the node: Replace <node> with the name or UUID of the node to clean. Replace <clean_mode> with the type of cleaning to perform on the node: erase_devices : Performs a full clean. erase_devices_metadata : Performs a metadata only clean. Wait for the clean to complete, then check the status of the node: manageable : The clean was successful, and the node is ready to provision. clean failed : The clean was unsuccessful. Inspect the last_error field for the cause of failure. Exit the openstackclient pod: 5.8. Attaching a virtual network interface (VIF) to a bare-metal instance To attach a bare-metal instance to the bare-metal network interface, the cloud user can use the Compute service (nova) or the Bare Metal Provisioning service (ironic). Compute service: Cloud users use the openstack server add network command. For more information, see Attaching a network to an instance . Note === When using the Compute service you must explicitly declare the port when creating the instance. When the Compute service makes a request to the Bare Metal Provisioning service to create an instance, the Compute service attempts to record all the VIFs the user requested to be attached in the Bare Metal Provisioning service to generate the metadata. You cannot specify which physical port to attach a VIF to when using the Compute service.If you want to explicitly declare which port to map to, then instead use the Bare Metal Provisioning service to create the attachment. === Bare Metal Provisioning service: Cloud users use the openstack baremetal node vif attach command to attach a VIF to a bare-metal instance. For more information about virtual network interfaces (VIFs), see Bare Metal Provisioning service virtual network interfaces (VIFs) . The following procedure uses the Bare Metal Provisioning service to attach a bare-metal instance to a network. The Bare Metal Provisioning service creates the VIF attachment by using the UUID of the port you created with the Networking service . Procedure Access the remote shell for the OpenStackClient pod from your workstation: Retrieve the UUID of the bare-metal instance you want to attach the VIF to: Retrieve the UUID of the VIF you want to attach to your node: Optional: Retrieve the UUID of the bare-metal port you want to map the VIF to: Attach the VIF to your bare-metal instance: Optional: Replace <port_uuid> with the UUID of the bare-metal port to attach the VIF to. Replace <node> with the name or UUID of the bare-metal instance you want to attach the VIF to. Replace <vif_id> with the name or UUID of the VIF to attach to the bare-metal instance. Exit the openstackclient pod: 5.8.1. How the Bare Metal Provisioning service attaches the VIF to a bare-metal instance When a cloud user requests that a virtual network interface (VIF) is attached to their bare-metal instance by using the openstack baremetal node vif attach command without a declared port or port group preference, the Bare Metal Provisioning service (ironic) selects a suitable unattached port or port group by evaluating the following criteria in order: Ports or port groups do not have a physical network or have a physical network that matches one of the VIF's available physical networks. Prefer ports and port groups that have a physical network to ports and port groups that do not have a physical network. Prefer port groups to ports. Prefer ports with PXE enabled. When the Bare Metal Provisioning service attaches any VIF to a bare-metal instance it explicitly sets the MAC address for the physical port to which the VIF is bound. If a node is already in an ACTIVE state, then the Networking service (neutron) updates the VIF attachment. When the Bare Metal Provisioning service unbinds the VIF, it makes a request to the Networking service to reset the assigned MAC address to avoid conflicts with the Networking service's unique hardware MAC address requirement. 5.8.2. Attaching and detaching virtual network interfaces The Bare Metal Provisioning service has an API that you can use to manage the mapping between virtual network interfaces. For example, the interfaces in the Networking service (neutron) and your physical interfaces (NICs). You can configure these interfaces for each bare-metal node to set the virtual network interface (VIF) to physical network interface (PIF) mapping logic. Procedure Access the remote shell for the OpenStackClient pod from your workstation: List the VIF IDs that are connected to the bare-metal node: Replace <node> with the name or UUID of the bare-metal node. After the VIF is attached, the Bare Metal Provisioning service updates the virtual port in the Networking service with the MAC address of the physical port. Check this port address: Create a new port on the network where you created the bare-metal node: Remove the port from the bare-metal instance it was attached to: Check that the IP address no longer exists on the list: Check if there are VIFs attached to the node: Add the newly created port: Verify that the new IP address shows the new port: Check if the VIF ID is the UUID of the new port: Check if the Networking service port MAC address is updated and matches one of the Bare Metal Provisioning service ports: Reboot the bare-metal node so that it recognizes the new IP address: After you detach or attach interfaces, the bare-metal OS removes, adds, or modifies the network interfaces that have changed. When you replace a port, a DHCP request obtains the new IP address, but this might take some time because the old DHCP lease is still valid. To initiate these changes immediately, reboot the bare-metal node.

Commands:
oc rsh -n openstack openstackclient
openstack server create --nic net-id=<network_uuid> --flavor baremetal --image <image_uuid> myBareMetalInstance
openstack server list --name myBareMetalInstance
exit
oc rsh -n openstack openstackclient
openstack baremetal node set --deploy-interface ramdisk
openstack baremetal node set <node_UUID> --instance-info boot_iso=<boot_iso_url>
openstack baremetal node deploy <node_UUID>
exit
oc rsh -n openstack openstackclient
openstack port create --network <network> <name>
openstack baremetal port create <physical_mac_address> --node <node_uuid> --local-link-connection switch_id=<switch_mac_address> --local-link-connection switch_info=<switch_hostname> --local-link-connection port_id=<switch_port_for_connection> --pxe-enabled true --physical-network <phys_net>
exit
oc rsh -n openstack openstackclient
openstack baremetal port group create --node <node_uuid> --name <group_name> [--address <mac_address>] [--mode <mode>] --property miimon=100 --property xmit_hash_policy="layer2+3" [--support-standalone-ports]
openstack baremetal port create --node <node_uuid> --address <mac_address> --port-group <group_name>
openstack baremetal port set <port_uuid> --port-group <group_uuid>
cat /proc/net/bonding/bondX
exit
oc rsh -n openstack openstackclient
openstack baremetal node show -f value -c provision_state <node>
openstack baremetal node manage <node>
openstack baremetal node clean <node> --clean-steps '[{"interface": "deploy", "step": "<clean_mode>"}]'
exit
oc rsh -n openstack openstackclient
openstack server list
openstack port list
openstack baremetal port list
openstack baremetal node vif attach [--port-uuid <port_uuid>] <node> <vif_id>
exit
oc rsh -n openstack openstackclient
openstack baremetal node vif list <node> +--------------------------------------+ | ID | +--------------------------------------+ | 4475bc5a-6f6e-466d-bcb6-6c2dce0fba16 | +--------------------------------------+
openstack port show 4475bc5a-6f6e-466d-bcb6-6c2dce0fba16 -c mac_address -c fixed_ips +-------------+-----------------------------------------------------------------------------+ | Field | Value | +-------------+-----------------------------------------------------------------------------+ | fixed_ips | ip_address='192.168.24.9', subnet_id='1d11c677-5946-4733-87c3-23a9e06077aa' | | mac_address | 00:2d:28:2f:8d:95 | +-------------+-----------------------------------------------------------------------------+
openstack port create --network baremetal --fixed-ip ip-address=192.168.24.24 <port_name>
openstack server remove port <instance_name> 4475bc5a-6f6e-466d-bcb6-6c2dce0fba16
openstack server list
openstack baremetal node vif list <node> openstack port list
openstack server add port <instance_name> <port_name>
openstack server list
openstack baremetal node vif list <node> +--------------------------------------+ | ID | +--------------------------------------+ | 6181c089-7e33-4f1c-b8fe-2523ff431ffc | +--------------------------------------+
openstack port show 6181c089-7e33-4f1c-b8fe-2523ff431ffc -c mac_address -c fixed_ips +-------------+------------------------------------------------------------------------------+ | Field | Value | +-------------+------------------------------------------------------------------------------+ | fixed_ips | ip_address='192.168.24.24', subnet_id='1d11c677-5946-4733-87c3-23a9e06077aa' | | mac_address | 00:2d:28:2f:8d:95 | +-------------+------------------------------------------------------------------------------+
openstack server reboot overcloud-baremetal-0