Chapter 8. Troubleshooting the Bare Metal Service The following sections contain information and steps that may be useful for diagnosing issues in a setup with the Bare Metal service enabled. 8.1. PXE Boot Errors Permission Denied Errors If you get a permission denied error on the console of your Bare Metal service node, ensure that you have applied the appropriate SELinux context to the /httpboot and /tftpboot directories as follows: Boot Process Freezes at /pxelinux.cfg/XX-XX-XX-XX-XX-XX On the console of your node, if it looks like you are getting an IP address and then the process stops as shown below: This indicates that you might be using the wrong PXE boot template in your ironic.conf file. The default template is pxe_config.template , so it is easy to omit the i and inadvertently turn this into ipxe_config.template . 8.2. Login Errors After the Bare Metal Node Boots When you try to log in at the login prompt on the console of the node with the root password that you set in the configurations steps, but are not able to, it indicates you are not booted in to the deployed image. You are probably stuck in the deploy-kernel/deploy-ramdisk image and the system has yet to get the correct image. To fix this issue, verify the PXE Boot Configuration file in the /httpboot/pxelinux.cfg/MAC_ADDRESS on the Compute or Bare Metal service node and ensure that all the IP addresses listed in this file correspond to IP addresses on the Bare Metal network. Note The only network the Bare Metal service node knows about is the Bare Metal network. If one of the endpoints is not on the network, the endpoint cannot reach the Bare Metal service node as a part of the boot process. For example, the kernel line in your file is as follows: Value in the above example kernel line Corresponding information http://192.168.200.2:8088 Parameter http_url in /etc/ironic/ironic.conf file. This IP address must be on the Bare Metal network. 5a6cdbe3-2c90-4a90-b3c6-85b449b30512 UUID of the baremetal node in openstack baremetal node list . deploy_kernel This is the deploy kernel image in the Image service that is copied down as /httpboot/<NODE_UUID>/deploy_kernel . http://192.168.200.2:6385 Parameter api_url in /etc/ironic/ironic.conf file. This IP address must be on the Bare Metal network. ipmi The IPMI Driver in use by the Bare Metal service for this node. deploy_ramdisk This is the deploy ramdisk image in the Image service that is copied down as /httpboot/<NODE_UUID>/deploy_ramdisk . If a value does not correspond between the /httpboot/pxelinux.cfg/MAC_ADDRESS and the ironic.conf file: Update the value in the ironic.conf file Restart the Bare Metal service Re-deploy the Bare Metal instance 8.3. The Bare Metal Service Is Not Getting the Right Hostname If the Bare Metal service is not getting the right hostname, it means that cloud-init is failing. To fix this, connect the Bare Metal subnet to a router in the OpenStack Networking service. The requests to the meta-data agent should now be routed correctly. 8.4. Invalid OpenStack Identity Service Credentials When Executing Bare Metal Service Commands If you are having trouble authenticating to the Identity service, check the identity_uri parameter in the ironic.conf file and ensure that you remove the /v2.0 from the keystone AdminURL. For example, set the identity_uri to http://IP:PORT . 8.5. Hardware Enrollment Issues with enrolled hardware can be caused by incorrect node registration details. Ensure that property names and values have been entered correctly. Incorrect or mistyped property names will be successfully added to the node's details, but will be ignored. Update a node's details. This example updates the amount of memory the node is registered to use to 2 GB: 8.6. No Valid Host Errors If the Compute scheduler cannot find a suitable Bare Metal node on which to boot an instance, a NoValidHost error can be seen in /var/log/nova/nova-conductor.log or immediately upon launch failure in the dashboard. This is usually caused by a mismatch between the resources Compute expects and the resources the Bare Metal node provides. Check the hypervisor resources that are available: The resources reported here should match the resources that the Bare Metal nodes provide. Check that Compute recognizes the Bare Metal nodes as hypervisors: The nodes, identified by UUID, should appear in the list. Check the details for a Bare Metal node: Verify that the node's details match those reported by Compute. Check that the selected flavor does not exceed the available resources of the Bare Metal nodes: Check the output of openstack baremetal node list to ensure that Bare Metal nodes are not in maintenance mode. Remove maintenance mode if necessary: Check the output of openstack baremetal node list to ensure that Bare Metal nodes are in an available state. Move the node to available if necessary: 8.7. Troubleshooting iDRAC issues Redfish management interface fails to set boot device When you use the idrac-redfish management interface with certain iDRAC firmware versions and attempt to set the boot device on a bare metal server with UEFI boot, iDRAC returns the following error: If you encounter this issue, set the force_persistent_boot_device parameter in the driver-info on the node to Never : Timeout when powering off Some servers can be too slow when powering off, and time out. The default retry count is 6 , which results in a 30 second timeout. To increase the timeout duration to 90 seconds, set the ironic::agent::rpc_response_timeout value to 18 in the undercloud hieradata overrides file and re-run the openstack undercloud install command: Vendor passthrough timeout When iDRAC is not available to execute vendor passthrough commands, these commands take too long and time out: To increase the timeout duration for messaging, increase the value of the ironic::default::rpc_response_timeout parameter in the undercloud hieradata overrides file and re-run the openstack undercloud install command:

Commands:
semanage fcontext -a -t httpd_sys_content_t "/httpboot(/.*)?" restorecon -r -v /httpboot semanage fcontext -a -t tftpdir_t "/tftpboot(/.*)?" restorecon -r -v /tftpboot
grep ^pxe_config_template ironic.conf pxe_config_template=USDpybasedir/drivers/modules/ipxe_config.template
kernel http://192.168.200.2:8088/5a6cdbe3-2c90-4a90-b3c6-85b449b30512/deploy_kernel selinux=0 disk=cciss/c0d0,sda,hda,vda iscsi_target_iqn=iqn.2008-10.org.openstack:5a6cdbe3-2c90-4a90-b3c6-85b449b30512 deployment_id= 5a6cdbe3-2c90-4a90-b3c6-85b449b30512 deployment_key=VWDYDVVEFCQJNOSTO9R67HKUXUGP77CK ironic_api_url= http://192.168.200.2:6385 troubleshoot=0 text nofb nomodeset vga=normal boot_option=netboot ip=USD{ip}:USD{next-server}:USD{gateway}:USD{netmask} BOOTIF=USD{mac} ipa-api-url= http://192.168.200.2:6385 ipa-driver-name= ipmi boot_mode=bios initrd= deploy_ramdisk coreos.configdrive=0 || goto deploy
openstack baremetal node set --property memory_mb=2048 NODE_UUID
openstack hypervisor stats show
openstack hypervisor list
openstack baremetal node list openstack baremetal node show NODE_UUID
openstack flavor show FLAVOR_NAME
openstack baremetal node maintenance unset NODE_UUID
openstack baremetal node provide NODE_UUID
Unable to Process the request because the value entered for the parameter Continuous is not supported by the implementation.
openstack baremetal node set --driver-info force_persistent_boot_device=Never USD{node_uuid}
ironic::agent::rpc_response_timeout: 18
openstack baremetal node passthru call --http-method GET aed58dca-1b25-409a-a32f-3a817d59e1e0 list_unfinished_jobs Timed out waiting for a reply to message ID 547ce7995342418c99ef1ea4a0054572 (HTTP 500)
ironic::default::rpc_response_timeout: 600