Chapter 9. Automatically provisioning and onboarding RHEL for Edge devices with FDO You can build a RHEL for Edge Simplified Installer image, and provision it to a RHEL for Edge image. The FIDO Device Onboarding (FDO) process automatically provisions and onboards your Edge devices, and exchanges data with other devices and systems connected on the networks. Important Red Hat provides the FDO process as a Technology Preview feature and should run on secure networks. Technology Preview features are not supported with Red Hat production service level agreements (SLAs) and might not be functionally complete. These features provide early access to upcoming product features, enabling customers to test functionality and provide feedback during the development process. See Technology Preview Features Support Scope on the Red Hat Customer Portal for information about the support scope for Technology Preview features. 9.1. The FIDO Device Onboarding (FDO) process The FIDO Device Onboarding (FDO) is the process that: Provisions and onboards a device. Automatically configures credentials for this device. The FDO process is an automatic onboarding mechanism that is triggered by the installation of a new device. Enables this device to securely connect and interact on the network. With FIDO Device Onboarding (FDO), you can perform a secure device onboarding by adding new devices into your IoT architecture. This includes the specified device configuration that needs to be trusted and integrated with the rest of the running systems. The FDO process is an automatic onboarding mechanism that is triggered by the installation of a new device. The FDO protocol performs the following tasks: Solves the trust and chain of ownership along with the automation needed to securely onboard a device at scale. Performs device initialization at the manufacturing stage and late device binding for its actual use. This means that actual binding of the device to a management system happens on the first boot of the device without requiring manual configuration on the device. Supports automated secure devices onboarding, that is, zero touch installation and onboarding that does not need any specialized person at the edge location. After the device is onboarded, the management platform can connect to it and apply patches, updates, and rollbacks. With FDO, you can benefit from the following: FDO is a secure and simple way to enroll a device to a management platform. Instead of embedding a Kickstart configuration to the image, FDO applies the device credentials during the device first boot directly to the ISO image. FDO solves the issue of late binding to a device, enabling any sensitive data to be shared over a secure FDO channel. FDO cryptographically identifies the system identity and ownership before enrolling and passing the configuration and other secrets to the system. That enables non-technical users to power-on the system. To build a RHEL for Edge Simplified Installer image and automatically onboard it, provide an existing OSTree commit. The resulting simplified image contains a raw image that has the OSTree commit deployed. After you boot the Simplified installer ISO image, it provisions a RHEL for Edge system that you can use on a hard disk or as a boot image in a virtual machine. The RHEL for Edge Simplified Installer image is optimized for unattended installation to a device and supports both network-based deployment and non-network-based deployments. However, for network-based deployment, it supports only UEFI HTTP boot. The FDO protocol is based on the following servers: Manufacturing server Generates the device credentials. Creates an Ownership voucher that is used to set the ownership of the device, later in the process. Binds the device to a specific management platform. Owner management system Receives the Ownership voucher from the Manufacturing server and becomes the owner of the associated device. Later in the process, it creates a secure channel between the device and the Owner onboarding server after the device authentication. Uses the secure channel to send the required information, such as files and scripts for the onboarding automation to the device. Service-info API server Based on Service-info API server's configuration and modules available on the client, it performs the final steps of onboarding on target client devices, such as copying SSH keys and files, executing commands, creating users, encrypting disks and so on Rendezvous server Gets the Ownership voucher from the Owner management system and makes a mapping of the device UUID to the Owner server IP. Then, the Rendezvous server matches the device UUID with a target platform and informs the device about which Owner onboarding server endpoint this device must use. During the first boot, the Rendezvous server will be the contact point for the device and it will direct the device to the owner, so that the device and the owner can establish a secure channel. Device client This is installed on the device. The Device client performs the following actions: Starts the queries to the multiple servers where the onboarding automation will be executed. Uses TCP/IP protocols to communicate with the servers. The following diagram represents the FIDO device onboarding workflow: Figure 9.1. Deploying RHEL for Edge in non-network environment At the Device Initialization , the device contacts the Manufacturing server to get the FDO credentials, a set of certificates and keys to be installed on the operating system with the Rendezvous server endpoint (URL). It also gets the Ownership Voucher, that is maintained separately in case you need to change the owner assignment. The Device contacts the Manufacturing server The Manufacturing server generates an Ownership Voucher and the Device Credentials for the Device. The Ownership Voucher is transferred to the Owner onboarding server. At the On-site onboarding , the Device gets the Rendezvous server endpoint (URL) from its device credentials and contacts Rendezvous server endpoint to start the onboarding process, which will redirect it to the Owner management system, that is formed by the Owner onboarding server and the Service Info API server. The Owner onboarding server transfers the Ownership Voucher to the Rendezvous server, which makes a mapping of the Ownership Voucher to the Owner. The device client reads device credentials. The device client connects to the network. After connecting to the network, the Device client contacts the Rendezvous server. The Rendezvous server sends the owner endpoint URL to the Device Client, and registers the device. The Device client connects to the Owner onboarding server shared by the Rendezvous server. The Device proves that it is the correct device by signing a statement with a device key. The Owner onboarding server proves itself correct by signing a statement with the last key of the Owner Voucher. The Owner onboarding server transfers the information of the Device to the Service Info API server. The Service info API server sends the configuration for the Device. The Device is onboarded. 9.2. Automatically provisioning and onboarding RHEL for Edge devices To build a RHEL for Edge Simplified Installer image and automatically onboard it, provide an existing OSTree commit. The resulting simplified image contains a raw image that has the OSTree commit deployed. After you boot the Simplified installer ISO image, it provisions a RHEL for Edge system that you can use on a hard disk or as a boot image in a virtual machine. The RHEL for Edge Simplified Installer image is optimized for unattended installation to a device and supports both network-based deployment and non-network-based deployments. However, for network-based deployment, it supports only UEFI HTTP boot. Automatically provisioning and onboarding a RHEL for Edge device involves the following high-level steps: Install and register a RHEL system Install RHEL image builder By using RHEL image builder, create a blueprint with customizations for RHEL for a rhel-edge-container image type. Import the RHEL for Edge Container blueprint in RHEL image builder Create a RHEL for Edge Container image Use the RHEL for Edge Container image to serve the OSTree commit, which will be later used when building the RHEL for Edge Simplified Installer image type Create a blueprint for and edge-simplified-installer image type with customizations for storage device path and FDO customizations Build a simplified installer RHEL for Edge image Download the RHEL for Edge simplified installer image At this point, the FDO server infrastructure should be up and running, and the specific onboarding details handled by the service-info API server, that is part of the owner's infrastructure, are configured Install the simplified installer ISO image to a device. The FDO client runs on the Simplified Installer ISO and the UEFI directory structure makes the image bootable. The network configuration enables the device to reach out to the manufacturing server to perform the initial device credential exchange. After the system reaches the endpoint, the device credentials are created for the device. The device uses the device credentials to reach the Rendezvous server, where it checks the cryptographic credentials based on the vouchers that the Rendezvous server has, and then the Rendezvous server redirects the device to the Owner server. The device contacts the Owner server. They establish a mutual trust and the final steps of onboarding happen based on the configuration of the Service-info API server. For example, it installs the SSH keys in the device, transfer the files, create the users, run the commands, encrypt the filesystem, and so on. Additional resources FDO automatic onboarding technologies 9.3. Generating key and certificates To run the FIDO Device Onboarding (FDO) infrastructure, you need to generate keys and certificates. FDO generates these keys and certificates to configure the manufacturing server. FDO automatically generates the certificates and .yaml configuration files when you install the services, and re-creating them is optional. After you install and start the services, it runs with the default settings. Important Red Hat provides the fdo-admin-tool tool as a Technology Preview feature and should run on secure networks. Technology Preview features are not supported with Red Hat production service level agreements (SLAs) and might not be functionally complete. These features provide early access to upcoming product features, enabling customers to test functionality and provide feedback during the development process. See Technology Preview Features Support Scope on the Red Hat Customer Portal for information about the support scope for Technology Preview features. Prerequisites You installed the fdo-admin-cli RPM package Procedure Generate the keys and certificates in the /etc/fdo directory: Check the key and certificates that were created in the /etc/fdo/keys directory: You can see the following output: Additional resources See the fdo-admin-tool generate-key-and-cert --help manual page 9.4. Installing and running the manufacturing server The fdo-manufacturing-server RPM package enables you to run the Manufacturing Server component of the FDO protocol. It also stores other components, such as the owner vouchers, the manufacturer keys, and information about the manufacturing sessions. During the device installation, the Manufacturing server generates the device credentials for the specific device, including its GUID, rendezvous information and other metadata. Later on in the process, the device uses this rendezvous information to contact the Rendezvous server. Important Red Hat provides the fdo-manufacturing-server tool as a Technology Preview feature and should run on secure networks because Technology Preview features are not supported with Red Hat production service level agreements (SLAs) and might not be functionally complete. These features provide early access to upcoming product features, enabling customers to test functionality and provide feedback during the development process. See Technology Preview Features Support Scope on the Red Hat Customer Portal for information about the support scope for Technology Preview features. To install the manufacturing server RPM package, complete the following steps: Procedure Install the fdo-admin-cli package: Check if the fdo-manufacturing-server RPM package is installed: Check if the files were correctly installed: You can see the following output: Optional: Check the content of each file, for example: Configure the Manufacturing server. You must provide the following information: The Manufacturing server URL The IP address or DNS name for the Rendezvous server The path to the keys and certificates you generated. See Generating key and certificates . You can find an example of a Manufacturing server configuration file in the /usr/share/doc/fdo/manufacturing-server.yml directory. The following is a manufacturing server.yml example that is created and saved in the /etc/fdo directory. It contains paths to the directories, certificates, keys that you created, the Rendezvous server IP address and the default port. Start the Manufacturing server. Check if the systemd unit file are in the server: Enable and start the manufacturing server. Open the default ports in your firewall: Ensure that the service is listening on the port 8080: Install RHEL for Edge onto your system using the simplified installer. See Building simplified installer images to provision a RHEL for Edge image . Additional resources The manufacturing-server.yml example FDO automatic onboarding terminology 9.5. Installing, configuring, and running the Rendezvous server Install the fdo-rendezvous-server RPM package to enable the systems to receive the voucher generated by the Manufacturing server during the first device boot. The Rendezvous server then matches the device UUID with the target platform or cloud and informs the device about which Owner server endpoint the device must use. Prerequisites You created a manufacturer_cert.pem certificate. See Generating key and certificates . You copied the manufacturer_cert.pem certificate to the /etc/fdo/keys directory in the Rendezvous server. Procedure Install the fdo-rendezvous-server RPM packages: Create the rendezvous-server.yml configuration file, including the path to the manufacturer certificate. You can find an example in /usr/share/doc/fdo/rendezvous-server.yml . The following example shows a configuration file that is saved in /etc/fdo/rendezvous-server.yml . Check the Rendezvous server service status: If the service is stopped and disabled, enable and start it: Check that the server is listening on the default configured port 8082: Open the port if you have a firewall configured on this server: 9.6. Installing, configuring, and running the Owner server Install the fdo-owner-cli and fdo-owner-onboarding-server RPM package to enable the systems to receive the voucher generated by the Manufacturing server during the first device boot. The Rendezvous server then matches the device UUID with the target platform or cloud and informs the device about which Owner server endpoint the device must use. Prerequisites The device where the server will be deployed has a Trusted Platform Module (TPM) device to encrypt the disk. If not, you will get an error when booting the RHEL for Edge device. You created the device_ca_cert.pem , owner_key.der , and owner_cert.pem with keys and certificates and copied them into the /etc/fdo/keys directory. Procedure Install the required RPMs in this server: Prepare the owner-onboarding-server.yml configuration file and save it to the /etc/fdo/ directory. Include the path to the certificates you already copied and information about where to publish the Owner server service in this file. The following is an example available in /usr/share/doc/fdo/owner-onboarding-server.yml . You can find references to the Service Info API, such as the URL or the authentication token. Create and configure the Service Info API. Add the automated information for onboarding, such as user creation, files to be copied or created, commands to be executed, disk to be encrypted, and so on. Use the Service Info API configuration file example in /usr/share/doc/fdo/serviceinfo-api-server.yml as a template to create the configuration file under /etc/fdo/ . Check the status of the systemd units: If the service is stopped and disabled, enable and start it: Note You must restart the systemd services every time you change the configuration files. Check that the server is listening on the default configured port 8083: Open the port if you have a firewall configured on this server: 9.7. Automatically onboarding a RHEL for Edge device by using FDO authentication To prepare your device to automatically onboard a RHEL for Edge device and provision it as part of the installation process, complete the following steps: Prerequisites You built an OSTree commit for RHEL for Edge and used that to generate an edge-simplified-installer artifact. Your device is assembled. You installed the fdo-manufacturing-server RPM package. See Installing the manufacturing server package . Procedure Start the installation process by booting the RHEL for Edge simplified installer image on your device. You can install it from a CD-ROM or from a USB flash drive, for example. Verify through the terminal that the device has reached the manufacturing service to perform the initial device credential exchange and has produced an ownership voucher. You can find the ownership voucher at the storage location configured by the ownership_voucher_store_driver: parameter at the manufacturing-sever.yml file. The directory should have an ownership_voucher file with a name in the GUID format which indicates that the correct device credentials were added to the device. The onboarding server uses the device credential to authenticate against the onboarding server. It then passes the configuration to the device. After the device receives the configuration from the onboarding server, it receives an SSH key and installs the operating system on the device. Finally, the system automatically reboots, encrypts it with a strong key stored at TPM. Verification After the device automatically reboots, you can log in to the device with the credentials that you created as part of the FDO process. Log in to the device by providing the username and password you created in the Service Info API. Additional resources Deploying the Simplified ISO image from a USB flash drive

Commands:
name = "rhel-edge-container" description = "Minimal RHEL for Edge Container blueprint" version = "0.0.1"
name = "rhel-edge-simplified-installer-with-fdo" description = "Minimal RHEL for Edge Simplified Installer with FDO blueprint" version = "0.0.1" packages = [] modules = [] groups = [] distro = "" [customizations] installation_device = "/dev/vda" [customizations.fdo] manufacturing_server_url = "http://10.0.0.2:8080" diun_pub_key_insecure = "true"
for i in "diun" "manufacturer" "device-ca" "owner"; do fdo-admin-tool generate-key-and-cert USDi; done ls keys device_ca_cert.pem device_ca_key.der diun_cert.pem diun_key.der manufacturer_cert.pem manufacturer_key.der owner_cert.pem owner_key.der
tree keys
- device_ca_cert.pem - device_ca_key.der - diun_cert.pem - diun_key.dre - manufacturer_cert.pem - manufacturer_key.der - owner_cert.pem - owner_key.pem
yum install -y fdo-admin-cli
rpm -qa | grep fdo-manufacturing-server --refresh
ls /usr/share/doc/fdo
Output: manufacturing-server.yml owner-onboarding-server.yml rendezvous-info.yml rendezvous-server.yml serviceinfo-api-server.yml
cat /usr/share/doc/fdo/manufacturing-server.yml
session_store_driver: Directory: path: /etc/fdo/stores/manufacturing_sessions/ ownership_voucher_store_driver: Directory: path: /etc/fdo/stores/owner_vouchers public_key_store_driver: Directory: path: /etc/fdo/stores/manufacturer_keys bind: "0.0.0.0:8080" protocols: plain_di: false diun: mfg_string_type: SerialNumber key_type: SECP384R1 allowed_key_storage_types: - Tpm - FileSystem key_path: /etc/fdo/keys/diun_key.der cert_path: /etc/fdo/keys/diun_cert.pem rendezvous_info: - deviceport: 8082 ip_address: 192.168.122.99 ownerport: 8082 protocol: http manufacturing: manufacturer_cert_path: /etc/fdo/keys/manufacturer_cert.pem device_cert_ca_private_key: /etc/fdo/keys/device_ca_key.der device_cert_ca_chain: /etc/fdo/keys/device_ca_cert.pem owner_cert_path: /etc/fdo/keys/owner_cert.pem manufacturer_private_key: /etc/fdo/keys/manufacturer_key.der
systemctl list-unit-files | grep fdo | grep manufacturing fdo-manufacturing-server.service disabled disabled
systemctl enable --now fdo-manufacturing-server.service
firewall-cmd --add-port=8080/tcp --permanent systemctl restart firewalld
ss -ltn
yum install -y fdo-rendezvous-server
storage_driver: Directory: path: /etc/fdo/stores/rendezvous_registered session_store_driver: Directory: path: /etc/fdo/stores/rendezvous_sessions trusted_manufacturer_keys_path: /etc/fdo/keys/manufacturer_cert.pem max_wait_seconds: ~ bind: "0.0.0.0:8082"
systemctl list-unit-files | grep fdo | grep rende fdo-rendezvous-server.service disabled disabled
systemctl enable --now fdo-rendezvous-server.service
ss -ltn
firewall-cmd --add-port=8082/tcp --permanent systemctl restart firewalld
dnf install -y fdo-owner-cli fdo-owner-onboarding-server
--- ownership_voucher_store_driver: Directory: path: /etc/fdo/stores/owner_vouchers session_store_driver: Directory: path: /etc/fdo/stores/owner_onboarding_sessions trusted_device_keys_path: /etc/fdo/keys/device_ca_cert.pem owner_private_key_path: /etc/fdo/keys/owner_key.der owner_public_key_path: /etc/fdo/keys/owner_cert.pem bind: "0.0.0.0:8081" service_info_api_url: "http://localhost:8083/device_info" service_info_api_authentication: BearerToken: token: Kpt5P/5flBkaiNSvDYS3cEdBQXJn2Zv9n1D50431/lo= owner_addresses: - transport: http addresses: - ip_address: 192.168.122.149 port: 8081 report_to_rendezvous_endpoint_enabled: true
--- service_info: initial_user: username: admin sshkeys: - "ssh-rsa AAAA...." files: - path: /root/resolv.conf source_path: /etc/resolv.conf commands: - command: touch args: - /root/test return_stdout: true return_stderr: true diskencryption_clevis: - disk_label: /dev/vda4 binding: pin: tpm2 config: "{}" reencrypt: true additional_serviceinfo: ~ bind: "0.0.0.0:8083" device_specific_store_driver: Directory: path: /etc/fdo/stores/serviceinfo_api_devices service_info_auth_token: Kpt5P/5flBkaiNSvDYS3cEdBQXJn2Zv9n1D50431/lo= admin_auth_token: zJNoErq7aa0RusJ1w0tkTjdITdMCWYkndzVv7F0V42Q=
systemctl list-unit-files | grep fdo fdo-owner-onboarding-server.service disabled disabled fdo-serviceinfo-api-server.service disabled disabled
systemctl enable --now fdo-owner-onboarding-server.service systemctl enable --now fdo-serviceinfo-api-server.service
ss -ltn
firewall-cmd --add-port=8081/tcp --permanent firewall-cmd --add-port=8083/tcp --permanent systemctl restart firewalld