<figure><img src="image_1.png" /></figure>

ifip

International Federation for Information Processing

This document is the original author manuscript of a paper submitted to an IFIP
conference proceedings or other IFIP publication by Springer Nature. As such, there
may be some differences in the official published version of the paper. Such
differences, if any, are usually due to reformatting during preparation for publication or
minor corrections made by the author(s) during final proofreading of the publication
manuscript.
Chapter 10

EVALUATING THE USE OF BOOT
IMAGE ENCRYPTION ON THE
TALOS II ARCHITECTURE

Calvin Muramoto, Scott Graham and Stephen Dunlap

**Abstract** Critical infrastructure devices operating in unprotected end-node environments are vulnerable to malicious actors who conduct hardware attacks such as reverse engineering and side-channel analysis. Boot data is rarely encrypted and typically travels across an accessible bus, enabling the data to be easily intercepted during system start-up. Encrypting the firmware would make reverse engineering extremely difficult for malicious actors and competitors. It would improve the effectiveness of tamper detection methods and deter zero-day vulnerability discovery. Increasing boot security could be a fundamental part of decreasing attack surfaces across the critical infrastructure sectors.

This chapter describes a Talos II architecture implementation that encrypts a section of the boot image and decrypts it during initial program load. During power-on, the encrypted image travels across the Low Pin Count bus into a POWER9 module Level 3 cache and is decrypted in the processor. Boot image encryption is implemented using ciphers of different strengths. An analysis of their efficiency is conducted to determine the optimal algorithm.

**Keywords:** Secure boot, hardware security, firmware encryption, Talos II system

# 1. Introduction

As the demand for critical infrastructure grows, the systems required to maintain consistent and reliable operation increase in complexity. This opens up attack surfaces for malicious actors and competitors because the systems are often placed in unprotected environments [3].

Electric power grids use phasor measurement units as sensors to measure voltages and current phasors in the power system. The data collected by each unit is sent to a phasor data collector that may contain
sensitive information. The phasor measurement units must be able to measure the power grid at system end-points, resulting in their placement in relatively-insecure environments [10]. Manipulating the data collected at these locations can cause problems, including power grid disruptions. Similar problems are also encountered in water management systems where water pressure sensors and data collection systems are placed at unprotected end-nodes. Protecting the boot process of devices such as phasor measurement units and water pressure sensors would require the addition of a layer of hardware security to industrial control systems.

Encrypting boot firmware can help prevent unauthorized entities from discovering vulnerabilities in infrastructure computer systems. Tamper detection checks should be executed during system boot sequences to detect intruders before the operating systems execute. However, an unauthorized entity who can observe the boot instructions executed during start-up may be able to avoid tamper detection to manipulate and gain access to the system. Boot firmware is rarely encrypted under the assumption that an unauthorized entity does not have physical access to the hardware. As the complexity and frequency of cyber attacks increase, implementing hardware security becomes a priority. Absent hardware security, adequate security can be achieved by encrypting boot firmware when it is stored in memory and decrypting it in the processor when the instructions are to be executed.

Secure boot has become a standard for improving hardware security – it works by ensuring that only signed firmware is executed on a system. Although it can prevent unauthorized firmware from executing on a system processor, it cannot protect against physical hardware attacks such as chip substitution and bus traffic recording [5]. This research attempts to prevent these types of attacks via boot image encryption. Efforts have also been made to encrypt bootloader firmware that is decrypted during start-up in microcontroller platforms [8]. However, microcontroller bootloaders are very simple and typically employ only two start-up stages. This research focuses on expanding the encrypted microcontroller boot firmware concept to a workstation incorporating a powerful POWER9 module.

Although this research implements boot image encryption on the Talos II architecture, it could also be applied to any system that utilizes a multi-stage bootloader. For example, Intel and Advanced Micro Devices include bootloaders in BIOS chips that function similarly to the Talos II firmware. Expanding the implementation of boot firmware encryption from Talos II to other architectures should be straightforward as long as their initial program load flows are similar.
Previous research has detailed the notional process of encrypting a section of a boot image along with the requirements to decrypt it during initial program load [9]. The research employed a simple XOR cipher to demonstrate a minimalist encryption approach as a proof of concept. This research extends the previous work by incorporating real encryption algorithms that require more processing and storage for boot images and processors. SPECK [1] was selected as the lightweight cipher and Advanced Encryption Standard (AES) as the heavyweight cipher to compare runtime performance. An analysis of the three boot firmware encryption methods is presented to provide a comprehensive comparison.

# 2. Background

This section describes the Talos II architecture, processor-based NOR (PNOR) image structure, initial program load, ciphers employed and secure key management.

## 2.1 Talos II Architecture

A Talos II workstation was selected as the platform to develop and test the boot encryption implementation. Designed by Raptor Computing Systems, Talos II is the world's first owner-controllable workstation-class motherboard that is compatible with open-source firmware. The Talos II architecture supports dual POWER9 central processing units (CPUs) and trusted boot, and is compatible with OpenBMC. The OpenBMC compatibility enables Talos II to boot from custom firmware, which is essential to this research.

A baseboard management controller (BMC) is a specialized service processor residing on a motherboard that monitors all physical and network data. It is primarily used in server environments to control and monitor multiple systems to ensure normal operation. The baseboard management controller provides an avenue for a system administrator to communicate with the server, specifically to enable remote power cycling and rebooting. Having full control of the baseboard management controller firmware enables the instructions executed during initial program load to be modified [4].

Initial program load refers to the operations that the Talos II system executes from power-on until the operating system starts up. During the initial start-up, several interfaces are used to transfer data between the PNOR flash memory, random access memory and POWER9 module. The Intelligent Platform Management Interface and Fast Serial Interface are two important communications interfaces that are used later during the initial program load. Since this research focuses on the early
portions of the boot flow, Intelligent Platform Management Interface and Fast Serial Interface are not covered. However, the Low Pin Count (LPC) and Pervasive Interconnect Bus (PIB) interfaces are essential to the boot process. The Low Pin Count bus connects the POWER9 module to external systems such as the baseboard management controller. If trusted boot is enabled, the Low Pin Count bus is used to communicate with the trusted platform module [5]. The Pervasive Interconnect bus located inside each POWER9 CPU provides read/write access to the various attached components such as the on-chip controller (OCC), self boot engine (SBE) and serial electrically-erasable programmable read-only memory (SEEPROM). All data transfers within the processor travel across the Pervasive Interconnect Bus and are considered to be secure because they are contained within the POWER9 module.

The Pervasive Interconnect Bus connects several components essential to starting the boot flow, including the self boot engine and SEEPROM [4]. The self boot engine is an auxiliary microprocessor in a POWER9 CPU that initializes the first core to begin the initial program load process. It executes from a Programmable PowerPC-lite Engine and its firmware is stored in the SEEPROM. Two redundant copies of the initial boot stage as a backup in the event that the system fails to reach the second stage bootloader. The SEEPROM also stores the root of trust hash when secure boot is enabled.

## 2.2 PNOR Image Structure

Memory space limitations imposed by the Level 3 (L3) cache of the POWER9 module are significant concerns in this research. The beginning portion of the initial program load executes code retrieved from the processor cache, which is limited to 10 MB. This could cause boot problems if the firmware requires more space than is available in the L3 cache. Another problem is posed by the limited amount of space for a PNOR image. Each partition in a PNOR image is allocated a specific amount of space so that the entire image can fit on the flash chip. Although a PNOR image has built-in buffer space for each partition, problems could occur if the compiled firmware overflows the space. This concern was addressed by ensuring that space requirement checks are performed during the firmware compilation stage.

The PNOR flash image contains all the instructions needed to boot the Talos II system. The image split into several sections defined in the table of contents (Figure 1). The table of contents contains essential data, including the partition name, physical offset and physical size. It is located at the beginning and end of the PNOR image and is queried
<figure><img src="image_2.png" /><figcaption>Figure 1. High-level PNOR flash image layout [4].</figcaption></figure>

multiple times by the POWER9 module during the initial program load. The PNOR version specific to booting the Talos II system has a capacity of 4 MB and contains 31 sections ranging from 28 KB of memory configuration to 1.8 MB of boot kernel firmware data [11]. Each section of the PNOR image contains a firmware module that is used during initial program load. The most important sections relevant to this research are the Hostboot and Skiboot partitions.

## 2.3 Initial Program Load

Initial program load is a term used in OpenPOWER systems that refers to the operations executed by the systems from power-on until their operating systems start up. Figure 2 shows the five main sections of the initial program load, two self boot engine stages, Hostboot, Skiboot and Petitboot. This research focuses on the stages of the initial program load up to Skiboot because the earlier sectors are easier to alter and contain important details about start-up. The initial program load is started when the baseboard management controller sends the system start signal to the SEEPROM through the Pervasive Interconnect Bus, which initiates the first stage bootloader.
<figure><img src="image_3.png" /><figcaption>Figure 2. Firmware boot flow [4].</figcaption></figure>

**First Stage Bootloader.** When altering the self boot engine firmware, it is important to consider that a first stage bootloader faces tight memory constraints. This is because the boot sector has to work with limited flash memory and a dedicated boot processor. Any additional code in the firmware must not exceed the flash memory space and can only contain instructions that the self boot engine can execute.

The first stage bootloader in a Talos II system has two sections. The first section is permanently written to one-time programmable read-only memory (OTPROM) using e-fuses on the POWER9 silicon. The self
boot engine firmware in one-time programmable read-only memory con-
tains the first instructions executed by the engine. These instructions
load the second section, self boot engine firmware from the SEEPROM
into the self boot engine core [4]. The self boot engine firmware from
the SEEPROM is responsible for initializing the first CPU core on which
the Hostboot bootloader (HBBL) executes. The firmware also loads the
Hostboot bootloader into the L3 cache.

**Hostboot.** Hostboot is a major portion of the initial program load because it configures all the interfaces needed by Skiboot and the operating system kernel. It acts as a cache-contained operating system for self-hosting chip initialization in POWER9 platforms. Virtual memory and a virtual filesystem layer are used for demand paging to bring code out of the flash chip as necessary [6]. This is because the code and data required for Hostboot do not fit in the 10 MB of available L3 cache. Hostboot is split into three main sections comprising the Hostboot bootloader, Hostboot base (HBB) and Hostboot extended image (HBI). These sections function differently and have unique purposes related to the initial program load.

The Hostboot bootloader, which is stored in the SEEPROM, contains the first instructions executed by the CPU core initiated by the self boot engine. It cryptographically verifies Hostboot if secure boot is enabled [6] and loads the Hostboot base binary from the PNOR image into the POWER9 L3 cache and start execution on the first CPU core.

The Hostboot bootloader partition has a base initialization service
task list that starts all the services needed by the Hostboot extended
image. The Hostboot bootloader contains the hostboot kernel and ser-
vices necessary to read and write to the PNOR image. It also initializes
the DRAM, processor bus and memory buffers, acting as the foundation
for the Hostboot extended image [12].

The Hostboot extended image stage is executed via an extended ini-
tialization service task list in the form of ISTEPs [6]. Each task in
the service task list serves to move the initialization of the chip. The
Hostboot extended image contains Hostboot sub-component code with
the Hostboot extended table of contents. The last two ISTEPs in the
Hostboot extended image are to load and start Skiboot. After the final
ISTEPS are complete, Hostboot releases control of the CPU to Skiboot.

**Skiboot.** Skiboot is late-stage boot firmware that provides the Open-
POWER abstraction layer runtime services used later by Skiroot. Ski-
boot provides wider platform initialization compared with Hostboot and
initializes the Peripheral Component Interconnect Express (PCIe) con-
trollers, device trees, real-time clock and several sensors [12]. The sensors are integral to the on-chip controller, which is also started in the Skiboot boot stage. When working with the compiled PNOR image, it is important to note that Skiboot is compressed to fit 16 MB of instructions into 1 MB of space. The decompression of the Skiboot image is executed at the end of the Hostboot stage in ISTEP 20.1. After Skiboot is complete, Skiroot and Petitboot are chain-loaded.

## 2.4 Ciphers

Two ciphers were selected to encrypt sections of the boot firmware, the lightweight SPECK [1] cipher and heavyweight AES cipher. In addition, the XOR cipher used in previous work [9] was employed due its simplicity as a proof of concept and possible use as a one-time pad. These three ciphers provide good benchmarks for comparing the encryption performance during the initial program load process.

**XOR.** An XOR cipher is an encryption algorithm that essentially uses a one-time pad. The one-time pad is an encryption technique that cannot be cracked because it requires a key that is as long as the plaintext and the key can only be used once. In the implementation, the key was set to 16 bits because the key has to be as long as Skiboot to be deemed a one-time pad. This is because it would take up an additional 1 MB of valuable memory just to store the decryption key. Instead, an XOR operation was employed to evaluate the capabilities of POWER9 during initial program load.

**SPECK.** SPECK is a lightweight block cipher that is optimized for software implementation. It was developed by the National Security Agency alongside SIMON, which is also a lightweight block cipher, but is optimized for hardware implementation [1]. The SPECK and SIMON ciphers offer several options for block size and key size. The two ciphers are designed for use in Internet of Things devices due to minimal performance impacts while providing an added layer of security. SPECK was selected in this research because it is one of the best-performing lightweight ciphers and does not use lookup tables. This reduces the size of the compiled code because the tables do not have to be stored in the Hostboot source code.

**Advanced Encryption Standard.** AES was selected because it is a widely-used block cipher and an encryption standard. Although AES supports a range of block sizes, this research opted to use AES-256 because it is the most secure configuration. The block size was a concern
because the limited memory space poses potential problems during run-
time.

## 2.5 Secure Key Management

Secure key management must be investigated when attempting to use
an encrypted PNOR image. The principal challenge to using encrypted
boot firmware is that a boot stage must exist in source code within the
hardware capable of decrypting the subsequent firmware section. If the
decryption boot stage is stored on the PNOR flash chip, the plaintext
code must travel across the Low Pin Count bus from the PNOR to the
POWER9 module, which is vulnerable to attackers. The data contained
in the POWER9 module is secure because it is assumed that adversaries
do not have the ability to sniff the interfaces in the processor. This
implies that data and keys are secure if they are stored in the POWER9
module, provided no methods are available for retrieving them using
external interfaces. Under these assumptions, three candidate methods
can address secure key management.

The first candidate method requires a one-time pad storage chip to
be incorporated in the POWER9 module, which is how manufacturers
typically manage keys in computer systems. The one-time pad storage
chip is connected to the Pervasive Interconnect Bus and transmits the
decryption key to the POWER9 L3 cache when the firmware decryp-
tion code executes. Although this implementation enables secure key
management for boot firmware encryption, it requires additional hard-
ware to be added to the POWER9 module. Since this research focuses
on firmware-only solutions for booting subsequent firmware that is en-
crypted, this method is not feasible.

The second candidate method requires a storage chip to be incorpo-
rated in the POWER9 module to utilize the existing SEEPROM memory
space on the self boot engine. The SEEPROM memory stores the self
boot engine firmware along with verification keys for the secure boot.
During secure boot, the verification keys are compared against the hash
of the boot image. The key management system can be used to securely
transport keys to the image decryption firmware as needed. However, a
significant constraint is that the SEEPROM memory space is extremely
limited in size. In fact, the verification keys have to be stored as hashes
because they require too much space as raw keys. This limitation im-
posed by the SEEPROM memory renders this method infeasible.

The third candidate method for secure key management can be im-
plemented in firmware by utilizing the structure of the initial program
load with boot firmware encryption. The Hostboot bootloader stage is
integral to this method because it is stored and runs in the self boot
engine. This means that it is isolated in the POWER9 module and se-
cured from unauthorized entities. In the next stage, the Hostboot base
image travels from the PNOR, across the Low Pin Count bus and into
the POWER9 L3 cache. The Hostboot base section can be secured by
implementing a decryption function in the bootloader and encrypting
the base section.

This option also enables a user to update the decryption firmware
via the PNOR image because the boot image contains a Hostboot boot-
loader section, meaning that the bootloader firmware can be altered and
recompiled into a new PNOR image. After the first successful Talos II
system boot with an altered Hostboot bootloader image, the self boot
engine detects a firmware difference and proceeds to update the SEEP-
ROM side 0 memory at the start of the initial program load. The system
then reboots from side 0 and, if the system successfully reaches ISTEP
10.5, the SEEPROM side 1 is also updated. This is important because
it enables a user to safely update the self boot engine and Hostboot
bootloader firmware.

After the final bootloader firmware is updated on both sides of the
SEEPROM, several steps are executed to ensure that the bootloader
remains protected. The first step is to ensure that SEEPROM-side up-
dates are disabled. This is required because the self boot engine au-
tomatically updates the Hostboot bootloader firmware if it detects a
firmware change. The step enables Hostboot bootloader encryption or
the removal of the Hostboot bootloader section in the PNOR image
while still using the unencrypted Hostboot bootloader stored in the self
boot engine, thereby protecting the key stored in the firmware.

To implement this method, a PNOR image containing the decryption
code in the Hostboot bootloader with an unencrypted Hostboot base has
to be flashed to the system. This is required because, on the initial boot,
the self boot engine does not have the decryption code in the Hostboot
bootloader to decrypt the Hostboot base. After reaching ISTEP 10.5,
the system flashes the new Hostboot bootloader to the self boot engine.
This endows the system with the ability to boot with an encrypted
Hostboot base section in subsequent system start-ups.

To eliminate secure key management, the method is executed in re-
verse. A PNOR image with the Hostboot bootloader firmware without
the decryption code and encrypted Hostboot base firmware must be
compiled and flashed. After the SEEPROM side update, the self boot
engine has the Hostboot bootloader firmware without the decryption
code, meaning it can boot an unencrypted Hostboot base. SEEPROM
side 2 is updated in ISTEP 10.5, enabling the system to boot with an un-
encrypted Hostboot base. A PNOR image without decryption firmware and an unencrypted Hostboot base is then flashed to enable normal PNOR updates.

## 3. Related Work

Before implementing the ciphers for boot firmware encryption, background research was conducted to understand previous work in this field. Three examples of successful boot firmware encryption were used as a foundation in this research. Although the implementations were intended for secure firmware updates, the structures for decrypting encrypted firmware updates are similar to the structure for boot firmware encryption.

### 3.1 Secure Firmware Updates Using AES

The process that encrypts sections of a bootloader before being loaded on a system is important in this research. A guide by Silicon Labs [13] introduced the idea of encrypting boot firmware and decrypting it in secure storage space. The goal was to provide a secure distribution system for firmware updates to microcontrollers. This is required because application-sensitive information stored in firmware updates could be accessed by malicious actors to develop zero-day vulnerabilities. The Silicon Labs implementation enables a microcontroller system to receive encrypted firmware and decrypt it after it is stored in its internal flash chip. This structure is applicable to boot firmware encryption in a Taos II system by applying the encryption and decryption portions to the initial program load flow.

Microchip Technology [8] has implemented an encrypted bootloader that functions similar to the AES-encrypted bootloader from Silicon Labs [13]. A vulnerable firmware update process for embedded systems deployed in the field would enable malicious actors to compromise system security and possibly gain complete control of the deployed systems. The purpose of the encrypted bootloader was to safely update ultra-low-power microprocessor firmware using encryption.

### 3.2 Image Encryption

Digi International [2] developed a method for encrypting signed firmware to obscure image data from unauthorized users. It was applied to U-Boot, an open-source bootloader used in embedded devices, and compiles the instructions necessary to boot embedded system operating system kernels, just like PNOR images in OpenPOWER systems. The processes for encrypting and decrypting firmware during the boot se-
quence are identical to those employed in this research. In fact, the
implementation of encrypted U-boot firmware can be applied in specific
stages to a PNOR image.

4. Experimental Setup

Boot firmware encryption is applicable to a wide range of systems, each with varying performance impacts due to the specific implementations. An experiment was designed to analyze various types of encryption to assess the impacts on performance. It was important that the experiment would provide adequate data to understand the boot firmware encryption options and their security-performance trade-offs.

Consistent test runs are required to gather reliable boot time data because artifacts from previous boot images could affect the boot times of future trials. To address this problem, the PNOR images were randomized to minimize the influence of noise and other anomalies. During a new PNOR image boot sequence, an error correction check was executed that updated the boot image. This was done during the first few boots of a new image so that each PNOR image had to be booted multiple times until it stabilized. The final images were downloaded after they booted without requiring a reboot during the initial program load.

4.1 Microcontroller Setup

Figure 3 shows the microcontroller setup with the Talos II system for collecting experimental data in an efficient manner. Instead of booting from the Talos-supplied ASPEED 2400 baseboard management controller, an ARM Cortex microcontroller was employed to accomplish the task. A field-programmable gate array was also used as the Low Pin Count bridge between the microcontroller and POWER9 module. This setup enabled the modification of how PNOR images were transferred to the Talos II system using custom OpenBMC firmware. Note that the Serial Peripheral Interface (SPI), External Peripheral Interface (EPI), Flexible Service Interface (FSI) and Inter-Integrated Circuit (I2C) are specialized buses supported by the POWER9 module and used as part of the boot process.

The PNOR images were supplied to the Talos II system through Ethernet via a User Datagram Protocol (UDP) server. The system was set up such that the microcontroller serving as the baseboard management controller would wait for a PNOR image over Ethernet during a boot. A laptop with all the PNOR images would select one image to send via UDP, saving three minutes per boot over the File Transfer Protocol (FTP). The ability to boot without flashing the PNOR chip also saved
<figure><img src="image_4.png" /><figcaption>Figure 3. Microcontroller setup.</figcaption></figure>

five minutes per boot. Although the PNOR images were supplied to the Talos II system over Ethernet, the performance of boot time decryption, which is dependent on the POWER9 module, was not impacted.

The microcontroller also executed a UDP logger that was set up to collect data from the boot process. All the boot and error logs were transferred to a laptop for straightforward data collection on repeated runs. The logger recorded the Low Pin Count data transfers, PNOR access messages, warning logs and POWER9 messages. The data of interest was stored in the POWER9 message log file.

# 5. Experimental Methodology

This section provides key experimental details, such as the experimental factors and data collection approach, to replicate the experiment and analyze the data. It also describes the application scenarios and assumptions.

## 5.1 Experimental Factors

Several factors must be considered when designing an experiment to capture boot image encryption performance. The effort focused on two experimental factors that directly answer the research questions. The main research objective was to understand the performance impacts of various types of encryption on boot firmware encryption. Therefore, the first experimental factor was the encryption type, which covered no encryption and the XOR, SPECK and AES ciphers. The second experimental factor was the size of the encrypted section, which is referred to as the encryption ratio. The Skiboot firmware was originally encrypted entirely, but was subsequently altered to measure the encryption overhead
per byte. In addition to collecting metrics for 100%-encrypted Skiboot, two more boot images were created to measure the 50%-encrypted and 25%-encrypted boot performance.

A full factorial design was employed when narrowing down the experimental factors. Since the experiment involved two factors, with each factor having three to four levels, an analysis would require at least 16 trials. However, it was decided to run 100 trials per factor level, resulting in 1,000 total runs. This helped determine the main effects and interactions on the boot time metrics. The full factor analysis was expected to reveal the optimal encryption type and encryption ratio for security and boot performance.

## 5.2 Data Collection

Data collection was accomplished using a Python script that ran boot encryption trials of the 10 PNOR images. The list of encrypted PNOR images was randomized, resulting in 1,000 total trials with 100 trials per PNOR image. The Python script looped through the randomized list of PNOR images and coordinated the boot over Ethernet and the UDP logger. The code also checked for a string that prints at the end of the boot sequence that signaled a complete boot, and proceeded to restart the data collection script for the next trial. The data collection script helped provide consistent boot time metrics for the complete analysis of boot firmware encryption.

The POWER9 module log data extracted for each trial enabled the characterization of each boot image based on its boot time performance. A Python script was created to process the log file and save the boot time data to a comma-separated value (CSV) file. This data represents the PNOR image file through boot time metrics. The log file contains the start and end timestamps for Hostboot, loading Skiboot, transferring data to the buffer, decrypting and decompressing the encrypted sector. These timestamps were used to compute the times taken for the various stages employed in the security-performance analysis.

## 5.3 Application Scenarios

Boot firmware encryption can be applied to devices used in a number of critical infrastructure sectors, including electric power and water distribution. In the power grid, phasor measurement units record phasor data at various points in the grid [10]. The collected data is aggregated at phasor data collectors. The data collectors may be vulnerable to hardware attacks that enable malicious actors to gain access to systems in the infrastructure. Manipulating the sensor systems could cause power
grid failures and damage power plant systems. The water distribution system faces similar threats due to pressure and water flow sensors positioned in vulnerable end-node environments. The data collectors for sensors used in electric power and water distribution could be protected via boot firmware encryption, preventing hardware attacks.

## 5.4 Assumptions

The experiment assumed that the system already supports secure key management. This assumption is necessary in a finalized boot image encryption system because the decryption key is currently stored in unencrypted code. In fact, it would defeat the purpose of boot image encryption because the attacks described above would enable malicious actors to observe details of initial program loads. The development of a secure key management solution is a topic for future research.

Another assumption is that a malicious actor is unable to remove the CPU from the motherboard and examine the L3 cache contents. Since the encrypted firmware is decrypted in the processor, the plaintext would be observable in the cache. It would be extremely difficult to pause the boot process at the right time and remove volatile memory from the POWER9 module, but it could be possible for a malicious actor with substantial hardware expertise.

# 6. Experimental Results and Analysis

After conducting the 1,000 boot data collection trials, a Python script was used to process the experimental data. The timestamps corresponding to the boot stages important to the research were extracted using regular expressions and inserted into data frames. The data was exported to a CSV file that was processed using RStudio.

## 6.1 Boot Image Encryption Performance Impact

The first research objective was to observe performance differences when implementing boot image encryption. Figure 4 shows the times required to complete the Hostboot phase of the initial program load. This was measured from ISTEP3 until the start of Skiboot, which helps place the performance impacts of the various boot image encryption methods in perspective. The median Hostboot time for AES was 25.51 s whereas the time for a normal boot was 24.69 s. This means that it took Hostboot 0.86 s longer to run by protecting the entire Skiboot firmware section with AES-256. The SPECK implementation had a median Hostboot execution time of 24.99 s whereas XOR had a median time of 25.06 s.
<figure><img src="image_5.png" /><figcaption>Figure 4. Hostboot execution times.</figcaption></figure>

Figure 5, which shows the times for loading Skiboot in ISTEP 20.1 for the encryption methods, clarifies the differences in performance overhead. The differences in Hostboot execution times are caused by the execution of the load Skiboot step, which contains the decryption function.

The median times were 1.0443 s for AES, 0.5923 s for SPECK, 0.5193 s for XOR and 0.2796 s for unencrypted firmware. The variation in Skiboot loading times are much smaller than the Hostboot execution times in Figure 5, which may be due to the earlier ISTEPs in Hostboot introducing variability.

One-way analysis of variance (ANOVA) tests were performed on the boot time data to see if significant differences exist between the encryption methods. The first ANOVA test on the Hostboot execution times revealed that the boot times had significant differences between the different factors. The second ANOVA test on the execution times for loading Skiboot also revealed significant differences between the encryption methods. These two tests answer the research question in that performance differences do exist between each encryption method and the unencrypted Skiboot implementation.

Figure 6 shows an interesting pattern related to image encryption implementations. In the case of a PNOR image without Skiboot en-
<figure><img src="image_6.png" /><figcaption>Figure 5. Skiboot loading times.</figcaption></figure>

cryption, the decompression code must wait for the compressed Skiboot
section to be transferred into working memory, which took 0.2654 s on
average. In the case of the decryption code for the encrypted Skiboot
firmware, memory was allocated for the decryption to execute before
decompression. The average decompression time for encrypted firmware
was 0.1281 s, 0.1373 s faster than the stock boot image. This helps offset
the decryption time by frontloading the effort to transfer PNOR image
data.

**6.2 Comparison of Encryption Methods**

The implementations of the three ciphers demonstrate the range of
encryption overhead that a POWER9 module can handle during initial
program load. Figure 7 shows the Skiboot decryption times for the
encryption methods. The XOR cipher takes an average of 0.0103 s, which
demonstrates its efficiency. The XOR implementation would be efficient
in systems that require minimal boot firmware execution delays while
providing several megabytes of free memory to store the one-time pad
key. The SPECK cipher also ran efficiently with decryption requiring
0.0875 s. Although the SPECK cipher is not as secure as the AES cipher,
it provides a layer of protection while decrypting Skiboot around six
<figure><img src="image_7.png" /><figcaption>Figure 6. Skiboot decompression times.</figcaption></figure>

<figure><img src="image_8.png" /><figcaption>Figure 7. Skiboot decryption times.</figcaption></figure>
<figure><img src="image_9.png" /><figcaption>Figure 8. Skiboot decryption times for different encryption methods and ratios.</figcaption></figure>

times faster. As expected, the AES cipher, which is the most secure encryption implementation, had the largest decryption time of 0.5397 s.

## 6.3 Encryption Ratio

The encryption ratio expresses the amount of Skiboot that is subject to encryption. Figure 8 shows the Skiboot decryption times for different encryption methods and ratios. The XOR cipher performance for each encryption ratio shows that the percentage of Skiboot that is encrypted does not affect the overall performance by a significant amount. The increase from 25% to 100% only increased the Skiboot decryption time by 0.00471 s. In contrast, the SPECK and AES implementations show linear increases in decryption time as the encryption ratio increases. AES takes longer to decrypt per byte compared with SPECK. Specifically, AES takes 0.00543 s for each percentage of Skiboot encrypted whereas SPECK takes 0.00089 seconds. A two-way ANOVA test revealed that significant differences exist for the encryption methods and encryption ratios.
# 7. Conclusions

This research demonstrates that the Talos II system is capable of decrypting firmware during initial program loads. The firmware comprises the decryption function and is encrypted before being flashed on the system. The experimental results reveal that AES encryption of a portion of the boot image is cost-efficient when considering the increase in hardware security. SPECK encryption would work effectively in an implementation where one second of boot time is significant. Since a system only needs to boot once in most implementations, a 0.86 s delay when using AES-256 to the boot firmware is acceptable.

A useful avenue for future research is to expand boot image encryption to Intel and AMD systems. Since both systems use BIOS or Unified Extensible Firmware Interface bootloaders, their initial program structures should be similar. With access to detailed documentation and source code of the boot firmware, implementing the decryption code should be straightforward. Accomplishing this goal would enable boot image encryption to be applied to consumer workstations.

The views expressed in this chapter are those of the authors, and do not reflect the official policy or position of the U.S. Air Force, U.S. Department of Defense or U.S. Government. This document has been approved for public release; distribution unlimited (Case #88ABW-2021-1005).

# References

[1] R. Beaulieu, D. Shors, J. Smith, S. Treatman-Clark, B. Weeks and L. Wingers, SIMON and SPECK: Block ciphers for the Internet of Things, presented at the *NIST Lightweight Cryptography Workshop*, 2015.

[2] Digi International, Image Encryption, Hopkins, Minnesota (www.digi.com/resources/documentation/digidocs/embedded/dey/2.6/cc8x/yocto-trustfence_t_image-encryption.html), 2019.

[3] A. Fournaris, Using Hardware Means to Secure Critical Infrastructure Devices, CIPSEC Project, University of Patras, Patras, Greece (www.cipsec.eu/content/using-hardware-means-secure-critical-infrastructure-devices), 2020.

[4] D. Heller and N. Sastry, OpenPower secure and trusted boot, Part 2: Protecting system firmware with OpenPOWER secure boot, *IBM Developer*, IBM, Armonk, New York (developer.ibm.com/articles/protect-system-firmware-openpower), 2019.
[5] IBM, Secure Initial Program Load (IPL) Process, Armonk, New York (www.ibm.com/docs/en/power9/9009-42A?topic=9009-42A /p9ia9/p9ia9_secure_ipl_proc_concept.htm), 2021.

[6] A. Jeffery, General Architecture of Hostboot (amboar.github.io/ notes/2018/08/19/hostboot-architecture.html), 2018.

[7] A. Jeffery, Hacking Hostboot (amboar.github.io/notes/2018/ 08/19/hacking-hostboot.html), 2018.

[8] Microchip Technology, AVR231: AES Bootloader, Application Note AN2462, DS00002462A, Chandler, Arizona (ww1.microchip.com/ downloads/en/AppNotes/00002462A.pdf), 2017.

[9] C. Muramoto, S. Dunlap and S. Graham, Improving hardware se- curity on the Talos II architecture through boot image encryption, Proceedings of the Seventeenth International Conference on Cyber Warfare and Security, pp. 489–496, 2022.

[10] R. Nuqui and A. Phadke, Phasor measurement unit placement tech- niques for complete and incomplete observability, IEEE Transac- tions on Power Delivery, vol. 20(4), pp. 2381–2388, 2005.

[11] Raptor Computing Systems, Default PNOR Layout, Belvidere, Illinois (git.raptorcs.com/git/pnor/tree/p9Layouts/default PnorLayout_64.xml), 2019.

[12] Raptor Computing Systems, OpenPOWER Firmware, Belvidere, Illinois (wiki.raptorcs.com/wiki/OpenPOWER_Firmware), 2021.

[13] Silicon Labs, AN0060: Boolloader with AES Encryption, Revi- sion 1.05, Austin, Texas (www.silabs.com/documents/public/ application-notes/an0060-bootloader-with-aes-encryption .pdf), 2016.