# Soft Error Assessment of Attitude Estimation Algorithms Running in a Resource-constrained Device under Neutron Radiation

Jonas Gava, Tarso Kraemer Sarzi Sartori, Alex Hanneman, Rafael Garibotti, Ney Calazans, Hassen Fourati, Rodrigo Possamai Bastos, Ricardo Reis, Luciano Ost

► To cite this version:

Jonas Gava, Tarso Kraemer Sarzi Sartori, Alex Hanneman, Rafael Garibotti, Ney Calazans, et al. Soft Error Assessment of Attitude Estimation Algorithms Running in a Resource-constrained Device under Neutron Radiation. RADECS 2023 - International Conference on Radiation and its Effects on Components and Systems, Sep 2023, Toulouse, France. pp.1-4. hal-04342734

HAL Id: hal-04342734

https://hal.science/hal-04342734v1

Submitted on 13 Dec 2023

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons CC BY 4.0 - Attribution - International License
# Soft Error Assessment of Attitude Estimation
# Algorithms Running in a Resource-constrained
# Device under Neutron Radiation

J. Gava, T. Sartori, A. Hanneman, R. Garibotti, N. Calazans, H. Fourati, R. Possamai Bastos, R. Reis and L. Ost

**Abstract**—This paper assesses the soft error reliability of attitude estimation algorithms running on a resource-constrained microprocessor under neutron radiation. Results suggest that the EKF algorithm has the best trade-off between reliability and runtime overhead.

**Index Terms**—Neutron Radiation, attitude estimation algorithms, Edge Devices.

## I. INTRODUCTION

Unmanned Aerial Vehicles (UAVs), also known as drones, are an inherent part of prominent roles in civilian and military aviation domains. UAVs are employed to accomplish different tasks ranging from monitoring the reef for signs of degradation [1] to enhancing public security and safety [2]. The UAVs can operate with distinct levels of autonomy [3] and rely on different types of navigation systems, which may use GPS/GNSS receivers, accelerometers, ultrasonic sensors, depending on the purpose they are designed for. For instance, an inertial navigation system (INS) uses accelerometers and gyroscopes to measure the acceleration and rotation of the UAV, which are further used to determine its position, velocity, and attitude. The error in the attitude estimation can cause unstable UAV navigation (e.g., pitch or roll oscillations), which may require more power to maintain its desired flight path [4].

Although most UAVs operate at low-level airspace, their components (e.g., INS sensors) are exposed to radiation-induced soft errors such as single event effects (SEEs) [5], and tackling their occurrence in underlying devices is a mandatory and substantial challenge. The occurrence of SEEs in INS sensors or during the execution of an attitude estimation algorithm can, ultimately, lead to dangerous situations since underlying systems share airspace with civil air traffic [4].

This work was partially funded by: MultiRad (PAI project funded by Région Auvergne-Rhône-Alpes); UK EPSRC (EP/R513088/1); IRT Nanoelec (ANR-10-AIRT-05 project funded by French PIA); UGA/LPSC/GENESIS platform; CAPES; CNPq (grant no. 317087/2021-5 and 407477/2022-5); FAPERGS. J. Gava and R. Reis are with the Instituto de Informatica, PGMicro, Universidade Federal do Rio Grande do Sul, Porto Alegre, Brazil.

R. Garibotti and N. Calazans, are with the School of Technology, Pontifical Catholic University of Rio Grande do Sul, Porto Alegre, Brazil.

T. Sartori and R. Possamai Bastos are with the Univ. Grenoble Alpes, CNRS, Grenoble INP, TIMA, 38000 Grenoble, France.

T. Sartori and H. Fourati are with the Univ. Grenoble Alpes, CNRS, Grenoble INP, GIPSA-Lab, 38000 Grenoble, France.

A. Hanneman and L. Ost are with the Wolfson School, Loughborough University, Loughborough, U.K.

Previously in [6], authors assessed the impact of neutron-induced soft error in the execution of the novel quaternion Kalman filter (NQKF) algorithm when running in a general-purpose multi-core Arm processor. Differently, this paper contributes by assessing the soft error reliability of three attitude estimation algorithms, which differ in terms of performance efficiency, running on an Arm Cortex-M4 microprocessor under neutron radiation. Gathered results suggest that the extended Kalman filter (EKF) algorithm provide the best MWTF$_{critical}$ result, which is about 3× more than the indirect Kalman filter (IKF) and 1.5× more w.r.t. the NQKF.

## II. ADOPTED ATTITUDE ESTIMATION ALGORITHMS

The purpose of an attitude estimation (AE) algorithm is to determine the position or alignment of an object in relation to a known point of reference, utilising measurements obtained from sensors. A simplified model for these sensors is depicted in Equation (1).

$$ egin{cases} w = w_0 + b_w + v_w \\ a = C_N^B[q].g + v_a \\ m = C_N^B[q].h + v_m \end{cases} \quad (1) $$

Equation (1) describes the measurements obtained from the gyroscope, accelerometer, and magnetometer, respectively. Theoretically, if the starting position is known, the gyroscope data alone would be sufficient to determine the attitude. However, the estimation error gradually increases over time due to factors such as gyroscope bias ($b_w$) added to the actual angular velocity ($w_0$) and inaccuracies introduced during integration [7], thus, additional sensors are utilised to mitigate these errors.

According to [6], the rotation matrix $C_N^B[q]$ in Equation (1) is a nonlinear function of the attitude quaternion. This matrix is capable of converting a vector defined in the navigation frame (N), such as gravity, into the body frame (B). It is important to consider each sensor's inherent measurement noise, represented by $v_w$, $v_a$, and $v_m$ in Equation (1). Various approaches have been proposed to filter out such noise and merge sensor data to estimate the attitude. One commonly used technique is the Kalman filter (KF), which is an optimal recursive algorithm used for state estimation of linear systems. The KF algorithm operates under the assumption that the
sensor noise follows a Gaussian normal distribution with a mean of zero and a specific standard deviation based on the sensor in question.

## A. Algorithms Description

The four AE algorithms assessed in this work are:

* EKF: the EKF algorithm is widely used for real-time spacecraft attitude estimation [8], [9]. However, the algorithm is designed for linear systems, which requires linearisation of the measurement equations (such as those for the accelerometer and magnetometer shown in Equation (1)) to enable appropriate utilisation in the estimation of the attitude quaternion.

* IKF: in [7], Suh Soo proposed an adaptive KF approach to offset external accelerations (other than gravity). Rather than estimating the quaternions directly, the algorithm estimates the attitude quaternion error, which depends on the gyroscope bias and noise. The error estimation is then converted into quaternions.

* NQKF: the process of linearization required for the EKF algorithm may lead to undesirable outcomes, including sensitivity to initial conditions and a rise in computational workload. Choukroun et al. [10] presented a novel algorithm to address these issues. This approach involves a pseudo-measurement linear equation that can be used along with the KF, removing the need for linearization and reducing sensitivity to initial AE errors.

The KF based algorithms possess some parameters that need to be adjusted according to the respective sensors' noises. It is necessary to set three covariance matrices, one for each sensor being used (gyroscope, accelerometer, and magnetometer), based on the standard deviation of the sensors' measurement noise.

## B. Input and Output datasets

To test the soft error reliability of processing AE algorithms under radiation, we generated an input dataset consisting of 333 input vectors. Each input vector comprises nine components that represent the sensors' measurements stacked for a specific moment in time. These sensors include an accelerometer, magnetometer, and gyroscope, each providing a three-dimensional physical measurement of acceleration, magnetic field, and angular velocity, respectively. To generate the output data, the AE algorithm under evaluation process the input dataset and estimate the quaternion $qk/k$ or simply $q$ using four components. As the input dataset contains 333 input vectors, the output dataset also contains 333 vectors.

# III. RADIATION TEST METHODOLOGY

This Section describes the methodology used to collect and present the results obtained with a 14MeV neutrons generator, which was used to assess the soft error resilience of attitude estimation algorithms running on a resource-constrained device under neutron radiation.

## A. Radiation Test Flow

Figure 1 shows the test flow schematic. The Universal Asynchronous Receiver Transmitter (UART)-based communication between the device under test (DUT) and the control computer (CC) has been verified via checkers (i.e., checksum) to isolate radiation-induced failures in the UART peripheral and the data communication channel between the DUT and the CC.

<figure><img src="image_2.png" /><figcaption>Fig. 1: Test set-up used in the radiation testing campaign.</figcaption></figure>

The steps to run the test are shown below:

1. Board programming using the Open On-Chip Debugger (openocd).

2. Synchronise both DUT and CC devices before the algorithm main function execution, i.e., send a message from CC to DUT and awaits the correct response. If the DUT takes more than 5 seconds to respond, the board is then reprogrammed;

3. Check the input checksum. The checksum is calculated in the DUT and sent to the CC for checking against the golden reference. If there is a mismatch in the input data, the board is reprogrammed. If the reprogramming fails, the relay is actioned, and a power cycle is done;

4. Algorithm main function execution;

5. Synchronise DUT and CC after the algorithm computation, i.e., send a message from CC to DUT and awaits the correct response;

6. Send output and output checksum from DUT to CC. The checksum is calculated in DUT and sent to CC. If there is a mismatch in the output, the procedure is the same as for the input data;

## B. Radiation Test Set-Up

The 14-MeV neutron radiation test campaign was performed at the Laboratory of Subatomic Physics & Cosmology (LPSC, Grenoble, France) in February 2023 using the neutron generator GENEPI2 (GGenerator of NEutrons Pulsed Intense). It generates a 14-MeV neutron beam with a maximum flux greater by a factor of $10^{10}$ than the 14-MeV neutron flux at 40,000 ft. A total fluence $> 5.85 \times 10^{11}$ neutron/cm$^2$ was chosen to achieve statistical significance. The average flux during the experiment was $1 \times 10^7$ neutron/cm$^2$/s.

The STM32 NUCLEO-L476RG board [11] was selected as the target device. The AE algorithms were compiled using Clang/LLVM 6.0.1 with O3 optimisation level. The DUTs
were placed in the first boards row of the neutron generator at a distance of 50mm. The flux was calibrated remotely to fit a proper operation of the DUTs, which is connected to a CC outside the radioactive chamber through a USB cable. The whole system (CPU, memory, communication peripherals) was under the beam. Figure 3 illustrates the set-up assembled at the LPSC.

<figure><img src="image_3.png" /><figcaption>Fig. 3: DUTs mounted at LPSC facility in February 2023.</figcaption></figure>

### C. Adopted Fault Classification and Reliability Metrics

Radiation results are classified as follows: Silent Data Corruption (SDC): the algorithm execution is done normally but the output is incorrect; critical fault: is a SDC sub-classification taking into account the Euler angles' mean absolute error (MAE) between the radiation testing data and the golden reference data. We chose 0.5 degree as the Euler angle's MAE threshold (see Figure 2); crash: the algorithm suffers from abnormal termination or hang. The communication between CC and DUT is lost during the algorithm execution, indicating that radiation effects have upset the DUT. In this situation, the board must be restarted.

The following metrics are used to compare the reliability of the AE algorithms. The Failure in Time (FIT) metric shows

how many failures occur in a billion hours. It depends on both the device sensitivity and the particle flux to which it will be exposed. Lower FIT is better. The JEDEC JESD89 standard [5] suggests to uses $13n/cm^2/h$ as flux at sea level. The cross section of the device for a singular test was calculated using the Equation (2), where $\sigma$ is the per-bit cross section, $N_e$ is the number of errors, and $\phi$ is the neutron fluence.

$$ \sigma = \frac{N_e}{\phi} \qquad (2) $$

This work also uses the Mean Work to Failure (MWTF) metric, which measures the tradeoff between reliability improvement and runtime overhead. Equation (3) shows how it is calculated.

$$ MWTF = (\sigma \times \text{flux} \times \text{execution time})^{-1} \qquad (3) $$

## IV. RADIATION RESULTS

Figure 4 shows the radiation experiment results considering three attitude estimation algorithms, as detailed in Section III. Each bar represents an event type (SDC, critical, crash) associated with the left y-axis. The red dots represent the sea-level FIT metric for each event, associated with the right y-axis. During the radiation experiment, a total of 241 events were observed. The number of SDCs is similar across all three algorithms. However, EKF and NQKF exhibited around 60-70% fewer critical faults than IKF, whereas IKF presented three times more crashes than the other algorithms. This is mainly due to the longer radiation exposure (see column 3 in Table I).

By utilising the radiation exposure time, the FIT rate can determine which algorithm is the most reliable when executed in a resource-constrained device. Note that a lower FIT rate indicates higher reliability. Despite having a similar number of SDCs, the $FIT_{SDC}$ for IKF is almost 2× better than the other algorithms. However, the $FIT_{crit}$ for EKF is 32% lower than NQKF and 70% lower than IKF. This means that, for the same radiation exposure time, IKF presents fewer SDC events, but a greater proportion of these errors become critical. Furthermore, EKF and NQKF have similar $FIT_{crash}$, which is about 35% lower than the IKF algorithm.

<figure><img src="image_4.png" /><figcaption>Fig. 2: Sub-figures (a,b,c) show examples of critical faults observed during the radiation experiment for each algorithm.</figcaption></figure>
<table><caption>TABLE I: Summary of the neutron radiation experiment results.</caption><thead><tr><th rowspan="2">Case-Study Scenarios</th><th rowspan="2">Runtime [s]</th><th rowspan="2">Fluence [10<sup>11</sup> neutrons/cm<sup>2</sup>]</th><th colspan="3">Events (FIT [Failures/10<sup>9</sup> h])</th><th colspan="2">*MWTF (σ [10<sup>-11</sup> cm<sup>2</sup>])</th><th colspan="2">Memory Usage [kB]</th></tr><tr><th>SDC</th><th>Critical</th><th>Crash</th><th>Critical</th><th>Crash</th><th>RAM</th><th>Flash</th></tr></thead><tbody><tr><td>EKF</td><td>163</td><td>1.52</td><td>61 (5.22)</td><td>4 (0.34)</td><td>8 (0.68)</td><td>3.08 (2.64)</td><td>2.85 (5.28)</td><td>4.3</td><td>174</td></tr><tr><td>IKF</td><td>296</td><td>2.90</td><td>59 (2.64)</td><td>13 (0.58)</td><td>24 (1.07)</td><td>1.00 (4.48)</td><td>1.00 (8.27)</td><td>4.3</td><td>173</td></tr><tr><td>NQKF</td><td>183</td><td>1.43</td><td>54 (4.90)</td><td>5 (0.45)</td><td>8 (0.73)</td><td>2.07 (3.50)</td><td>2.38 (5.60)</td><td>4.3</td><td>199</td></tr></tbody></table>

* Normalised MWTF

<figure><img src="image_5.png" /><figcaption>Fig. 4: Radiation-induced failures in each case-study scenario.</figcaption></figure>

Table I presents a summary of the data collected from the radiation experiment. Despite a slight variation in the number of runs, the effective fluence ranges from $1.43 \times 10^{11}$ neutrons/cm$^2$ to $2.90 \times 10^{11}$ neutrons/cm$^2$. This difference is due to the varying execution time of each run, which ranges from 163s to 296s. While EKF and NQKF have similar execution times, IKF takes twice as long. Therefore, metrics that consider both reliability and execution time are crucial for a fair comparison. In this regard, Table I also shows the normalised MWTF metric values for critical and crash occurrences. Note that the higher the value, the better the algorithm. For critical events, NQKF and EKF show a $2 \times$ and $3 \times$ improvement, respectively, compared to IKF when scaled to the level of terrestrial radiation flux. For crash events, the MWTF improvement values range from $2.38 \times$ for NQKF to $2.85 \times$ for EKF. The cross-section is another widely used metric. For critical faults, EKF shows the lowest value of $2.64 \times 10^{-11}$ n/cm$^2$, while IKF has the highest value of $4.48 \times 10^{-11}$ n/cm$^2$. For crashes, the values range from $5.28 \times 10^{-11}$ n/cm$^2$ (EKF) to $8.27 \times 10^{-11}$ n/cm$^2$ (IKF). RAM and flash memory usage is similar for all algorithms, with the exception of NQKF's flash memory usage. However, radiation has a minimal impact on flash memory, and no errors were detected in input checks performed on flash memory during all runs. The findings indicate that the IKF algorithm is more resilient to SDCs caused by soft errors induced by neutron particles. However, IKF takes almost twice as long to run compared to the other algorithms. Relying solely on the FIT metric may result in incorrect conclusions. With that in mind, the MWTF metric was employed, revealing that EKF provides

the best trade-off between reliability improvement and runtime overhead against critical faults and crashes. Note that the algorithms were exposed to radiation for a long time in each run (i.e., 3-5 min), leading to the accumulation of multiple bit-flips until the end of each run. One of the factors contributing to the low performance is that the Arm Cortex-M4 processor does not support some floating-point operations, requiring the use of inefficient math libraries. Further investigations are ongoing to determine the potential benefits of adopting a board with an Arm Cortex-M7 processor that directly supports performing these operations in hardware.

## V. CONCLUSION

This paper assessed the soft error resilience of three attitude estimation algorithms running on a resource-constrained device under neutron radiation. Results suggest that EKF is the best algorithm in terms of $MWTF_{critical}$, i.e., about $3 \times$ more than IKF and $1.5 \times$ more than NQKF. For future work, a board with more floating-point resources will be evaluated using the same AE algorithms.

## REFERENCES

[1] J. Boyd, "Drones survey the great barrier reef: Aided by AI, hyperspectral cameras can distinguish bleached from unbleached coral - [News]," *IEEE Spectrum*, vol. 56, no. 7, pp. 7-9, 2019.

[2] R. Clarke and L. Bennett Moses, "The regulation of civilian drones' impacts on public safety," *Computer Law & Security Review*, vol. 30, no. 3, pp. 263-285, 2014.

[3] E. Sholes, "Evolution of a UAV Autonomy Classification Taxonomy," in *IEEE Aerospace Conference*, 2007, pp. 3201-3216.

[4] K. W. Eure, C. C. Quach, S. L. Vazquez, E. F. Hogge, and B. L. Hill, "An application of UAV attitude estimation using a low-cost inertial navigation system," *NASA, Tech. Rep.*, 2013.

[5] C. Slayman, "Jedec standards on measurement and reporting of alpha particle and terrestrial cosmic ray induced soft errors," in *Soft Errors in Modern Electronic Systems*, 2011, pp. 55-76.

[6] T. Kraemer Sarzi Sartori, H. Fourati, M. Letiche, and R. P. Bastos, "Assessment of Radiation Effects on Attitude Estimation Processing for Autonomous Things," *IEEE Transactions on Nuclear Science*, vol. 69, no. 7, pp. 1610-1617, 2022.

[7] Y. S. Suh, "Orientation Estimation Using a Quaternion-Based Indirect Kalman Filter With Adaptive Estimation of External Acceleration," *IEEE Transactions on Instrumentation and Measurement*, vol. 59, no. 12, pp. 3296-3305, 2010.

[8] J. L. Crassidis, F. L. Markley, and Y. Cheng, "Survey of Nonlinear Attitude Estimation Methods," *Journal of Guidance, Control, and Dynamics*, vol. 30, no. 1, pp. 12-28, 2007.

[9] A. Sabatini, "Quaternion-based extended Kalman filter for determining orientation by inertial and magnetic sensing," *IEEE Transactions on Biomedical Engineering*, vol. 53, no. 7, pp. 1346-1356, 2006.

[10] D. Choukroun, I. Bar-Itzhack, and Y. Oshman, "Novel quaternion Kalman filter," *IEEE Transactions on Aerospace and Electronic Systems*, vol. 42, no. 1, pp. 174-190, 2006.

[11] STMicroelectronics, "STM32 Nucleo L476RG," 2023. [Online]. Available: https://www.st.com/en/evaluation-tools/nucleo-l476rg.html