# Comparing Application-Level Hardening Techniques for Neural Networks on GPUs
## 1 Introduction  
Artificial intelligence (AI) is increasingly utilized in various fields, particularly in consumer electronics and safety-critical systems, often employing neural networks (NNs). In autonomous vehicles, NNs process large amounts of sensor data in real time for decision-making and action. Hence, the reliability of NNs is vital for system performance. Edge AI allows for the implementation of NN algorithms in Internet of Things (IoT) applications and devices with limited resources [1, 2, 3]. Techniques like split computing [4] help integrate complex neural networks into systems using _commercial off-the-shelf (COTS)_ devices, such as drones. Recent advancements in semiconductor technology have also made it possible to include AI accelerators, like GPUs, in embedded IoT devices [5]. Unfortunately, in small technology scaling (7 nm and below), semiconductor devices are more susceptible to permanent hardware faults caused by different phenomena (e.g., premature degradation, aging, or manufacturing defects) [6, 7, 8]. In fact, such faultscan silently propagate as errors through the NN execution, jeopardizing the application reliability (e.g., faults inducing wrong predictions that cause drone accidents) [9; 10; 11; 12]. Fault effects can be reduced in AI hardware design by adding specialized structures for detection and mitigation. Nonetheless, these solutions may increase costs and power consumption and may not be available in COTS devices. Alternatively, software-based hardening techniques (HTs) seek to mitigate the effect of hardware faults in NNs by adopting NN architectural-level modification, including sometimes additional NN retraining. Although such NN architectural-level hardening solutions are flexible and cheaper than hardware-based solutions, there are still two main concerns about these solutions, (i) the assessment of NN hardening solutions may lead to optimistic results when using hardware-agnostic fault injections (FIs) [13; 14; 15; 16; 17], and (ii) to the best of our knowledge, every NN hardening solution is evaluated under different conditions (i.e., NN models, and error models), making it hard to determine which solution fits better in a different context. In the first case, most of the reported works in the literature assess the effectiveness of NN hardening strategies against transient faults by resorting to random error corruptions claiming to describe hardware fault effects in different hardware elements [13; 14; 15; 16; 17]. Nonetheless, none of such works consider the interaction of such faulty hardware with the application execution (e.g., tiling matrix multiplication in GPUs). For example, the evaluations in [13] focus on data path errors (due to transient faults) with the corruption of weights and neurons according to an error rate. However, the error rates are arbitrarily chosen within the range \([10^{-6},10^{-1}]\) and with random patterns that do not consider how the accelerator propagates the fault as error across the layer computation. In ref. [18], the authors resorted to a bit-masks approach (describing the deviations of weights and neurons due to the energy spent in memory access) proposed in [19] as error modeling mimicking the effect of the fault occurring in the SRAM cells with error rates that range in \([10^{-3},10^{-1}]\). Moreover, the authors of [14; 15] injected multiple random bit-flips in neurons' output to mimic the effect of transient faults due to a particle that strikes a flip-flop during MAC operations or a memory unit. These evaluation approaches often overlook propagation errors across NN layers, particularly where convolutional or linear layers are implemented, as in tiling matrix multiplications on embedded GPUs. A faulty GPU core affects only certain matrix multiplication tiles, leading to unique error effects at the NN layer that may influence the effectiveness of hardening techniques differently than standard evaluation methods do [20; 21].