# Comparing Application-Level Hardening Techniques for Neural Networks on GPUs
## 1 Introduction
While instruction-level FIs can provide a more realistic reliability assessment of NNs regarding transient [22; 23; 24] or permanent faults [21; 25; 26], their application is limited to specific hardware devices, making it challenging to adopt such strategies on COTS embedded GPUs when evaluating hardening approaches. Furthermore, executing instruction-level FIs through an ad hoc framework [27; 28; 29] significantly increases simulation time. Alternatively, hardware-aware evaluations at the application level offer a good balance between accuracy and evaluation time for assessing NN reliability and hardening strategies. For instance, refs. [30; 31] use weight single bit-flip (WSBF) to evaluate permanent faults in hardware memory, resorting to the stuck-at fault model. Additionally, the authors of [21] explored the effect of faults in MAC units during NN layer execution through the neuron bit error rate (NBER) error model inspired by the results gathered in [20]. In particular their findings emphasize the necessity for error models that account for faulty matrix multiplication in Edge AI applications. On the other hand, it is important to note that when assessing HTs, most research activities use varying test benches (e.g., NN architecture, datasets, target components, and other HTs). Consequently, results from previous studies are not applicable for selecting a specific hardening technique for a given application. For example, the authors of [14; 15]evaluated the proposed HT on 7 NNs on five datasets, while the authors of [16] evaluated their HT on three NNs and one dataset different from the previous work. Hence, the achieved results cannot be generalized for the selection of a specific HT for different test bench configurations. Moreover, as the training process strongly depends on the NN architecture along with the selected datasets, selecting a different test bench implies a different and non generalizable training setup when the HT requires a training step. This specificity hinders the generalizability of the HT implementation descriptions provided in the respective studies. In addition, to the best of our knowledge, there is no standard procedure for establishing a comprehensive ranking of HTs for edge computing scenarios. Therefore, it is still unclear (i) whether the application-level hardening techniques are effective in terms of reliability enhancement when they are evaluated with hardware-aware error models and when they are implemented on any neural network and (ii) which HT shows the best trade-off between the cost for its implementation on the normal functioning of the NNs and the capabilities of the HT in enhancing NN reliability. Based on the studied literature, we expect that not all the application-level HTs enhance the reliability of the NNs when they are subject to both WSBF and NBER error models considering different NN architectures and datasets. Furthermore, performing an impact analysis on the implementation of the HTs can yield valuable insights regarding their feasibility of implementation on COTS constrained devices with limited computational capabilities employed in Edge AI domains, prior to evaluating their effectiveness in terms of reliability enhancement. To prove our hypotheses, this work proposes a systematic evaluation methodology that combines two main aspects of the NN HTs: (i) the features of the HT when incorporated in a given NN and (ii) the fault mitigation capabilities. In the first aspect, we proposed a combination of qualitative (e.g., effort of implementation) and quantitative metrics (e.g., inference time overhead) to evaluate the cost for the HT implementation. In the second aspect, we used hardware-aware error models WSBF and NBER to assess the effectiveness of a HT to mitigate hardware faults in terms of accuracy degradation. The proposed evaluation methodology enables a direct comparison of the HTs in terms of the feasibility implementation and effectiveness as a fault countermeasure.