# Comparing Application-Level Hardening Techniques for Neural Networks on GPUs
## 7 Conclusions  
Given the lack of a standardized framework for evaluating the application-level HTs proposed in the literature, our study proposes an evaluation strategy that allows us to present the first comprehensive and independent assessment based on (i) the impact that the HT implementation has on the original NN architecture and (ii) the HTs effectiveness when subject to hardware-aware error models.  
In our study, we conducted hardware-aware fault injection campaigns to systematically evaluate five prominent application-level HTs: Adaptive Clipper, Ranger, Swap ReLU6, median filter, and fine-grain TMR. We employed hardware-aware error models, i.e., WSBF and NBER to assess the effectiveness of these HTs in mitigating faults across a diverse range of NN architectures, including LeNet5, MnasNet, MobileNet V2, ResNet18, and SqueezeNet.  
The experimental results show that implementing hardening techniques (HTs) requires significant effort, which can render them impractical. For example, applying Swap ReLU6 to MobileNet V2 with WSBF results in a modest accuracy drop to 78.99%, despite being superior to other HTs. Its implementation effort is rated at 6 out of 10, which may lead to a preference for alternative HTs.  
In contrast, range-restriction-based HTs, such as Adaptive Clipper and Ranger, offer a better trade-off between implementation effort and performance, with Adaptive Clipper rated up to 6/10 for effort and a maximum accuracy degradation of 4.5%. Additionally, Adaptive Clipper incurs only 3.52% inference time overhead when implemented on Mnasnet, while Ranger shows a higher overhead of 7.54%. Furthermore, range-restriction-based HTs can effectively mask fault effects in over 58% of cases during hardware-aware fault injections, particularly when Adaptive Clipper is used on Lenet5 compared to the unhardened version.  
In the future, we plan to expand our test bench to include more neural network architectures commonly used in Edge AI applications, such as split computing NNs, quantized NNs, and sparse NNs. Moreover, we will develop a new software-based hardening technique based on the results presented in this work.  
**Author Contributions:** Conceptualization: G.E., J.-D.G.-B. and J.E.R.C.; methodology: G.E., J.-D.G.-B., J.E.R.C. and M.S.R.; software/hardware: G.E. and J.-D.G.-B.; validation, G.E., J.-D.G.-B. and J.E.R.C.; formal analysis: G.E., J.-D.G.-B., J.E.R.C. and M.S.R.; writing--original draft preparation: G.E., J.-D.G.-B.; writing--review and editing: G.E., J.-D.G.-B., J.E.R.C. and M.S.R., J.-D.G.-B., J.E.R.C. and M.S.R.; visualization: G.E., J.-D.G.-B., J.E.R.C. and M.S.R. All authors have read and agreed to the published version of the manuscript.  
**Funding:** This work has been supported by the National Resilience and Recovery Plan (PNRR) through the National Center for HPC, Big Data and Quantum Computing. HPC CINECA provided the computational resources through the projects try24_limas and try24_guerrero from ISCRA project.  
**Data Availability Statement:** Data are contained within the article.  
**Conflicts of Interest:** The authors declare no conflicts of interest.