# Comparing Application-Level Hardening Techniques for Neural Networks on GPUs
## 6 Discussion  
Most of the prior studies utilize different and hardware-agnostic error models to evaluate the effectiveness of the proposed hardening techniques. Moreover, they employ different NN architectures and datasets, raising concerns on the applicability of the achieved results to other test benches and evaluation setups. In addition, the previous works evaluate the HTs, neglecting the effort required for the HT implementation. Both the lack of a standardized approach for evaluating hardening techniques under hardware-aware error models and neglecting the effort required for the HT implementation limits the possibility to identify the optimal HT. The technique should find the best trade-off between the impact of its implementation on the normal functioning of the NN and the effectiveness in the NN reliability enhancement. Our evaluation methodology allows us to assess the feasibility of implementing an HT on a specific NN and to evaluate the effectiveness of application-level HTs across various NN architectures and under hardware-aware error models. The employed hardware-aware error models (WSBF and NBER) simulate realistic fault conditions in GPU-based systems, providing a more accurate assessment of HT effectiveness. We used our evaluation methodology to assess both the effort of implementation and the effectiveness in terms of reliability enhancement of five state-of-the-art hardening techniques. In Table 6, we summarize the most important results of our evaluations that come from both the HT impact assessment (considering the EoI and the ITO) and the reliability enhancement effectiveness (considering the RAD computed at the end of the WSBF FI when the bit-flip is performed on the bit 30 and at the end of the NBER FI campaign when the simulation is run at the highest bit error rate, i.e., 16%). As a first result, we can notice that as a result of the training process of Swap ReLU6 when it is implemented on Mnasnet, as well as the training process of median filter when it is implemented on Mnasnet, Mobilenet V2 and Squeezenet, failed to converge. For this reason, the cells of the RAD corresponding to these NN architectures are empty. Specifically, median filter makes Mnasnet, Mobilenet, and Squeezenet converge at the accuracy scores of 35.40%, 54.50%, and 62.34%, respectively. These NNs involve convolutions with the so-called grouping strategy, an optimized version of the original convolution algorithm. However, the grouping strategy involves the processed tensors being shaped accordingly, which makes the implementation of the median filter at the application level almost impractical. On the other side, Mnasnet converged to an accuracy of 35.40% when Swap ReLU6 was implemented due to the higher variance in the distribution of the intermediate feature compared to the variance of the intermediate feature distributions of the original NN architecture (up to 10%). Adaptive Clipper was initially designed for split computing NNs, but in this work, it has been adapted to general-purpose NNs. The article proposing Adaptive Clipper originally evaluated the technique against WSBF FI. When Adaptive Clipper is implemented on Lenet5, it achieved an EoI score of 4/10 and an ITO of 0.29% and demonstrated a significant resilience against WSBF, decreasing the RAD from 39.06% to 9.75%. Similarly, against NBER, Adaptive Clipper effectively improved Lenet5 resilience, leading to a reduced RAD