# Comparing Application-Level Hardening Techniques for Neural Networks on GPUs
## 5 Experimental Results
### Fault Mitigation with Respect to NBER  
Figure 7 reports the analysis of the effect of the selected HTs when multiple bit-flips are performed in different locations of the intermediate output tensor according to the NBER model used in this paper. As mentioned in Section 3.1.2, the NBER determines that the intermediate FM is divided into patches (tiles) based on the TS, and the number of selected patches is selected based on the BIER. The number of target neurons among the selected patches is based on the NER. Thus, considering that the bit_location is fixed for each injection, \(BER=NER\times BIER\). In Figure 7, the accuracy trend is reported with respect to the NBER. In general, we can notice that observing the behavior of HTs under NBER, the HTs' effectiveness, thus the final ranking, significantly depends on the NN architecture where they are implemented. When the baseline LeNet architecture is affected by faults, the accuracy faces a significant drop from 89.60% to 71.50%. At the same time, the resiliency of the hardened version of Lenet is proven by the lower slope of the accuracy drop at increasing BER. For example, fine-grain TMR registers a decreasing trend with a similar slope as Baseline but going from 98% to 75.37%. When Ranger and Adaptive Clipper are implemented, Lenet5 experiences an accuracy drop of only 4.2% and 0.5% from the lowest (0.16%) to the highest BER (16%), respectively. When median filter and Swap ReLU6 are implemented on Lenet5, the NN is less impacted by the bit error rate (BER) FI reaching the highest BER, the accuracy scores of 84.80% and 88.61%, respectively. Moving to Mnasnet, we can notice that fine-grain TMR outperforms the range-restriction-based HTs, keeping a constant increase in accuracy of up to 17.9% with respect to the Baseline regardless of the BER, which highlights the importance of introducing redundancy, even at the application level, in lightweight NNs. Concerning Adaptive Clipper, the reliability enhancement effect is more sensible, leading to an increase of 3.26% with respect to the baseline with the highest BER. On the other hand, Ranger is ineffective when it is implemented on Mnasnet due to the lower accuracy with respect to the Baseline. In fact, the Baseline can reach an accuracy higher than Ranger of up to 1%. When Ranger is implemented on Mobilenet V2, it shows better reliability enhancement effectiveness with an average increase in accuracy by 2.7% with respect to the Baseline. Nonetheless, fine-grain TMR outperforms other HTs, reporting an accuracy of 72.96% and 68.93% when the lowest and highest error rates are injected, respectively. Interestingly, Swap ReLU6 and Adaptive Clipper reported the lowest accuracy scores (e.g., 52.23% and 46.07%, respectively, with the highest BER) due to the ineffectiveness of the additional training step that the implementation procedure of the HTs include. Concerning Resnet18, the decreased slope of all curves with respect to the Baseline suggests that all HTs improve the resiliency to injected errors, except for median filter. However, the Baseline accuracy ranges from 77.89% (with \(\text{BER}=0.16\%\)) to 71.12% (with \(\text{BER}=16\%\)), and Baseline consistently outperforms Swap ReLU6, which reports an average accuracy of 71%. On the other hand, Adaptive Clipper, Ranger, and fine-grain TMR achieve an average accuracy of 75.98%, 80.39%, and 81.64%, respectively. Finally, considering the Squeezenet architecture, Ranger and Adaptive Clipper are capable of limiting the NN accuracy degradation with respect to the fault-free case record  
Figure 7: Accuracy degradation for HTs when injecting errors in multiple FM bit locations. ing a drop of 9.7% for Ranger and 9.06% for Adaptive Clipper with \(\text{BER}=16\%\). On the other hand, similarly to the Baseline, Swap ReLU6 reports an average accuracy of 48.98% and remains stable around that value regardless of the BER due to the lack of batch normalization layers in the original Squeezenet architecture, which limits the Swap ReLU6 implementation.