# Comparing Application-Level Hardening Techniques for Neural Networks on GPUs
## 5 Experimental Results
### HT Impact Assessment
#### 5.1.1 Trading off EoI and Accuracy
Moreover, both Swap ReLU6 and median filter incur a slight accuracy reduction compared to the baseline model, with drops of \(1\%\) and \(2\%\), respectively. The increased effort involved in implementing these methods must also be considered. Regarding Mnasnet, Swap ReLU6 and median filters failed to converge at an acceptable accuracy level (i.e., the best-reached accuracy is \(70.30\%\) and \(35.40\%\)). At the same time, Adaptive Clipper and Ranger do not face any accuracy drop with respect to the baseline model. Furthermore, when Ranger and Adaptive Clipper are implemented, the EoI scores increase in complexity. Adaptive Clipper is the only HT requiring a training step that does not face any accuracy drop. In particular, as the original Mnasnet architecture involves \(35\) activation functions, Adaptive Clipper has increased complexity in the implementation to \(3/5\). From the training adaptation standpoint, the assigned score is \(3/5\) due to the same hyperparameter setup as the baseline model required during the training stage. Despite the EoI scores of \(6/10\) and \(3/5\), Adaptive Clipper and Ranger do not cause an accuracy drop, maintaining it at \(83.00\%\). On the other hand, despite fine-grain TMR not facing an accuracy drop and not needing training adaptations, the implementation complexity is \(5/5\). In MobileNet V2, the median filter reaches an EoI score of \(10/10\) due to the need for a custom operation and the extensive training process involved. In addition, this results in an accuracy of \(54.50\%\).The limitation stems from the grouped convolutions inherent in the original MobileNet V2 architecture, which highlights challenges in causing the training algorithm to converge when integrating the median filter within the current architecture. On the other hand, both the Ranger and Adaptive Clipper methods achieved the same EoI scores as MnasNet. This can be attributed to the inclusion of \(38\) activation functions and a training process that closely resembles that of the baseline Mobilenet V2 during the implementation of Adaptive Clipper. After completing the training for Adaptive Clipper, the accuracy of MobileNet V2 only decreased from a baseline of \(78.99\%\) to \(74.46\%\). Interestingly, despite a medium EoI score of \(6/10\), it still resulted in an accuracy increase of \(5.37\%\). Additionally, fine-grain TMR does not lead to a drop in accuracy for MobileNet V2, and it received a complexity score of \(5/5\). When Resnet18 architecture is considered, all HTs achieve an acceptable level of accuracy. The original architecture uses only eight activation functions, which leads to complexity scores of \(2/5\) for both Adaptive Clipper and Ranger. Although Adaptive Clipper requires retraining, it has a training adaptation score of \(3/5\) due to the similarities of the new training process to one of the original NN. Furthermore, Adaptive Clipper experiences a significant accuracy drop of \(3.13\%\), while Ranger does not require any additional training steps, allowing it to maintain a stable accuracy of \(85.72\%\). In contrast, when Swap ReLU6 is implemented on ResNet18, the neural network achieves an EoI score of \(6/10\) due to the interchange of batch normalization and ReLU layers. In addition to this high EoI score, the network experiences a considerable accuracy drop from \(85.72\%\) to \(77.00\%\). Additionally,implementing a median filter on ResNet18 results in an accuracy drop of 9.72%, along with an EOI score of 10/10. Finally, all HTs impact the accuracy of the neural NN when implemented on SqueezeNet. Specifically, the accuracy increases by 5.00% compared to the baseline NN when using the Adaptive Clipper with a medium score of 6/10 for EOI. In contrast, the Ranger implementation avoids an additional training step and achieves the same accuracy as the baseline model, which is 77.50%. Similarly to Adaptive Clipper, the final EOI score for Ranger is 3/10.