# Exploring Synergy of Denoising and Distillation: Novel Method for Efficient Adversarial Defense
## 4 Experiment and Evaluation
### Experimental Result and Evaluation
#### 4.2.1 Evaluation of Defensive Performance  
We conducted performance assessments focusing on DiNo-Net by distinguishing between instances with and without FGSM attacks. The experimental results are presented in Table 1. Before proceeding with the comparative evaluations based on various defense techniques, we performed a defense performance evaluation for a general model that did not incorporate a defense function. Although it showed respectable classification performance with over 91% accuracy without any attacks, the accuracy fell sharply to nearly 25% when an attack was executed, indicating acute defensive vulnerability. As discussed in Section 2 (Related Works), the denoising technique matches the characteristics of obfuscated gradients and thus does not constitute a fundamental defense technique. Furthermore, if an attacker identifies a denoising module and generates an adversarial example, the denoising technique will become ineffective against adversarial attacks. Consequently, the denoising methods are vulnerable and difficult to employ independently. For the second model in Table 1, this study observed a decrease in accuracy of approximately 10% compared with the standard model without defense capabilities. This is because the denoising performed on the original dataset damaged the pixel values of the original data. Moreover, this study analyzed the defensive vulnerability using a defense success rate of 31.7%, as the attacker generated an adversarial example by considering the denoising module and initiated an attack. In the context of adversarial training, this study demonstrated superior effectiveness in acquiring defensive attributes. However, this poses a drawback in that it requires additional learning time during adversarial training. Furthermore, this may have decreased the accuracy of the original dataset. The third model in Table 1 pertains to a ResNet that has undergone adversarial training. We determined that it possesses a defensive performance of 71.8%, suggesting that it can effectively defend against adversarial attacks. However, a 15 % reduction in accuracy was observed compared to the regular model (the first model in Table 1), underscoring the limitations associated with performance reduction. We also validate a defense technique that integrates denoising and adversarial training. The fourth model in Table 1 presents the experimental results. The fusion of denoising and adversarial training demonstrated a defense performance of 72.0%, thereby validating it as the most efficient defense technique among the compared models. Although our analysis shows that this model has 77% accuracy, it does not overcome the decreased accuracy issues stemming from adversarial training. The assessment of the defense model using knowledge distillation aligns with the fifth model in Table 1. This model demonstrated 84.8% accuracy for benign inputs, exhibiting almost identical performance to the 84.7% accuracy of DiNo-Net. Furthermore, it was observed to have the highest defense success rate of 71.6% among the comparison models. However, it displayed a lower defense performance than the 72.7% defense success rate of DiNo-Net. The proposed DiNo-Net employs knowledge distillation and incorporates a denoising module into the student model to eliminate perturbations and augment the defense capabilities. Therefore, we verified that it achieved a 1.1% performance enhancement over the best attack accuracy of 56.4% among the compared models, with an attack accuracy of 57.5%. Furthermore, DiNo-Net demonstrated a superior defensive performance of 72.7%, outperforming both standalone feature denoising and the combined use of denoising and adversarial training. Hence, it has been proven that the fusion of the denoising module and knowledge distillation can improve defense performance against adversarial attacks. Moreover, DiNo-Net does not directly participate in adversarial training. Consequently, it reduces the accuracy degradation issue, which is a commonly mentioned limitation of adversarial training.