# Exploring Synergy of Denoising and Distillation: Novel Method for Efficient Adversarial Defense
## 4 Experiment and Evaluation  
### Experimental Environment  
This study aims to enhance robustness against adversarial attacks. For this purpose, we conducted experiments using benchmark datasets that provide an impartial measure of AI model performance. We used the CIFAR-10 dataset [49], which is one of the most commonly used datasets in computer vision. The CIFAR-10 dataset was composed of 10 classes of 32 \(\times\) 32 pixel RGB images and included 60,000 images: 50,000 for training and 10,000 for testing. An experiment comparing the proposed method with previous methods was conducted using an Intel Core i9 CPU (Intel, Santa Clara, CA, USA) and an NVIDIA RTX 3090 GPU (NVIDIA, Santa Clara, CA, USA). The software environment included Python version 3.8.13 and PyTorch version 1.12.1. Regarding the dataset of the proposed model, we divided the dataset into training, validation, and testing sets. We also employed k-fold validation, with K set to 5, to address potential test data bias and overfitting issues and to optimize the hyperparameters. The performance of the final model was assessed using a test dataset. The models used for comparison were divided into convolutional neural network (CNN) and knowledge distillation-based models. All models were trained for 300 epochs. For the CNN models, we used the cross-entropy loss as the criterion, Adam as the optimizer, and cosine-annealing LR as the scheduler. In addition, the learning rate was set to 1 \(\times\) 10\({}^{-4}\), and the batch size was set to 128. By contrast, the knowledge distillation models used soft distillation loss as the criterion, with the temperature set to 10. All other learning conditions were the same as those used for the CNN-based models.  
\[Attack\;Success\;Rate=Benign\;Accuracy-Attacked\;Accuracy \tag{7}\]  
\[Defense\;Success\;Rate=100-Attack\;Success\;Rate \tag{8}\]  
\[Accuracy=\frac{(TP+TN)}{(TP+TN+FP+FN)} \tag{9}\]  
\[\text{F1-Score}=2\times\frac{(Precision\times Recall)}{(Precision+Recall)} \tag{10}\]  
For the practical application of the model, not only its defensive performance but factors such as model parameters and learning time are also crucial. Therefore, we primarily focused on the analysis of the following metrics for a comprehensive comparison of the implementation of the defense model: (1) Accuracy, F1-score, and defense success rate; (2) Model parameters and model learning time. The main metric for assessing defensive performance is the attack success rate, with a lower rate indicating a superior defense. This rate is often referred to as the rate of accuracy decrease, as described in Equation (7). In addition, we employ another metric, the defense success rate, as detailed in Equation (8).  
The model performance was evaluated using accuracy and the F1-score, as outlined in Equations (9) and (10).  
In Equation (7), \"benign accuracy\" signifies the accuracy evaluated using the original test dataset that is devoid of any attacks, whereas \"attacked accuracy\" represents the accuracy calculated from adversarial examples. Equation (8) introduces the defense success rate for a more straightforward evaluation of the model's defensive abilities. A higher rate signifies a stronger defense model. We used the defense success rate, accuracy, and F1-score as metrics to assess the performance and defensive robustness of the model. The feasibility of model deployment was assessed based on its parameters and learning time. In addition, all attacks were conducted in a white-box setting [50], operating under the assumption that the attacker was fully informed about the model. We utilized the FGSM method based on the \(L_{\infty}\) norm for model attacks. The perturbation magnitude (\(\epsilon\)) was automatically adjusted by the foolbox library [51] used in our experiments to ensure effective and bounded perturbations.