# Improving Adversarial Robustness via Distillation-Based Purification
## 5 Conclusions  
In this paper, we proposed a novel adversarial purification framework for improving the robustness of deep neural networks against adversarial attacks. Our approach utilizes a convolutional autoencoder to capture image features and spatial structure, and a student model is trained on the purified images using knowledge distillation from two teacher models. Experimental results demonstrate that our proposed method can effectively remove adversarial noise from input images and improve model robustness against both white-box and black-box attacks. We also find that the number of teacher models used for knowledge distillation and the way they are trained affects the performance of the student model, and that using two different teacher models improves the performance of the purifier the most. Our approach also outperforms existing state-of-the-art methods in terms of accuracy and robustness. Future work will focus on exploring the effectiveness of our approach on other types of neural networks and datasets.  
Conceptualization, I.K. and D.-K.C.; methodology, I.K.; software, I.K.; validation, I.K. and D.-K.C.; formal analysis, I.K.; investigation, I.K.; resources, I.K.; data curation, I.K.; writing--original draft preparation, I.K. and D.-K.C.; writing--review and editing, D.-K.C. and  
\begin{table}
\begin{tabular}{c c c c} \hline \hline
**AT** & **NT** & **KD** & **PGD8** & **PGD16** & **FGSM** & **BIM** & **CW** & **AA** \\ \hline ✓ & - & - & 34.16 & 28.01 & 38.07 & 33.29 & 56.54 & 43.20 \\ ✓ & - & ✓ & 35.46 & 28.70 & 38.42 & 34.92 & 56.32 & 42.77 \\ - & ✓ & ✓ & 35.77 & 29.47 & 36.93 & 34.88 & **57.90** & 40.50 \\ ✓ & ✓ & ✓ & **40.20** & **33.57** & **40.12** & **38.37** & 56.87 & **46.91** \\ \hline \hline \end{tabular}
\end{table}
Table 2: Resultsof our ablation study. ✓ indicates that the corresponding component is adopted. Each bold number means the best performance against each adversarial attack.  
\begin{table}
\begin{tabular}{l c c c} \hline \hline  & **Ours** & **NRP [(29)]** & **SOAP [(30)]** \\ \hline Original & 89.46 & 89.46 & 89.46 \\ PGD8 & **40.20** & 35.07 & 39.14 \\ PGD16 & 33.57 & 35.57 & **35.65** \\ FGSM & **40.12** & 39.79 & 37.43 \\ BIM & 38.37 & 31.96 & **40.18** \\ CW & 56.87 & 57.24 & **79.22** \\ AA & **46.91** & 12.34 & 41.03 \\ \hline \hline \end{tabular}
\end{table}
Table 1: Comparison of purification performance against various adversarial attacks. Each bold number means the best performance against each adversarial attack.  
S.-C.L.; visualization, I.K.; supervision, D.-K.C. and S.-C.L.; project administration, D.-K.C. and S.-C.L.; funding acquisition, S.-C.L. All authors have read and agreed to the published version of the manuscript.  
This work was partly supported by the DGIST R&D program of the Ministry of Science and ICT of KOREA (23-IT-10-03 and 23-DPIC-08) and the Institute of Information & communications Technology Planning & Evaluation (IIITP) grant funded by the Korea government (MSIT) (No.2020-0-01373, Artificial Intelligence Graduate School Program (Hanyang University)).  
**Infrared Consent Statement:** Not applicable.  
**Data Availability Statement:** CIFAR10 can be found here: [[https://www.cs.toronto.edu/~kriz/cifar.html](https://www.cs.toronto.edu/~kriz/cifar.html)]([https://www.cs.toronto.edu/~kriz/cifar.html](https://www.cs.toronto.edu/~kriz/cifar.html)), accessed on 5 September 2023.  
The authors declare no conflict of interest.