# Exploring Synergy of Denoising and Distillation: Novel Method for Efficient Adversarial Defense
## 1 Introduction  
Technological advancements in artificial intelligence (AI) and computational capabilities continue to progress. The commercialization of AI is rapidly expanding at an unprecedented rate [1]. Recently, research on model lightweighting for the commercialization of AI has been actively conducted, and technologies incorporating AI in various fields, such as autonomous driving, robotics engineering, Internet of Things (IoT), and medical AI, are already being applied in real life [2, 3, 4, 5]. However, if malicious attacks are carried out against AI models, the damage could be significant, and caution is required for actual use [6, 7]. In particular, the damage could be even greater when AI models were attacked, including autonomous driving [8] and medical AI [9, 10], which have already been commercialized and used [11, 12]. In such situations, research on building more advanced and efficient defense systems from a security perspective is required for the practical use of AI technology. From a security perspective, AI systems can be exposed to various security threats throughout the pipeline, from data collection to model deployment and utilization. During the data collection and transmission stages, there exist a man-in-the-middle attack [13], where an attacker intercepts or manipulates communication, and a data extraction attack, which captures and leaks sensitive data such as personal information [14]. In addition, during the data preprocessing stage, there is a data tampering attack, in which attackers maliciously tamper with the data to include malicious data in the preprocessing process [15, 16]. Among the various attacks that threaten AI, adversarial attacks are the most fatal [17]. Adversarial attacks can carry out various attacks, depending on the attacker's purpose, by reverse engineering the model information for targeting the target model. Adversarial attacks can be classified into three types depending on their purposes. An evasion attack can be used to insert noise (perturbation) into normal data to tamper with the data [18], whereas there exists a poisoning attack that inserts malicious data into the dataset during model learning to interfere with model learning and prevent the AI model from conducting normal learning [19]. In addition, an inversion attack extracts the learning data in reverse by analyzing the output value for a given input from the model [20], which can cause serious security threats to the AI model. Such adversarial attacks pose serious security threats to AI models, making it essential to develop robust countermeasures. These attacks can critically undermine AI model reliability and safety, hindering secure deployment in real-world applications. Therefore, defending against these threats is crucial to ensure AI models operate safely and effectively, especially as they become increasingly integrated into sensitive domains. Therefore, this study conducts a vulnerability analysis related to evasion attacks, which tamper with data to prevent the model from functioning normally among adversarial attacks, and aims to build a defense model that is robust and efficient in terms of AI security. To maximize the denoising effect, we first designed a model using knowledge distillation techniques [21]. We then incorporated a denoising module [12] into the student model, which users will ultimately use. By constructing a composite defense model using knowledge distillation and denoising modules, it is possible to respond to adversarial attacks more efficiently than the existing defense models. The research process was as follows:  
1. Review current defense strategies against adversarial attacks and classification of defense mechanisms;
2. Analysis of the transitivity of adversarial robustness and the definition of considerations required to build an adversarial defense model;
3. Conceptualization and design of a defense model based on knowledge distillation and feature denoising for constructing an effective adversarial attack defense measure.