# Exploring Synergy of Denoising and Distillation: Novel Method for Efficient Adversarial Defense
## 2 Related Works
### Adversarial Attack
#### 2.1.2 Fast Gradient Sign Method  
FGSM [20] is a prominent adversarial attack technique that manipulates the gradient of a neural network to confuse a target model. A key component of FGSM's strategy is the generation of perturbations by leveraging the target model gradient. These perturbations are then utilized to produce adversarial examples, which are a unique type of dataset constructed with the intent of confusing neural networks.  
Adversarial examples were generated by evaluating the contribution of each pixel in an image to the loss value using a gradient. Subsequently, distortions are introduced into each pixel value in proportion to its contribution, producing datasets that are likely to induce misclassification in the trained models. By implementing this mechanism, FGSM can launch a detrimental attack on a target model.  
[PERSON] et al. [20] observed that the decision boundaries of most deep-learning models were excessively linear. This characteristic facilitates the estimation of the decision boundary, rendering models susceptible to security breaches.  
Adversarial examples that can trigger misclassifications in the model can be broadly represented, as shown in Equation (1).  
\[x_{adv}=x+\delta,\|\delta\|_{\infty}\leq\varepsilon \tag{1}\]  
\[\delta=\epsilon sign(\bigtriangledown_{x}L(\theta,x,y)) \tag{2}\]  
In Equation (1), \(x\) denotes the original dataset while \(x_{adv}\) symbolizes the adversarial example that is being crafted. The variable \(\delta\) illustrates the perturbation generated by the adversarial assault. Consequently, an adversarial example can be created by integrating this perturbation into the original data. More importantly, \(\delta\) was confined to a certain range under the \(L_{\infty}\) norm constraint. Hence, \(\|\delta\|_{\infty}\leq\varepsilon\) signifies that the \(L_{\infty}\)-norm constraint is not violated. The magnitude of perturbation that can result in misclassification is given by Equation (2). In this equation, \(\varepsilon\) is represented by an epsilon that indicates the perturbation intensity. The loss function \(L\) delineates the process of updating the gradient in the direction opposite to the label \(y\) using the \(\theta\) parameter of the deep learning model. This operation  
Figure 1: Illustration of the adversarial attack process on the decision boundary. The (**left**) plot shows the initial decision boundary separating two groups of data points. In the (**right**) plot, adversarial perturbations are applied to selected data points, shifting them across the decision boundary and causing misclassification.  
is known as the gradient ascent. Consequently, the implementation of gradient ascent can induce a misclassification in the model. The FGSM poses a formidable challenge for AI models. The FGSM technique offers the flexibility to adjust the attack intensity subtly through an epsilon hyperparameter, as illustrated in Figure 2. An increase in the epsilon value intensifies the attack but simultaneously leads to the creation of unnatural adversarial examples, making their detection by humans relatively easy. Conversely, a smaller epsilon value makes adversarial examples less noticeable to humans yet allows AI to counter adversarial attacks more efficiently. Therefore, an attacker must judiciously select the epsilon value to ensure an effective attack. The versatility of the FGSM in executing attacks of varying intensities underscores its potential to severely compromise the target model [20].