# Stylized Pairing for Robust Adversarial Defense
## 2 Related Works
### Adversarial Defense  
Since DNNs are vulnerable to adversarial examples [1], many researchers have been devoted to investigating adversarial defense. The adversarial defense methods can be divided into three categories: Robust Optimization, Input pre-processing, and Adversarial Detection.  
* Robust Optimization-based methods aim to improve the robustness of models either by introducing regularization term [11], certification bounds [10], adversarial training [5] or explainability [14; 15].
* Input pre-processing-based methods are based on the intuition to counteract the effect of adversarial perturbation, and they are usually achieved by data compressing [21], input encoding [22], input transforming [23], and so on.
* Adversarial Detection-based methods can be further divided into two categories: the methods utilizing the prediction inconsistency [24], and methods utilizing statistical characteristics to distinguish between adversarial and normal example [25].  
We concentrate on Robust Optimization with explainability in this study. There are works trying to train robust models by adding better intuition to the models through explainability. [PERSON] [14] utilized the edge map as an additional channel to guide models to learn the shape-bias representation. In this approach, the edge map is obtained by the Canny edge detector, and they perform adversarial training over the 2D (Gray+Edge) or 4D (RGB+Edge) input. [PERSON] et al. [15] thought that unlike DNNs, people perceive images based on their predominant features; therefore, they attempted to train networks to form coarse impressions based on the information in higher bit planes and utilized the lower bit planes only to refine their prediction. Compared with other robust optimization methods, the robust optimization method based on explainability needs more domain knowledge, so the related research work is in its infancy, but it is obviously a field of great research value. Meanwhile, there are some works utilizing stylized images for other problems. [PERSON] et al. [26] and [PERSON] [27] augmented the dataset with stylized images to address the Domain Generalization problem, where the classifier must generalize to an unknown target domain. Different from our work, these works see stylized images more as a way of data augmentation.  
In addition, it is worth mentioning that [PERSON] et al. [28] proposed the concept of the logit pairing in which they introduced adversarial logit pairing and clear logit pairing. The adversarial logit pairing encourages logits for pairs of clear examples and their corresponding adversarial example to be similar, and the clean logit pairing selected two random clean training examples (typically not even from the same class) to perform pairing training in order to minimize the logit outputs between two classes. [PERSON] et al. [29] proposed stylized adversarial training which utilizes the content and the style of the target image as well as the classifier boundary information to generate adversarial perturbation and perform adversarial training. Unlike these methods, our proposed stylized pairing emphasizes a semantic similarity, while adversarial logit pairing and stylized adversarial training focus on a pixel-level similarity directly relate to adversarial perturbation, and clear logit pairing plays a role of regularity terms to minimize the difference between samples from different classes.