# Stylized Pairing for Robust Adversarial Defense
## 5 Discussions  
In this section, we discuss the usefulness and influence on model performance and the selection of style images of the proposed stylized pairing method to guide how to use our method. **Necessity for pairing loss:** The pairing loss is proposed based on the observation that standard stylized training without the pairing loss term sees stylized images as data augmentation and has no access to receiving any information to indicate clear images similar to the corresponding stylized images. In order to verify whether pairing loss is really useful, we conduct an experiment to compare stylized training and stylized pairing training. In Figure 5, we show the adversarial accuracy on the ImageNet dataset using the ResNet34 model architecture (other datasets and model architectures in terms of perturbation strength measured by its \(L_{2}\) and \(L_{\infty}\) norms show similar performance). We can find that the model with stylized pairing training shows better performance compared to the model with stylized training without the pairing loss term. This indicates that pairing loss is useful to improve the performance against adversarial attacks. We conjecture that the semantic similarity of the stylized images and their corresponding normal images has been significantly enhanced, and for stylized training, semantic similarity is more important compared with pixel-level similarity. Unlike the work [28] where they utilize the adversarial logit pairing and clear logit pairing to train robust models, even though they use the pairing loss, for adversarial logit, the difference between adversarial examples and the corresponding original examples is still very small. In essence, it still pursues a pixel-level similarity. For clean logit, they aim to smooth the gradient of the cost function with regard to inputs by forcing the output of samples from different classes to be similar; therefore, in fact, it penalizes the sensitivity of the divergence between the predictions and uniform uncertainty. **Standard accuracy:** We studied the performance of models on normal examples in order to test whether stylized pairing training would reduce the performance of models. In Tables 1 and 2, we compared the models with stylized pairing training and normal training and reported the accuracy rate of three model architectures on CIFAR10, CIFAR100, and ImageNet datasets, respectively. We can find that there exists a decline in accuracy rate in most cases, and stylized pairing training would lead to about a 1% accuracy reduction on clean examples. Similar situations also exist in adversarial training. [PERSON] et al. [34] pointed out that while adversarial training can improve robust accuracy in terms of adversarial examples, it hurts standard accuracy (when there is no adversary). There is a tradeoff between standard and robust accuracy. We speculate that similar to adversarial training, the stylized pairing loss acts as a regularization term leading to the reduction of performance, and the poor generating quality of stylized images is another main reason for affecting the performance of models. As a meaningful aspect, we will try to improve the performance of models with stylized pairing training in future work either by increasing the quality of stylized images or by losing the constraint. Figure 4: The performance comparisons between models with stylized pairing training and those with adversarial training on ImageNet. The red lines refer to stylized pairing training and the green lines refer to adversarial training. **Shape-biased features:** The motivation of stylized pairing training is to enhance the model's capability on learning shape-biased features and therefore to enhance the model's robustness against adversarial examples.