Title: IMPROVING THE GENERALIZATION OF ADVERSARIAL TRAINING WITH DOMAIN ADAPTATION

Abstract: promising for improving the robustness of deep learning models. However, most existing adversarial training approaches are based on a specific type of adversarial attack. It may not provide sufficiently representative samples from the adversarial domain, leading to a weak generalization ability on adversarial examples from other attacks. Moreover, during the adversarial training, adversarial perturbations on inputs are usually crafted by fast single-step adversaries so as to scale to large datasets. This work is mainly focused on the adversarial training yet efficient FGSM adversary. In this scenario, it is difficult to train a model with great generalization due to the lack of representative adversarial samples, aka the samples are unable to accurately reflect the adversarial domain. To alleviate this problem, we propose a novel Adversarial Training with Domain Adaptation (ATDA) method. Our intuition is to regard the adversarial training on FGSM adversary as a domain adaption task with limited number of target domain samples. The main idea is to learn a representation that is semantically meaningful and domain invariant on the clean domain as well as the adversarial domain. Empirical evaluations on Fashion-MNIST, SVHN, CIFAR-10 and CIFAR-100 demonstrate that ATDA can greatly improve the generalization of adversarial training and the smoothness of the learned models, and outperforms state-of-the-art methods on standard benchmark datasets. To show the transfer ability of our method, we also extend ATDA to the adversarial training on iterative attacks such as PGD-Adversial Training (PAT) and the defense performance is improved considerably.

1. INTRODUCTION:

Deep learning techniques have shown impressive performance on image classification and many other computer vision tasks. However, recent works have revealed that deep learning models are often vulnerable to adversarial examples (Szegedy et al., 2014; Goodfellow et al.; Papernot et al., 2016) , which are maliciously designed to deceive the target model by generating carefully crafted adversarial perturbations on original clean inputs. Moreover, adversarial examples can transfer across models to mislead other models with a high probability (Papernot et al., 2017; Liu et al., 2017) . How to effectively defense against adversarial attacks is crucial for security-critical computer vision systems, such as autonomous driving.

As a promising approach, adversarial training defends from adversarial perturbations by training a target classifier with adversarial examples. Researchers have found (Goodfellow et al.; Kurakin et al., 2016b; Madry et al., 2018) that adversarial training could increase the robustness of neural networks. However, adversarial training often obtains adversarial examples by taking a specific attack technique (e.g., FGSM) into consideration, so the defense targeted such attack and the trained model exhibits weak generalization ability on adversarial examples from other adversaries (Kurakin et al., 2016b) . Tramèr et al. (2018) showed that the robustness of adversarial training can be easily circumvented by the attack that combines with random perturbation from other models. Accordingly, for most existing adversarial training methods, there is a risk of overfitting to adversarial examples crafted on the original model with the specific attack.

In this paper, we propose a novel adversarial training method that is able to improve the generalization of adversarial training. From the perspective of domain adaptation (DA) (Torralba & Efros, 2011) , there is a big domain gap between the distribution of clean examples and the distribution of adversarial examples in the high-level representation space, even though adversarial perturbations are imperceptible to humans. Liao et al. (2018) showed that adversarial perturbations are progressively amplified along the layer hierarchy of neural networks, which maximizes the distance between the original and adversarial subspace representations. In addition, adversarial training simply injects adversarial examples from a specific attack into the training set, but there is still a large sample space for adversarial examples. Accordingly, training with the classification loss on such a training set will probably lead to overfitting on the adversarial examples from the specific attack. Even though Wong & Kolter (2018) showed that adversarial training with iterative noisy attacks has stronger robustness than the adversarial training with single-step attacks, iterative attacks have a large computational cost and there is no theoretical analysis to justify that the adversarial examples sampled in such way could be sufficiently representative for the adversarial domain.

Our contributions are focused on how to improve the generalization of adversarial training on the simple yet scalable attacks, such as FGSM (Goodfellow et al.) . The key idea of our approach is to formulate the learning procedure as a domain adaptation problem with limited number of target domain samples, where target domain denotes adversarial domain. Specifically, we introduce unsupervised as well as supervised domain adaptation into adversarial training to minimize the gap and increase the similarity between the distributions of clean examples and adversarial examples. In this way, the learned models generalize well on adversarial examples from different ∞ bounded attacks. We evaluate our ATDA method on standard benchmark datasets. Empirical results show that despite a small decay of accuracy on clean data, ATDA significantly improves the generalization ability of adversarial training and has the transfer ability to extend to adversarial training on PGD (Madry et al., 2018) .

2. BACKGROUND AND RELATED WORK:

In this section, we introduce some notations and provides a brief overview of the current advanced attack methods, as well as the defense methods based on adversarial training.

2.1. NOTATION:

Denote the clean data domain and the adversarial data domain by D and A respectively, we consider a classifier based on a neural network f (x) : R d → R k . f (x) outputs the probability distribution for an input x ∈ [0, 1] d , and k denotes the number of classes in the classification task. Let ϕ be the mapping at the logits layer (the last neural layer before the final softmax function), so that f (x) = sof tmax(ϕ(x)). Let be the magnitude of the perturbation. Let x adv be the adversarial image computed by perturbing the original image x. The cost function of image classification is denoted as J(x, y). We define the logits as the logits layer representation, and define the logit space as the semantic space of the logits layer representation.

We divide attacks into two types: white-box attacks have the complete knowledge of the target model and can fully access the model; black-box attacks have limited knowledge of the target classifier (e.g.,its architecture) but can not access the model weights.

2.2. ATTACK METHODS:

We consider four attack methods to generate adversarial examples. For all attacks, the components of adversarial examples are clipped in [0, 1].

Fast Gradient Sign Method (FGSM). Goodfellow et al. introduced FGSM to generate adversarial examples by applying perturbations in the direction of the gradient.

x adv = x + • sign(∇ x J(x, y true ))

(1)

As compared with other attack methods, FGSM is a simple, yet fast and efficient adversary. Accordingly, FGSM is particularly amenable to adversarial training.

Projected Gradient Descent (PGD). The Projected Gradient Descent (PGD) adversary was introduced by Madry et al. (2018) without random start, which is a stronger iterative variant of FGSM. This method applies FGSM iteratively for k times with a budget α instead of a single step.

x adv0 = x

x advt+1 = x advt + α • sign(∇ x J(x advt , y true ))

x advt+1 = clip(x advt+1 , x advt+1 -, x advt+1 + )

x adv = x adv k (2)

Here clip(•, a, b) function forces its input to reside in the range of [a, b] . PGD usually yields a higher success rate than FGSM does in the white-box setting but shows weaker capability in the black-box setting.

RAND+FGSM (R+FGSM). Tramèr et al. (2018) proposed R+FGSM against adversarially trained models by applying a small random perturbation of step size α before applying FGSM.

x = x + α • sign(N (0 d , I d ))

EQUATION

Momentum Iterative Method (MIM). MIM (Dong et al., 2018) is a modification of the iterative FGSM and it won the first place of NIPS 2017 Adversarial Attacks Competition. Its basic idea is to utilize the gradients of the previous t steps with a decay factor µ to update the gradient at step t + 1 before applying FGSM with a budget α.

x adv0 = x, g 0 = 0 g t+1 = µ • g t + ∇ x J(x advt , y true ) ∇ x J(x advt , y true ) 1

x advt+1 = x advt + α • sign(g t+1 )

x advt+1 = clip(x advt+1 , x advt+1 -, x advt+1 + ) For adversarial training, another approach is to train only with adversarial examples. Nøkland (2015) proposed a specialization of the method (Goodfellow et al.) that learned only with the objective function of adversarial examples. Madry et al. (2018) demonstrated successful defenses based on adversarial training with the noisy PGD, which randomly initialize an adversarial example within the allowed norm ball before running iterative attack. However, this technique is difficult to scale to large-scale neural networks (Kurakin et al., 2016a) as the iterative attack increases the training time by a factor that is roughly equal to the number of iterative steps. Wong & Kolter (2018) developed a robust training method by linear programming that minimized the loss for the worst case within the perturbation ball around each clean data point. However, their approach achieved high test error on clean data and it is still challenging to scale to deep or wide neural networks.

EQUATION

EQUATION

As described above, though adversarial training is promising, it is difficult to select a representative adversary to train on and most existing methods are weak in generalization for various adversaries, as the region of the adversarial examples for each clean data is large and contiguous (Tramèr et al., 2017; Tabacof & Valle, 2016) . Furthermore, generating a representative set of adversarial examples for large-scale datasets is computationally expensive. The adversarial examples are obtained by sampling (x i , y true ) from D, computing small perturbations on x i to generate adversarial perturbations, and outputting (x adv i , y true ). It's known that there is a huge shift in the distributions of clean data and adversarial data in the high-level representation space. Assume that in the logit space, data from either the clean domain or the adversarial domain follow a multivariate normal distribution, i.e., D ∼ N (µ D , Σ D ), A ∼ N (µ A , Σ A ). Our goal is to learn the logits representation that minimizes the shift by aligning the covariance matrices and the mean vectors of the clean distribution and the adversarial distribution.

3. ADVERSARIAL TRAINING WITH DOMAIN ADAPTATION:

To implement the CORrelation ALignment (CORAL), we define a covariance distance between the clean data and the adversarial data as follows.

L CORAL (D, A) = 1 k 2 C ϕ(D) -C ϕ(A) 1 (6)

where C ϕ(D) and C ϕ(A) are the covariance matrices of the clean data and the adversarial data in the logit space respectively, and • 1 denotes the L 1 norm of a matrix. Note that L CORAL (D, A) is slightly different from the CORAL loss proposed by Sun & Saenko (2016) .

Similarly, we use the standard distribution distance metric, Maximum Mean Discrepancy (MMD) (Borgwardt et al., 2006) , to minimize the distance of the mean vectors of the clean data and the adversarial data.

L M M D (D, A) = 1 k 1 |D| x∈D ϕ(x) - 1 |A| x adv ∈A ϕ(x adv ) 1 (7)

The loss function for Unsupervised Domain Adaptation (UDA) can be calculated as follows.

L U DA (D, A) = L CORAL (D, A) + L M M D (D, A) (8) 3.1.2 SUPERVISED DOMAIN ADAPTATION

Even though the unsupervised domain adaptation achieves perfect confusion alignment, there is no guarantee that samples of the same label from clean domain and adversarial domain would map nearby in the logit space. To effectively utilize the labeled data in the adversarial domain, we introduce a supervised domain adaptation (SDA) by proposing a new loss function, denoted as margin loss, to minimize the intra-class variations and maximize the inter-class variations on samples of different domains. The SDA loss is shown in Eq. ( 9).

EQUATION

Here sof tplus denotes a function ln(1 + exp(•)); c ytrue ∈ R k denotes the center of y true class in the logit space; C = { c j | j = 1, 2, ..., k} is a set consisting of the logits center for each class, which will be updated as the logits changed. Similar to the center loss (Wen et al., 2016) , we update center c j for each class j:

EQUATION

)

where 1 condition = 1 if the condition is true, otherwise 1 condition = 0; α denotes the learning rate of the centers. During the training process, the logits center for each class can integrate the logits representation from both the clean domain and the adversarial domain.

3.2. ADVERSARIAL TRAINING:

For adversarial training, iterative attacks are fairly expensive to compute and single-step attacks are fast to compute. Accordingly, we use a variant of FGSM attack (Kurakin et al., 2016b ) that avoids the label leaking effect to generate a new adversarial example x adv i for each clean example x i .

EQUATION

where y target denotes the predicted class arg max{ϕ(x i )} of the model. Compute the loss by Eq. ( 12) and update parameters of network f by back propagation; 10: until the training converges.

L(D:

EQUATION

4. EXPERIMENTS:

In this section, we evaluate our ATDA method on various benchmark datasets to demonstrate the robustness and contrast its performance against other competing methods under different white-box and black-box attacks with bounded ∞ norm. Code for these experiments is available at https: //github.com/JHL-HUST/ATDA.

4.1. EXPERIMENTAL SETUP:

Datasets. We consider four popular datasets, namely Fashion-MNIST (Xiao et al., 2017) , SVHN (Netzer et al., 2011) , CIFAR-10 and CIFAR-100 (Krizhevsky & Hinton, 2009) . For all experiments, we normalize the pixel values to [0, 1] by dividing 255.

Baselines. To evaluate the generalization power on adversarial examples in both the white-box and black-box settings, we report the clean test accuracy, the defense accuracy on FGSM, PGD, R+FGSM and MIM in the non-targeted way. The common settings for these attacks are shown in Table 5 of the Appendix. We compare our ATDA method with normal training as well as several state-of-the-art adversarial training methods:

• (Wong & Kolter, 2018) . Training with cross-entropy loss on the worst case in the ∞ ball of radius around each clean training data point. It could be seen as training with a complicated method of sampling in the ∞ ball of radius . Evaluation Setup. For each benchmark dataset, we train a normal model and various adversarial models with perturbation on a main model with ConvNet architecture, and evaluate them on various attacks bounded by . Moreover, for Ensemble Adversarial Training (EAT), we use two different models as the static pre-trained models. For black-box attacks, we test trained models on the adversarial examples transferred from a model held out during the training. All experiments are implemented on a single Titan X GPU. For all experiments, we set the hyper-parameter λ in Eq. ( 12) to 1/3 and the hyper-parameter α in Eq. ( 10) to 0.1. For more details about neural network architectures and training hyper-parameters, see Appendix A. We tune the networks to make sure they work, not to post concentrates on optimizing these settings.

4.2. COMPARISON OF DEFENSE PERFORMANCE ON ACCURACY:

We evaluate the defense performance of our ATDA method from the perspective of classification accuracy on various datasets, and compare with the baselines.

Evaluation on Fashion-MNIST. The accuracy results on Fashion-MNIST are reported in Table 1a . NT yields the best performance on the clean data, but generalizes poorly on adversarial examples. SAT and EAT overfit on the clean data and the adversarial data from FGSM. PRT achieves lower error against various adversaries, but higher error on the clean data. ATDA achieves stronger robustness against different ∞ bounded adversaries as compared to SAT (adversarial training on FGSM).

Evaluation on SVHN. The classification accuracy on SVHN are summarized in Table 1b . PRT seems to degrade the performance on the clean testing data and exhibits weak robustness on various attacks. As compared to SAT, ATDA achieves stronger generalization ability on adversarial examples from various attacks and higher accuracy on the white-box adversaries, at the same time it only loses a negligible performance on clean data.

Evaluation on CIFAR-10. Compared with Fashion-MNIST and SVHN, CIFAR-10 is a more difficult dataset for classification. As PRT is challenging and expensive to scale to large neural networks due to its complexity, the results of PRT are not reported. The accuracy results on CIFAR-10 are summarized in Table 1c . ATDA outperforms all the competing methods on most adversaries, despite a slightly lower performance on clean data.

Evaluation on CIFAR-100. The CIFAR-100 dataset contains 100 image classes, with 600 images per class. Our goal here is not to achieve state-of-the-art performance on CIFAR-100, but to compare the generalization ability of different training methods on a comparatively large dataset. The results on CIFAR-100 are summarized in Table 1d . Compared to SAT, ATDA achieves better generalization on various adversarial examples and it does not degrade the performance on clean data.

In conclusion, the accuracy results provide empirical evidence that ATDA has great generalization ability on different adversaries as compared to SAT and outperforms other competing methods.

4.3. FURTHER ANALYSIS ON THE DEFENSE PERFORMANCE:

To further investigate the defence performance of the proposed method, we compute two other metrics: the local loss sensitivity to perturbations and the shift of adversarial data distribution with respect to the clean data distribution.

Local Loss Sensitivity. One method to quantify smoothness and generalization to perturbations for models is the local loss sensitivity (Arpit et al., 2017) . It is calculated in the clean testing data as follows. The lower the value is, the smoother the loss function is.

EQUATION

The results of the local loss sensitivity for the aforementioned learned models are summarized in Table 2 . The results suggest that adversarial training methods do increase the smoothness of the model as compared with the normal training and ATDA performs the best.

Distribution Discrepancy. To quantify the dissimilarity of the distributions between the clean data and the adversarial data, we compare our learned logits embeddings with the logits embeddings of the competing methods on Fashion-MNIST. We use t-SNE (Maaten & Hinton, 2008) for the comparison on the training data, testing data and adversarial testing data from the white-box FGSM or PGD. The comparisons are illustrated in Figure 2 and we report the detailed MMD distances across domains in Table 3 . Compared with NT, SAT and EAT actually increase the MMD distance across domains of the clean data and the adversarial data. In contrast, PRT and ATDA can learn domain invariance between the clean domain and the adversarial domain. Furthermore, our learned logits representation achieves the best performance on domain invariance. average accuracy rates over all white-box attacks and all black-box attacks, respectively. The results illustrate that, by aligning the covariance matrix and mean vector of the clean and adversarial examples, UDA plays a key role in improving the generalization of SAT on various attacks. In general, the aware of margin loss on SDA can also improve the defense quality on standard adversarial training, but the effectiveness is not very stable over all datasets. By combining UDA and SDA together with SAT, our final algorithm ATDA can exhibits stable improvements on the standard adversarial training. In general, the performance of ATDA is slightly better than SAT+UDA. As shown in Table 4 , we evaluate the defense performance of PAT and PATDA on various datasets. On Fashion-MNIST, we observe that PATDA fails to increase robustness to most adversaries as compared to PAT. On SVHN, PAT and PATDA fail to converge properly. The results are not surprising, as training with the hard and sufficient adversarial examples (from the noisy PGD) requires the neural networks with more parameters. On CIFAR-10 and CIFAR-100, PATDA achieves stronger robustness to various attacks than PAT. In general, PATDA exhibits stronger robustness to various adversaries as compared to PAT. The results indicate that domain adaptation can be applied flexibly to adversarial training on other adversaries to improve the defense performance. 

5. CONCLUSION:

In this study, we regard the adversarial training as a domain adaptation task with limited number of target labeled data. By combining adversarial training on FGSM adversary with unsupervised and supervised domain adaptation, the generalization ability on adversarial examples from various attacks and the smoothness on the learned models can be highly improved for robust defense. In addition, ATDA can easily be extended to adversarial training on iterative attacks (e.g., PGD) to improve the defense performance. The experimental results on several benchmark datasets suggest that the proposed ATDA and its extension PATDA achieve significantly better generalization results as compared with current competing adversarial training methods.

:



Figures and Tables:

Kurakin et al. (2016b)  scaled the adversarial training to ImageNet(Russakovsky et al., 2015) and showed better results by replacing half the clean example at each batch with the corresponding adversarial examples. Meanwhile,Kurakin et al. (2016b)  discovered the label leaking effect and suggested not to use the FGSM defined with respect to the true label y true . However, their approach has weak robustness to the RAND+FGSM adversary.Tramèr et al. (2018) proposed an ensemble adversarial training to improve robustness on black-box attacks by injecting adversarial examples transferred from a number of fixed pre-trained models into the training data.

However, in this case, the sampled adversarial examples are aggressive but not sufficiently representative due to the fact that the sampled adversarial examples always lie at the boundary of the ∞ ball of radius (see Figure1) and the adversarial examples within the boundary are ignored. For adversarial training, if we train a deep neural network only on the clean data and the adversarial data from the FGSM attack, the adversarially trained model will overfit on these two kinds of data and exhibits weak generalization ability on the adversarial examples sampled from other attacks. From a different perspective, such problem can be viewed as a domain adaptation problem with limited number of labeled target domain samples, as only some special data point can be sampled in the adversarial domain by FGSM adversary.

Figure 1: Illustration of the adversarial sampling by FGSM for x i ∈ R 2 . The blue dot (in the center) represents a clean example and the red dots (along the boundary) represent the potential adversarial examples for the clean example. Consequently, it is natural to combine the adversarial training with domain adaptation to improve the generalization ability on adversarial data. We generate new adversarial examples by the variant of FGSM attack shown in Eq. (11), then we use the following loss function to meet the criteria of domain adaptation while training a strong classifier.

Here λ is the hyper-parameter to balance the regularization term; m is the number of input clean examples; D indicates the input clean examples {x i }, and A the corresponding adversarial examples {x adv i }; L C denotes the classification loss. The training process is summarized in Algorithm 1. Algorithm 1 Adversarial training with domain adaptation on network f (x) : R d → R k . Parameters: Size of the training minibatch is m. 1: Randomly initialize network f (x) and logits centers {c j | j = 1, 2, ..., k}; 2: Number of iterations t ← 0; of data D b = {x 1 , ..., x m } from the training set; 6: Use the current state of network f to generate adversarial examples A b = {x adv 1 , ..., x adv m } by the FGSM variant that avoids label leaking; 7: Extract logits for examples D b , A b by performing forward-backward propagation from the input layer to the logits layer ϕ(x); 8: Update parameters c j for each class j by c t+1 j

Normal Training (NT). Training with cross-entropy loss on the clean training data. • Standard Adversarial Training (SAT) (Goodfellow et al.). Training with the cross-entropy on the clean training data and the adversarial examples from the FGSM variant with perturbation to avoid label leaking. • Ensemble Adversarial Training (EAT) (Tramèr et al., 2018). Training with cross-entropy on the clean training data and the adversarial examples crafted from the currently trained model and the static pre-trained models by the FGSM variant with the perturbation to avoid label leaking. • Provably Robust Training (PRT)

Figure 2: t-SNE visualizations for the embeddings of training data, testing data, and adversarial testing data from FGSM and PGD in the logit space for Fashion-MNIST. The first row to the fifth row correspond to NT, SAT, EAT, PRT and ATDA, respectively.

Figure 3: Ablation experiments for ATDA to investigate the impact of Standard Adversarial Training (SAT), Unsupervised Domain Adaptation (UDA), and Supervised Domain Adaptation (SDA). We report the average accuracy rates over all white-box attacks and all black-box attacks, respectively. 4.5 EXTENSION TO PGD-ADVERSARIAL TRAINING ATDA can simply be extended to adversarial training on other adversaries. We now consider to extend the ATDA method to PGD-Adversarial Training (PAT) (Madry et al., 2018): adversarial training on the noisy PGD with perturbation . By combining adversarial training on the noisy PGD with domain adaptation, we implement an extension of ATDA for PAT, called PATDA. For the noisy PGD, we set the iterated step k as 10 and the budget α as /4 according to Madry et al. (2018).

In this work, instead of focusing on a better sampling strategy to obtain representative adversarial data from the adversarial domain, we are especially concerned with the problem of how to train with clean data and adversarial examples from the efficient FGSM, so that the adversarially trained model is strong in generalization for different adversaries and has a low computational cost during the training.

Table content: <table/>

The accuracy of defense methods on the testing datasets and the adversarial examples generated by various adversaries. On SVHN. The magnitude of perturbations is 0.02 in ∞ norm. On CIFAR-10. The magnitude of perturbations is 4/255 in ∞ norm. On CIFAR-100. The magnitude of perturbations is 4/255 in ∞ norm.

Table content: <table><tr><td/><td colspan="8">(a) On Fashion-MNIST. The magnitude of perturbations is 0.1 in ∞ norm.</td><td/></tr><tr><td/><td/><td/><td colspan="2">White-Box Attack (%)</td><td/><td/><td colspan="2">Black-Box Attack (%)</td><td/></tr><tr><td colspan="2">Defense Clean (%)</td><td colspan="8">FGSM PGD R+FGSM MIM FGSM PGD R+FGSM MIM</td></tr><tr><td>NT</td><td>90.5</td><td>8.3</td><td>0.1</td><td>15.0</td><td>0.1</td><td>52.1</td><td>52.6</td><td>68.2</td><td>44.3</td></tr><tr><td>SAT</td><td>90.9</td><td>88.8</td><td>7.4</td><td>31.2</td><td>9.4</td><td>79.8</td><td>80.1</td><td>81.8</td><td>80.0</td></tr><tr><td>EAT</td><td>90.8</td><td>89.0</td><td>4.3</td><td>31.6</td><td>6.6</td><td>80.8</td><td>81.4</td><td>82.3</td><td>78.8</td></tr><tr><td>PRT</td><td>76.9</td><td>67.4</td><td>66.8</td><td>72.2</td><td>66.7</td><td>75.5</td><td>75.5</td><td>76.4</td><td>75.4</td></tr><tr><td>ATDA</td><td>85.5</td><td>78.2</td><td>68.6</td><td>77.0</td><td>68.8</td><td>83.8</td><td>83.7</td><td>84.5</td><td>83.3</td></tr><tr><td colspan="2">(b) Defense Clean (%)</td><td colspan="8">White-Box Attack (%) FGSM PGD R+FGSM MIM FGSM PGD R+FGSM MIM Black-Box Attack (%)</td></tr><tr><td>NT</td><td>84.9</td><td>19.6</td><td>3.6</td><td>33.3</td><td>4.6</td><td>64.3</td><td>68.0</td><td>76.5</td><td>64.8</td></tr><tr><td>SAT</td><td>86.6</td><td>52.1</td><td>44.4</td><td>70.4</td><td>46.1</td><td>79.0</td><td>79.7</td><td>83.3</td><td>78.7</td></tr><tr><td>EAT</td><td>88.6</td><td>47.1</td><td>34.4</td><td>67.6</td><td>36.6</td><td>80.4</td><td>81.3</td><td>85.3</td><td>80.2</td></tr><tr><td>PRT</td><td>58.5</td><td>41.7</td><td>41.1</td><td>49.5</td><td>41.2</td><td>53.7</td><td>54.9</td><td>56.1</td><td>54.0</td></tr><tr><td>ATDA</td><td>82.9</td><td>57.2</td><td>53.2</td><td>70.6</td><td>53.9</td><td>75.3</td><td>76.4</td><td>79.5</td><td>75.4</td></tr><tr><td colspan="2">(c) Defense Clean (%)</td><td colspan="8">White-Box Attack (%) FGSM PGD R+FGSM MIM FGSM PGD R+FGSM MIM Black-Box Attack (%)</td></tr><tr><td>NT</td><td>86.9</td><td>4.3</td><td>0.5</td><td>19.6</td><td>0.9</td><td>41.3</td><td>23.8</td><td>60.9</td><td>25.8</td></tr><tr><td>SAT</td><td>86.2</td><td>52.4</td><td>49.5</td><td>70.2</td><td>50.5</td><td>80.5</td><td>80.5</td><td>83.5</td><td>80.3</td></tr><tr><td>EAT</td><td>86.0</td><td>46.7</td><td>43.5</td><td>67.5</td><td>44.5</td><td>80.0</td><td>80.1</td><td>83.2</td><td>79.8</td></tr><tr><td>PRT</td><td>-</td><td>-</td><td>-</td><td>-</td><td>-</td><td>-</td><td>-</td><td>-</td><td>-</td></tr><tr><td>ATDA</td><td>84.8</td><td>60.7</td><td>58.1</td><td>73.2</td><td>59.0</td><td>80.7</td><td>80.7</td><td>83.0</td><td>80.6</td></tr><tr><td colspan="2">(d) Defense Clean (%)</td><td colspan="8">White-Box Attack (%) FGSM PGD R+FGSM MIM FGSM PGD R+FGSM MIM Black-Box Attack (%)</td></tr><tr><td>NT</td><td>59.0</td><td>0.2</td><td>0.4</td><td>6.1</td><td>0.4</td><td>28.8</td><td>23.8</td><td>41.6</td><td>24.2</td></tr><tr><td>SAT</td><td>58.7</td><td>17.7</td><td>18.0</td><td>34.8</td><td>17.9</td><td>53.2</td><td>53.1</td><td>55.8</td><td>53.0</td></tr><tr><td>EAT</td><td>59.1</td><td>12.5</td><td>13.5</td><td>31.1</td><td>13.2</td><td>52.0</td><td>52.2</td><td>55.7</td><td>51.9</td></tr><tr><td>PRT</td><td>-</td><td>-</td><td>-</td><td>-</td><td>-</td><td>-</td><td>-</td><td>-</td><td>-</td></tr><tr><td>ATDA</td><td>61.6</td><td>29.3</td><td>26.2</td><td>43.0</td><td>27.3</td><td>56.0</td><td>56.0</td><td>58.7</td><td>56.0</td></tr></table>

The local loss sensitivity analysis for defense methods.

Table content: <table><tr><td>Dataset</td><td>NT</td><td>SAT</td><td>Local loss sensitivity EAT</td><td>PRT</td><td>ATDA</td></tr><tr><td>Fashion-MNIST</td><td>5.84</td><td>5.52</td><td>3.24</td><td>0.56</td><td>0.49</td></tr><tr><td>SVHN</td><td>13.48</td><td>2.03</td><td>2.41</td><td>1.79</td><td>1.64</td></tr><tr><td>CIFAR-10</td><td>6.56</td><td>1.61</td><td>2.08</td><td>-</td><td>0.90</td></tr><tr><td>CIFAR-100</td><td>23.16</td><td>7.13</td><td>8.40</td><td>-</td><td>2.67</td></tr><tr><td>4.4 ABLATION STUDIES</td><td/><td/><td/><td/><td/></tr></table>

The MMD distance across domains in the logit space for defense methods on Fashion-MNIST. D denotes the distribution of the clean testing data; A F GSM and A P GD denote the distributions of the adversarial testing data generated by the white-box FGSM and PGD, respectively.

Table content: <table><tr><td>MMD Distance</td><td>NT</td><td>SAT</td><td>Defense Method EAT</td><td>PRT</td><td>ATDA</td></tr><tr><td>MMD(D, A F GSM )</td><td>2.174</td><td>5.353</td><td>5.120</td><td>0.098</td><td>0.005</td></tr><tr><td>MMD(D, A P GD )</td><td>5.287</td><td>2.909</td><td>1.239</td><td>0.101</td><td>0.019</td></tr></table>

The accuracy of PAT and PATDA on the testing datasets and the adversarial examples generated by various adversaries. The magnitude of perturbations in ∞ norm is 0.1 for Fashion-MNIST, 0.02 for SVHN, and 4/255 for CIFAR-10 and CIFAR-100.

Table content: <table><tr><td>Dataset</td><td>Defense Clean (%)</td><td/><td colspan="2">White-Box Attack (%)</td><td/><td/><td colspan="2">Black-Box Attack (%)</td><td/></tr><tr><td/><td/><td colspan="8">FGSM PGD R+FGSM MIM FGSM PGD R+FGSM MIM</td></tr><tr><td>Fashion-MNIST</td><td>PAT PATDA 83.2 85.3</td><td>78.7 77.0</td><td>76.5 75.5</td><td>81.7 79.8</td><td>76.7 75.7</td><td>81.8 84.0</td><td>83.9 81.7</td><td>84.8 82.4</td><td>83.8 81.6</td></tr></table>

Peer review (iclr2019):
 The paper casts the problem of learning from adversarial examples to make models resistant to adversarial perturbations to a domain adaptation problem. The proposed method Adversarial training with Domain adapatation( ATDA) learns a representation that is invariant to clean and adversarial data achieving state of the art results on CIFAR. quality - Paper is well written, explanation of the mathematical parts are good, experimental quality can be much better. clarity - the problem motivation as well as the methodology is clearly explained. the learning from the experiments are unclear and need more work. originality - The casting of the problem as domain adaptation is original but from the experiments it was not conclusive as to how much benefit we get. significance of this work - Current models being sensitive to adversarial perturbations is quite a big problem so the particular problem authors are trying to address is very significant. pros A good idea, enough experiments that indicate the benefit of casting this as a domain adaptation problem. cons I feel, the authors should have extended the experiments to ImageNet which is a much larger dataset and validate the findings still hold, I feel the discussion section and comparison to other methods needs to be worked to be more thorough and to tease out the benefit of each of the various terms added to the loss functions as currently all we have is final numbers without much explanation and details. TSNE embeddings part is also very qualitative and while the plots indicate a better separation for ATDA, I feel authors should do more quantitative analysis on the embeddings instead of just qualitative plots.