Year: 2021

Abstract:
The phenomenon of adversarial examples illustrates one of the most basic vulnerabilities of deep neural networks. Among the variety of techniques introduced to surmount this inherent weakness, adversarial training has emerged as the most effective strategy to achieve robustness. Typically, this is achieved by balancing robust and natural objectives. In this work, we aim to further optimize the tradeoff between robust and standard accuracy by enforcing a domain-invariant feature representation. We present a new adversarial training method, Domain Invariant Adversarial Learning (DIAL), which learns a feature representation that is both robust and domain invariant. DIAL uses a variant of Domain Adversarial Neural Network (DANN) on the natural domain and its corresponding adversarial domain. In the case where the source domain consists of natural examples and the target domain is the adversarially perturbed examples, our method learns a feature representation constrained not to discriminate between the natural and adversarial examples, and can therefore achieve a more robust representation. Our experiments indicate that our method improves both robustness and standard accuracy, when compared to other state-of-the-art adversarial training methods.

heading: 1 INTRODUCTION
text: Deep learning models have achieved impressive success on a wide range of challenging tasks. However, their performance was shown to be brittle to adversarial examples: small, imperceptible perturbations in the input that drastically alter the classification (Carlini & Wagner, 2017a;b; Goodfellow et al., 2014; Kurakin et al., 2016b; Moosavi-Dezfooli et al., 2016; Szegedy et al., 2013; Tramèr et al., 2017; Dong et al., 2018; Tabacof & Valle, 2016; Xie et al., 2019b; Rony et al., 2019). Designing reliable robust models has gained significant attention in the arms race against adversarial examples. Adversarial training (Szegedy et al., 2013; Goodfellow et al., 2014; Madry et al., 2017; Zhang et al., 2019b) has been suggested as one of the most effective approaches to defend against such examples, and can be described as solving the following min-max optimization problem:
min θ
E(x,y)∼D [
max x′:‖x′−x‖p≤
` (x′, y; θ) ] ,
where x′ is the -bounded perturbation in the `p norm and ` is the loss function. Different unrestricted attacks methods were also suggested, such as adversarial deformation, rotations, translation and more (Brown et al., 2018; Engstrom et al., 2018; Xiao et al., 2018; Alaifari et al., 2018; Gilmer et al., 2018).
The resulting min-max optimization problem can be hard to solve in general. Nevertheless, in the context of -bounded perturbations, the problem is often tractable in practice. The inner maximization is usually approximated by generating adversarial examples using projected gradient descent (PGD) (Kurakin et al., 2016a; Madry et al., 2017). A PGD adversary starts with randomly initialized perturbation and iteratively adjust the perturbation while projecting it back into the -ball:
xt+1 = ΠB (x0) (xt + α · sign(∇xt`(G(xt), y))),
where x0 is the natural example (with or without random noise), and ΠB (x) is the projection operator onto the -ball, G is the network, and α is the perturbation step size. As was shown by Athalye et al. (2018), PGD-based adversarial training was one of the few defenses that were not broken under strong attacks.
That said, the gap between robust and natural accuracy remains large for many tasks such as CIFAR10 (Krizhevsky et al., 2009) and ImageNet (Deng et al., 2009). Generally speaking, Tsipras et al. (2018) suggested that robustness may be at odds with natural accuracy, and usually the trade-off is inherent. Nevertheless, a growing body of work aimed to improve the standard PGD-based adversarial training introduced by Madry et al. (2017) in various ways such as improved adversarial loss functions and regularization techniques (Kannan et al., 2018; Wang et al., 2019b; Zhang et al., 2019b), semi-supervised approaches(Carmon et al., 2019; Uesato et al., 2019; Zhai et al., 2019), adversarial perturbations on model weights (Wu et al., 2020), utilizing out of distribution data (Lee et al., 2021) and many others. See related work for more details.
Our contribution. In this work, we propose a novel approach to regulating the tradeoff between robustness and natural accuracy. In contrast to the aforementioned works, our method enhances adversarial training by enforcing a feature representation that is invariant across the natural and adversarial domains. We incorporate the idea of Domain-Adversarial Neural Networks (DANN) (Ganin & Lempitsky, 2015; Ganin et al., 2016) directly into the adversarial training process. DANN is a representation learning approach for domain adaptation, designed to ensure that predictions are made based on invariant feature representation that cannot discriminate between source and target domains. Intuitively, the tasks of adversarial training and of domain-invariant representation have a similar goal: given a source (natural) domainX and a the target (adversarial) domainX ′, we hope to achieve g(X) ≈ g(X ′), where g a feature representation function (i.e., neural network). Achieving such a dual representation intuitively yields a more general feature representation.
In a comprehensive battery of experiments on MNIST (LeCun et al., 1998), SVHN (Netzer et al., 2011), CIFAR-10 (Krizhevsky et al., 2009) and CIFAR-100 (Krizhevsky et al., 2009) datasets, we demonstrate that by enforcing domain-invariant representation learning using DANN simultaneously with adversarial training, we gain a significant and consistent improvement in both robustness and natural accuracy compared to other state-of-the-art adversarial training methods, under AutoAttack (Croce & Hein, 2020) and various strong PGD (Madry et al., 2017), CW (Carlini & Wagner, 2017b) adversaries in white-box and black-box settings. Additionally, we evaluate our method using unforeseen "natural" corruptions (Hendrycks & Dietterich, 2018), unforeseen adversaries (e.g., `1, `2), transfer learning, and perform ablation studies. Finally, we offer a novel score function for quantifying the robust-natural accuracy tradeoff.

heading: 2 RELATED WORK

heading: 2.1 DEFENSE METHODS
text: A variety of theoretically principled (Cohen et al., 2019; Raghunathan et al., 2018a; Sinha et al., 2017; Raghunathan et al., 2018b; Wong et al., 2018; Wong & Kolter, 2018; Gowal et al., 2018)
and empirical defense approaches (Bai et al., 2021) were proposed to enhance robustness since the discovery of adversarial examples. Among the empirical defence techniques we can find adversarial regularization (Kurakin et al., 2016a; Madry et al., 2017; Zhang et al., 2019b; Wang et al., 2019b; Kannan et al., 2018), curriculum-based adversarial training (Cai et al., 2018; Zhang et al., 2020; Wang et al., 2019a), ensemble adversarial training (Tramèr et al., 2017; Pang et al., 2019; Yang et al., 2020), adversarial training with adaptive attack budget (Ding et al., 2018; Cheng et al., 2020), semi-supervised and unsupervised adversarial training (Carmon et al., 2019; Uesato et al., 2019; Zhai et al., 2019), robust self/pre-training (Jiang et al., 2020; Chen et al., 2020), efficient adversarial training (Shafahi et al., 2019; Wong et al., 2020; Andriushchenko & Flammarion, 2020; Zhang et al., 2019a), and many other techniques (Zhang & Wang, 2019; Goldblum et al., 2020; Pang et al., 2020b; Lee et al., 2020). In an additional research direction, researchers suggested to add new dedicated building blocks to the network architecture for improved robustness (Xie & Yuille, 2019; Xie et al., 2019a; Liu et al., 2020). Liu et al. (2020) hypothesised that different adversaries belong to different domains, and suggested gated batch normalization which is trained with multiple perturbation types. Others focused on searching robust architectures against adversarial examples (Guo et al., 2020).
Our work belongs to the the family of adversarial regularization techniques, for which we elaborate on common and best performing methods, and highlight the differences compared to our method.
Madry et al. (2017) proposed a technique, commonly referred to as Adversarial Training (AT), to minimize the cross entropy loss on adversarial examples generated by PGD. Zhang et al. (2019b) suggested to decompose the prediction error for adversarial examples as the sum of the natural error and boundary error, and provided a differentiable upper bounds on both terms. Motivated by this decomposition, they suggested a technique called TRADES that uses the Kullback-Leibler (KL) divergence as a regularization term that will push the decision boundary away from the data. Wang et al. (2019b) suggested that misclassified examples have a significant impact on final robustness, and proposed a technique called MART that differentiate between correctly classified and missclassified examples during training.
Another area of research aims at revealing the connection between the loss weight landscape and adversarial training (Prabhu et al., 2019; Yu et al., 2018; Wu et al., 2020). Specifically, Wu et al. (2020) identified a correlation between the flatness of weight loss landscape and robust generalization gap. They proposed the Adversarial Weight Perturbation (AWP) mechanism that is integrated into existing adversarial training methods. More recently, this approach was formalized from a theoretical standpoint by Tsai et al. (2021). However, this method forms a double-perturbation mechanism that perturbs both inputs and weights, which may incur a significant increase in calculation overhead. Nevertheless, we show that DIAL still improves state-of-the-art results when combined with AWP.
A related approach to ours, called ATDA, was presented by Song et al. (2018). They proposed to add several constrains to the loss function in order to enforce domain adaptation: correlation alignment and maximum mean discrepancy (Borgwardt et al., 2006; Sun & Saenko, 2016). While the objective is similar, using ideas from domain adaptation for learning better representation, we address it in two different ways. Our method fundamentally differs from Song et al. (2018) since we do not enforce domain adaptation by adding specific constrains to the loss function. Instead, we let the network learn the domain invariant representation directly during the optimization process, as suggested by Ganin & Lempitsky (2015); Ganin et al. (2016). Moreover, Song et al. (2018) focused mainly of FGSM. We empirically demonstrate the superiority of our method in Section 4. In a concurrent work, Qian et al. (2021) utilized the idea of exploiting local and global data information, and suggested to generate the adversarial examples by attacking an additional domain classifier.

heading: 2.2 ROBUST GENERALIZATION
text: Several works investigated the sample complexity requires the ensure adversarial generalization compared to the non-adversarial counterpart. Schmidt et al. (2018) has shown that there exists a distribution (mixture of Gaussians) where ensuring robust generalization necessarily requires more data than standard learning. This has been furthered investigated in a distribution-free models via the Rademacher Complexity, VC-dimension (Yin et al., 2019; Attias et al., 2019; Khim & Loh, 2018; Awasthi et al., 2020; Cullina et al., 2018; Montasser et al., 2019; Tsai et al., 2021) and additional settings (Diochnos et al., 2018; Carmon et al., 2019).

heading: 3 DOMAIN INVARIANT ADVERSARIAL LEARNING APPROACH
text: In this section, we introduce our Domain Invariant Adversarial Learning (DIAL) approach for adversarial training. The source domain is the natural dataset, and the target domain is generated using an adversarial attack on the natural domain. We aim to learn a model that has low error on the source (natural) task (e.g., classification) while ensuring that the internal representation cannot discriminate between the natural and adversarial domains. In this way, we enforce additional regularization on the feature representation, which enhances the robustness.

heading: 3.1 MODEL ARCHITECTURE AND REGULARIZED LOSS FUNCTION
text: Let us define the notation for our domain invariant robust architecture and loss. Let Gf (·; θf ) be the feature extractor neural network with parameters θf . Let Gy(·; θy) be the label classifier with parameters θy , and letGd(·; θd) the domain classifier with parameters θd. That is,Gy(Gf (·; θf ); θy) is essentially the standard model (e.g., wide residual network (Zagoruyko & Komodakis, 2016)), while in addition, we have a domain classification layer to enforce a domain invariant on the feature representation. An illustration of the architecture is presented in Figure 1.
Given a training set {(xi, yi)}ni=1, the natural loss is defined as:
Lynat = 1n ∑n i=1 CE(Gy(Gf (xi; θf ); θy), yi).
We consider two basic forms of the robust loss. One is the standard cross-entropy (CE) loss between the predicted probabilities and the actual label, which we refer to later as DIALCE. The second is the Kullback-Leibler (KL) divergence between the adversarial and natural model outputs (logits), as in Zhang et al. (2019b); Wang et al. (2019b), which we refer to as DIALKL.
LCErob = 1n ∑n i=1 CE(Gy(Gf (x ′ i; θf ); θy), yi),
LKLrob = 1n ∑n i=1 KL(Gf (x ′ i; θf ) ‖ Gf (xi; θf )),
where {(x′i, yi)}ni=1 are the generated corresponding adversarial examples. Next, we define source domain label di as 0 (for natural examples) and target domain label d ′
i as 1 (for adversarial examples). Then, the natural and adversarial domain losses are defined as:
Ldnat = 1n ∑n i=1 CE(Gd(Gf (xi; θf ); θd), di),
Ldadv = 1n ∑n i=1 CE(Gd(Gf (x ′ i; θf ); θd), d ′ i).
We can now define the full domain invariant robust loss:
DIALCE = Lynat + λLCErob − r(Ldnat + Ldadv),
DIALKL = Lynat + λLKLrob − r(Ldnat + Ldadv).
The goal is to minimize the loss on the natural and adversarial classification while maximizing the loss for the domains. The reversal-ratio hyper-parameter r is inserted into the network layers as a gradient reversal layer (Ganin & Lempitsky, 2015; Ganin et al., 2016) that leaves the input unchanged during forward propagation and reverses the gradient by multiplying it with a negative scalar during the back-propagation. The reversal-ratio parameter is initialized to a small value and is gradually increased to r, as the main objective converges. This enforces a domain-invariant representation as the training progress: a larger value enforces a higher fidelity to the domain. A comprehensive algorithm description can be found in Appendix A.
Modularity and semi-supervised extensions. We note that the domain classifier is a modular component that can be integrated into existing models for further improvements. Moreover, since the domain classifier does not require the class labels, additional unlabeled data can be leveraged in future work for improved results.

heading: 3.2 THE BENEFITS OF INVARIANT REPRESENTATION TO ADVERSARIAL EXAMPLES
text: The motivation behind the proposed method is to enforce an invariant feature representation to adversarial perturbations. Given a natural example x and its adversarial counterpart x′, if the domain classifier manages to distinguish between them, this means that the perturbation has induced a significant difference in the feature representation. We impose an additional loss on the natural and adversarial domains in order to discourage this behavior.
We demonstrate that the feature representation layer does not discriminate between natural and adversarial examples, namely Gf (x; θf ) ≈ Gf (x′; θf ). Figure 2 presents the scaled mean and standard deviation (std) of the absolute differences between the natural examples from test and their corresponding adversarial examples on different features from the feature representation layer. Smaller differences in the mean and std imply a higher domain invariance — and indeed, DIAL achieves near-zero differences almost across the board. Moreover, DIAL’s feature-level invariance almost consistently outperforms the naturally trained model and the model trained using standard adversarial training techniques (Madry et al., 2017). We provide additional features visualizations in Appendix H.

heading: 4 EXPERIMENTS
text: In this section we conduct comprehensive experiments to emphasise the effectiveness of DIAL, including evaluations under white-box and black-box settings, robustness to unforeseen adversaries, robustness to unforeseen corruptions, transfer learning, and ablation studies. Finally, we present a new measurement to test the balance between robustness and natural accuracy, which we named F1-robust score.

heading: 4.1 A CASE STUDY ON SVHN AND CIFAR-100
text: In the first part of our analysis, we conduct a case study experiment on two benchmark datasets: SVHN (Netzer et al., 2011) and CIFAR-100 Krizhevsky et al. (2009). We follow common experiment settings as in Rice et al. (2020); Wu et al. (2020). We used the PreAct ResNet-18 (He et al., 2016) architecture on which we integrate a domain classification layer. The adversarial training is done using 10-step PGD adversary with perturbation size of = 0.031 and a step size of 0.003 for SVHN and 0.007 for CIFAR-100. The batch size is 128, weight decay is 7e−4 and the model is trained for 100 epochs. For SVHN, the initial learinnig rate is set to 0.01 and decays by a factor of 10 after 55, 75 and 90 iteration. For CIFAR-100, the initial learning rate is set to 0.1 and decays by a factor of 10 after 75 and 90 iterations. Results are averaged over 3 restarts while omitting one standard deviation. As can be seen by the results in Tables 1 and 2, DIAL presents consistent improvement in robustness (e.g., 5.75% improved robustness on SVHN against AA) compared to the standard AT while also improving the natural accuracy. More results are presented in Appendix B.

heading: 4.2 BENCHMARKING THE STATE-OF-THE-ART ROBUSTNESS
text: In this part, we evaluate the performance of DIAL compared to other state-of-the-art methods on CIFAR-10. We follow the same experiment setups as in Madry et al. (2017); Wang et al. (2019b);
Zhang et al. (2019b). When experiment settings are not identical between tested methods, we choose the most commonly used settings, and apply it to all experiments. This way, we keep the comparison as fair as possible and avoid reporting changes in results which are caused by inconsistent experiment settings (Pang et al., 2020a). To show that our results are not caused because of what is referred to as obfuscated gradients (Athalye et al., 2018), we evaluate our method with same setup as in our defense model, under strong attacks (e.g., PGD1000) in both white-box, black-box settings, AutoAttack (Croce & Hein, 2020), unforeseen "natural" corruptions (Hendrycks & Dietterich, 2018), and unforeseen adversaries. To make sure that the reported improvements are not caused by adversarial overfitting (Rice et al., 2020), we report best robust results for each method on average of 3 restarts, while omitting one standard deviation. Additional results for CIFAR-10 as well as comprehansive evaluation on MNIST can be found in Appendix D and E.
CIFAR-10 setup. We use the wide residual network (WRN-34-10) (Zagoruyko & Komodakis, 2016) architecture. Sidelong this architecture, we integrate a domain classification layer. To generate the adversarial domain dataset, we use a perturbation size of = 0.031. We apply 10 of inner maximization iterations with perturbation step size of 0.007. Batch size is set to 128, weight decay is set to 7e−4, and the model is trained for 100 epochs. Similar to the other methods, the initial learning rate was set to 0.1, and decays by a factor of 10 at iteration 75 and 90. We also introduce a version of our method that incorporates the AWP double-perturbation mechanism, named DIAL-AWP. For black-box attacks, we used two types of surrogate models (1) surrogate model trained independently without adversarial training, with natural accuracy of 95.61% and (2) surrogate model trained using one of the adversarial training methods. Additional training details can be found in Appendix C.
White-box/Black-box robustness. As reported in Table 3, our method achieves better robustness over the other state-of-the-art methods with respect to the different attacks. Specifically, in white-box settings, we see that our method improves robustness over Madry et al. (2017) by more than 2%, and roughly 2% over TRADES using the common PGD20 attack while keeping higher natural accuracy. We also observe better natural accuracy of 1.65% over MART while also achieving better robustness over all attacks. Moreover, our method presents significant improvement of up to 15% compared to the the domain invariant method suggested by Song et al. (2018) (ATDA). When incorporating AWP, our method improves the TRADES-AWP variant by almost 2% Additional results are available in Appendix E. When tested on black-box settings, DIALCE presents a significant improvement of more than 4.4% over the second-best performing method, and up to 13%. In Table 4, we also present the black-box results when the source model is taken from one of the adversarially trained models. In addition to the improvement in black-box robustness, DIALCE also manages to achieve better clean accuracy of more than 4.5% over the second-best performing method. Moreover, based on the
auto-attack leader-board 1, our method achieves the 1st place among models without additional data using the WRN-34-10 architecture.

heading: 4.2.1 ROBUSTNESS TO UNFORESEEN ATTACKS AND CORRUPTIONS
text: Unforeseen Adversaries. To further demonstrate the effectiveness of our approach, we test our method against various adversaries that were not used during the training process. We attack the model under the white-box settings with `2-PGD, `1-PGD, `∞-DeepFool and `2-DeepFool (Moosavi-Dezfooli et al., 2016) adversaries using Foolbox (Rauber et al., 2017). We applied commonly used attack budget (perturbation for PGD adversaries and overshot for DeepFool adversaries) with 20 iterations for the PGD adversaries and 50 for the DeepFool adversaries. Results are presented in Table 5. As can be seen by the results, our approach gains an improvement of up to 4.73% over the second best method under the various attack types and average improvement of 3.7% over all threat models.
Unforeseen Corruptions. We further demonstrate that our method consistently holds against unforeseen "natural" corruptions, consists of 18 unforeseen diverse corruption types proposed by Hendrycks & Dietterich (2018) on CIFAR-10, which we refer to as CIFAR10-C. The CIFAR10-C benchmark covers noise, blur, weather, and digital categories. As can be shown in Figure 3, our method gains a significant and consistent improvement over all the other methods. Our approach leads to an average improvement of 4.7% with minimum improvement of 3.5% and maximum improvement of 5.9% compared to the second best method over all unforeseen attacks. See Appendix F for the full experiment results.

heading: 4.2.2 TRANSFER LEARNING
text: Recent works (Salman et al., 2020; Utrera et al., 2020) suggested that robust models transfer better on standard downstream classification tasks. In Table 6 we demonstrate the advantage of our method when applied for transfer learning across CIFAR10 and CIFAR100 using the common linear evaluation protocol. see Appendix G for detailed experiment settings.

heading: 4.2.3 ABLATION STUDIES
text: In this part, we conduct ablation studies to further investigate the contribution of the additional domain head component introduced in our method. Experiment configuration are as in 4.2, and robust accuracy is reported on white-box PGD20. We use the CIFAR-10 dataset and train WRN-3410. We remove the domain head from both DIALKL and DIALCE (equivalent to r = 0) and report the natural and robust accuracy. We perform 3 random restarts and omit one standard deviation from the results. Results are presented in Figure 4. Both DIAL variants exhibits stable improvements on both natural accuracy and robust accuracy. DIALCE and DIALKL present an improvement of 1.82% and 0.33% on natural accuracy and 2.5% and 1.87% on robust accuracy, respectively.

heading: 4.2.4 VISUALIZING DIAL
text: To further illustrate our method, we visualize the model outputs using the different methods under natural test data and adversarial test data generated using PGD20 white-box attack with step size 0.003 and = 0.031 on CIFAR-10. Figure 5 shows the embedding received after applying t-SNE (Van der Maaten & Hinton, 2008) with two components on the model output for our method and for TRADES. DIAL seems to preserve strong separation between classes on both natural test data and adversarial test data. Additional illustrations for the other methods are attached in Appendix H.

heading: 4.3 BALANCED MEASUREMENT FOR ROBUST AND NATURAL ACCURACY
text: One of the goals of our method is to better balance between robust and natural accuracy under a given model. For a balanced metric, we adopt the idea of F1-score, which is the harmonic mean between the precision and recall. However, rather than using precision and recall, we measure the F1-score between robustness and natural accuracy, using a measure we call the F1-robust score.
F1-robust = true_robust
true_robust + 12 (false_robust + false_natural) ,
where true_robust are the adversarial examples that were correctly classified, false_robust are the adversarial examples that where miss-classified, and false_natural are the natural examples that were miss-classified. We tested the proposed F1-robust score using PGD20 on CIFAR-10 dataset in whitebox and black-box settings. Results are presented in Table 7 and show that our method achieves the best F1-robust score in both settings, which supports our findings from previous sections.

heading: 5 CONCLUSION
text: In this paper, we investigated the hypothesis that domain invariant representation can be beneficial for robust learning. With this idea in mind, we proposed a new adversarial learning method, called Domain Invariant Adversarial Learning (DIAL) that incorporates Domain Adversarial Neural Network into the adversarial training process. The proposed method is generic and can be combined with any network architecture in a wide range of tasks. Our evaluation process included strong adversaries , unforeseen adversaries , unforeseen corruptions, transfer learning tasks, and ablation studies. Using the extensive empirical analysis, we demonstrate the significant and consistent improvement obtained by DIAL in both robustness and natural accuracy compared to other defence methods on benchmark datasets.
ETHICS STATEMENT
We proposed DIAL to improve models’ robustness against adversarial attacks. We hope that it will help in building more secure models for real-world applications. DIAL is comparable to the stateof-the-art methods we tested in terms of training times and other resources. That said, this work is not without limitations: adversarial training is still a computationally expensive procedure that requires extra computations compared to standard training, with the concomitant environmental costs. Even though our method introduced better standard accuracy, adversarial training still degrades the standard accuracy. Moreover, models are trained to be robust using well known threat models such as the bounded `p norms. However, once a model is deployed, we cannot control the type of attacks it faces from sophisticated adversaries. Thus, the general problem is still very far from being fully solved.
REPRODUCIBILITY STATEMENT
In this paper, great efforts were made to ensure that comparison is fair, and all necessary information for reproducibility is present. Section 4 and Appendix B and C contains all experiment settings for SVHN, CIFAR-10 and CIFAR-100 experiments. Appendix D contains all experiment details and results for MNIST experiments. Appendix G contains experiment settings for the transfer learning experiment. In the supplementary material, we provided the source code to train and evaluate DIAL.

heading: A DOMAIN INVARIANT ADVERSARIAL LEARNING ALGORITHM
text: Algorithm 1 describes a pseudo-code of our proposed DIALCE variant. As can be seen, a target domain batch is not given in advance as with standard domain-adaptation task. Instead, for each natural batch we generate a target batch using adversarial training. The loss function is composed of natural and adversarial losses with respect to the main task (e.g., classification), and from natural and adversarial domain losses. By maximizing the losses on the domain we aim to learn a feature representation which is invariant to the natural and adversarial domain, and therefore more robust.
Algorithm 1: Domain Invariant Adversarial Learning Input: Source data S = {(xi, yi)}ni=1 and network architecture Gf , Gy, Gd Parameters: Batch size m, perturbation size , pgd attack step size τ , adversarial trade-off λ,
initial reversal ratio r, and step size α Initialize: Y0 and Y1 source and target domain vectors filled with 0 and 1 respectively Output: Robust network G = (Gf , Gy, Gd) parameterized by θ̂ = (θf , θy, θd) respectively

heading: B ADDITIONAL RESULTS ON CIFAR-100 AND SVHN
text: White-box Black-Box
Defense Model Natural PGD20 PGD100 PGD1000 CW∞ PGD20 PGD100 PGD1000 CW∞ AA
TRADES 90.35 57.10 54.13 54.08 52.19 86.89 86.73 86.57 86.70 49.5 DIALKL (Ours) 90.66 58.91 55.30 55.11 53.67 87.62 87.52 87.41 87.63 51.00 DIALCE (Ours) 92.88 55.26 50.82 50.54 49.66 89.12 89.01 88.74 89.10 46.52

heading: C CIFAR-10 ADDITIONAL EXPERIMENTAL SETUP DETAILS
text: Additional defence setup. For being consistent with other methods, the natural images are padded with 4-pixel padding with 32-random crop and random horizontal flip. Furthermore, all methods are trained using SGD with momentum 0.9. For DIALKL, we balance the robust loss with λ = 6 and the domains loss with r = 4. For DIALCE, we balance the robust loss with λ = 1 and the domains loss with r = 2. For DIAL-AWP, we used the same learning rate schedule used in Wu et al. (2020), where the initial 0.1 learning rate decays by a factor of 10 after 100 and 150 iterations.

heading: D BENCHMARKING THE STATE-OF-THE-ART ON MNIST
text: Defence setup. We use the same CNN architecture as used in Zhang et al. (2019b) which consists of four convolutional layers and three fully-connected layers. Sidelong this architecture, we integrate a domain classification layer. To generate the adversarial domain dataset, we use a perturbation size of = 0.3. We apply 40 iterations of inner maximization with perturbation step size of 0.01. Batch size is set to 128 and the model is trained for 100 epochs. Similar to the other methods, the initial learning rate was set to 0.01, and decays by a factor of 10 after 55 iterations, 75 and 90 iterations. All the models in the experiment are trained using SGD with momentum 0.9. For our method, we balance the robust loss with λ = 6 and the domains loss with r = 0.1.
White-box/Black-box robustness. We evaluate all defense models using PGD40, PGD100, PGD1000 and CW∞ (`∞ version of Carlini & Wagner (2017b) attack optimized by PGD-100) with step size 0.01. We constrain all attacks by the same perturbation = 0.3. For our black-box setting, we use a naturally trained surrogate model with natural accuracy of 99.51%. As reported in Table 10, our method achieves improved robustness over the other methods under the different attack types, while preserving the same level of natural accuracy, and even surpassing the naturally trained model. We should note that in general, the improvement margin on MNIST is more moderate compared to CIFAR-10, since MNIST is an easier task than CIFAR-10 and the robustness range is already high to begin with. Additional results are available in Appendix E.

heading: E ADDITIONAL RESULTS ON MNIST AND CIFAR-10
text: In Table 11 we present additional results using the PGD1000 threat model. We use step size of 0.003 and constrain the attacks by the same perturbation = 0.031. Table 12 presents a comparison of our method combined with AWP to other the variants of AWP that were presented in Wu et al. (2020). In addition, in Table 13 we add the F1-robust scores for different variants of AWP.

heading: F EXTENED RESULTS ON UNFORESEEN CORRUPTIONS
text: We present full accuracy results against unforeseen corruptions in Tables 14 and 15. We also visualize it in Figure 6.
Table 14: Accuracy (%) against unforeseen corruptions.
Defense Model brightness defocus blur fog glass blur jpeg compression motion blur saturate snow speckle noise
TRADES 82.63 80.04 60.19 78.00 82.81 76.49 81.53 80.68 80.14 MART 80.76 78.62 56.78 76.60 81.26 74.58 80.74 78.22 79.42 AT 83.30 80.42 60.22 77.90 82.73 76.64 82.31 80.37 80.74 ATDA 72.67 69.36 45.52 64.88 73.22 63.47 72.07 68.76 72.27 DIAL (Ours) 87.14 84.84 66.08 81.82 87.07 81.20 86.45 84.18 84.94
Table 15: Accuracy (%) against unforeseen corruptions.

heading: G TRANSFER LEARNING SETTINGS
text: The models used are the same models from previous experiments. We follow the common procedure of “fixed-feature” setting, where only a linear layer on top of the pre-trained network is trained. We train a linear classifier on CIFAR-100 on top of the pre-trained network which was trained on CIFAR-10. We also train a linear classifier on CIFAR-10 on top of the pre-trained nwork which was trained on CIFAR-100. We train the linear classifier for 100 epochs, and an initial learning rate of 0.1 which is decayed by a factor of 10 at epochs 50 and 75. We used SGD optimizer with momentum 0.9.

heading: H EXTENDED VISUALIZATIONS
text: In Figure 8, we provide additional visualizations of the different adversarial training methods presented above. We visualize the models outputs using t-SNE with two components on the natural test data and the corresponding adversarial test data generated using PGD20 white-box attack with step size 0.003 and = 0.031 on CIFAR-10.
In Figure 7 we visualize statistical differences between natural and adversarial examples in the feature representation layer. Specifically, we show the differences in mean and std on thirty random feature values from the feature representation layer as we pass through a network the natural test examples and their corresponding adversarial examples. We present the results on same network architecture (WRN-34-10), trained using three different training procedures: naturally trained network, network trained using standard adversarial training (AT) Madry et al. (2017), and DIAL on the CIFAR-10 dataset. When the statistical characteristics of each feature differ from each other, it implies that the features layer is less domain invariant. That is, smaller differences in mean/std yields a better invariance to adversarial examples. One can observe that for DIAL, there is almost no differences between the mean/std of natural examples and their corresponding adversarial examples. Moreover, for the vast majority of the features, DIAL present smaller differences compared to the naturally trained model and the model trained with standard adversarial training. Best viewed in colors.

Review summaries:
The paper describes an adversarial training approach that, in addition to the commonly used robustness loss, requires the network to extract similar representation distributions for clean and attacked data. The proposed method is inspired by domain adaptation approaches that require a model to extract domain invariant/agnostic features from two domains. In the context of this paper, the two domains are the clean and adversarially perturbed images, and the network is required to extract domain invariant representation. To achieve domain invariance, the authors propose a domain classifier ( i.e., an adversarial network) that discriminates the representations from clean and attacked images. The feature extractor is then required to generate features that fool the domain classifier. The authors then provide extensive experiments on small-scale benchmark datasets (SVHN, CIFAR10, CIFAR 100, and MNIST in the supplementary material ) to show the robustness of their proposed approach against the state-of-the-art robustness methods under white-box and black-box attacks. The authors show that their proposed method provides: 1) higher accuracy on attacked data (more robustness), and 2) higher accuracy on clean data, closing the gap between the performance on clean and attacked data. In addition, the paper provides insightful experiments on robustness to unforeseen adversaries, robustness to unforeseen corruptions, transfer learning, and ablation studies.  

This paper proposes a domain invariant adversarial training (DIAL) method, which learns the feature representation that is both robust and domain invariant.  Apart from the label classifier, the model is equipped with a domain classifier that constrains the model not to discriminate between natural examples and adversarial examples, thus achieving a more robust feature representation. Extensive experiments on image classification benchmark the robustness compared to other state-of-the-art methods.