Domain Invariant Adversarial Learning
Authors: Matan Levi
Idan Attias
Aryeh Kontorovich
Venue: Accepted by TMLR ()

Abstract: The phenomenon of adversarial examples illustrates one of the most basic vulnerabilities of deep neural networks. Among the variety of techniques introduced to surmount this inherent weakness, adversarial training has emerged as the most effective strategy for learning robust models. Typically, this is achieved by balancing robust and natural objectives. In this work, we aim to further optimize the trade-off between robust and standard accuracy by enforcing a domain-invariant feature representation. We present a new adversarial training method, Domain Invariant Adversarial Learning (DIAL), which learns a feature representation that is both robust and domain invariant. DIAL uses a variant of Domain Adversarial Neural Network (DANN) on the natural domain and its corresponding adversarial domain. In the case where the source domain consists of natural examples and the target domain is the adversarially perturbed examples, our method learns a feature representation constrained not to discriminate between the natural and adversarial examples, and can therefore achieve a more robust representation. DIAL is a generic and modular technique that can be easily incorporated into any adversarial training method. Our experiments indicate that incorporating DIAL in the adversarial training process improves both robustness and standard accuracy.

Published in Transactions on Machine Learning Research (09/2022)
Domain Invariant Adversarial Learning
Matan Levi
mulanle@@post.bguco
Deportment of Computer Science Ben- Ben-Gurion University of the Negev
Idan Attias
ponuuttt@@pos.bbg..o.o
Department Computer Science Ben- Gurion University of the Negev
Aryeh Kontorovich
karyeh Shgg.h@@bg.acc ac.il
Deportment of Computer Science
Ben- Cur-Gurion University of the Negev
Reviewed OpenReview: https: openrevveee. net/ forum? id= U8w.AAUMz
Abstract
The phenomenon of adversarial examples illustrates one of the most basic vulnerabilities of deep neural networks. Among the variety techniques introduced surmount this inherent weakness, adversarial training has emerged the most effective strategy for learning robust models. Typically, this is achieved by balaneing robust and natural objectives. In this work, aim to further optimize the trade-off between robust and standard accuracy by enforcing domain-invariand feature representation. We present new adversarial training method, Domain Invariant Adversarial Learning (DIAL), which learns feature representation that is both robust and domain invariant. DIAL uses a variant of Domain Adversarial Neural Network (DANN) on the natural domain and its corresponding adversarial domain. Tn the case where the source domain consists of natural examples and the target domain is the adversarially perturbed examples, our method learns a feature representation constrained not to discriminate between the natural and adversarial examples, and can therefore achieve a more robust representation. DIAL is generic and modular technique that can be casily incorporated into any adversarial training method. Our experiments indicate that incorporating DIAL in the adversarial training process improves both robustness and standard accuracy.
1 Introduction
Deep learning models have achieved impressive success on a wide range of challenging tasks. However, their performance was shown to brittle the face of adversarial ezamples: small, imperceptible perturbations
in the input that drastically alter the classification (Carlini & Wagner 2017a b Goodfellow et al.) 2014
Kurakin et al.) 2016b Moosavi-Dezfooli et al. 2016 Szegedy et al. 2013 Tramer et al. 2017 Dong al,
2018 Tabacof & Valle 2016 Xic et al. 2019b: Rony et al.] 2019). The problem of designing reliable robust
models has gained significant allention in the arms race against adversarial examples. Adversarial training
(Szegedy et al. 2013 Goodfellow et al. 2014 Madry et al. 2017 Zhang et al. 2019b) has been proposed as one of the most effective approaches to defend against such examples, and can be described as solving the
following min-max optimization problem:
ming B(Z [max (x', vy: 0]0
Our code is available http::///ggtthbb.coo/a/aaaaaaaaa

---

Published in Transactions on Machine Learning Research (09/2022)
where x' is the e-bounded perturbation in the Ep norm and L is the loss function. Different unrestricted attacks methods were also suggested, such as adversarial deformation, rotations, translation and more (Brown
et al. 2018 Engstrom et al. 2018 Xiao et al.) 2018 Alaifari et al. 2018 Gilmer et al. 2018
The resulting min-max optimization problem can be hard to solve in general. Nevertheless, in the context of e-bounded perturbations, the problem often tractable in practice. The inner maximization is usually approximated by generating adversarial examples using projected gradient descent (PGD) (Kurakin al,, 2016a; Madry et al.) 2017). A PGD adversary starts with randomly initialized perturbation and iteratively adjust the perturbation while projecting it back into the e-ball:
1++1 = (x0) (ote + a
where is the natural example (with or without random noise), and (x) is the projection operator onto the e-ball, G is the network, and a is the perturbation step size. As was shown by |Ataaly et al. (2018), PGD-based adversarial training was one of the few defenses that were not broken under strong attacks.
That said, the gap between robust and natural accuracy remains large for many tasks such as CIFAR-10 (Krizhevsky et al. 2009) and ImageNet (Deng al. 2009). Generally speaking, Tsipras et al. (2018) suggested that robustness may be at odds with natural accuracy, and usually the trade-off is inherent. Nevertheless,
growing body of work aimed to improve the standard PGD-based adversarial training introduced by Madry
et al.
(017) in various ways such as improved adversarial loss functions and regularization techniques (Kannan
ct al.
2018
Wang et al. 2019b Zhang et al. 2019b), semi-supervised
Prrooachss(Carmann et al. 2019 Uesato
et al.
2019
Zhai et al.j 2019) adversarial perturbations on model weights (Wu et al.) 2020), utilizing out of
distribution data (Lee 2021 and many others. We refer related work for more extensive literature
review.
Our contribution. In this work, we propose a novel approach to regulating the tradeoff between robustness and natural accuracy. In contrast to the aforementioned works, our method enhances adversarial training by enforcing a feature representation that is invariant across the natural and adversarial domains. We incorporate the idea of Domain-Adversaria Neural Networks (DANN) (Ganin & Lempitsky 2015 Ganin
et al. 2016) directly into the adversarial training process. DANN is a representation learning approach for domain adaptation, designed to ensure that predictions are made based on invariant feature representation that cannot discriminate between source and target domains. This tochnique is modular and can be easily incorporated into any standard adversarial training algorithm. Intuitively, the tasks of adversarial training and of domain-invariant representation have similar goal: given source (natural) domain X and a target (adversarial) domain X', we hope to achieve g ((X) x g(X',, where g is a feature representation function (i.e., neural network). As we present in section 3.3 our work is also theoretically motivated by the domain adaptation generalization bounds.
In a comprehensive battery of experiments on MNIST (LeCun et al. 1998), SVHN (Netzer al. 2011 CIFAR-10 (Krizhevsky et al. 2009) and CIFAR-100 Krizhevsky al. 2009 datasets, we demonstrate that by enforcing domain-invariant representation learning using DANN simultaneously with adversarial training, we gain a significant and consistent improvement in both robustness and natural accuracy compared to other state-of-the-arl adversarial training methods, under Auto-Attack (Croce & Hein 2020) and various strong PGD (Madry et al. 2017), and CW (Carlini & Wagner 2017b) adversaries white box and black- box settings. Additionally, we evaluate our method using unforeseen "natural" corruptions (Hendrycks & Dictterich] [2018), unforeseen adversarics (e.g., L,, E2), transfer learning, and perform ablation studies. Finally, we offer a novel score function for quantifying the robust-natural accuracy trade-off.
2

---

Published in Transactions on Machine Learning Research (09/2022)
Natunil Inpur
Fasleze Eximelor
Label Clasiller
Examples
Generation
Advensrially Perturhed Ixpui
Gradient Layer
Demnin
Chestfeer
Figure Illustration of the proposed architecture enforce domain invariant representation, The feature extractor and label classifier form the regular DNN architecture that can be used for the main natural task. The domain classifier is incorporated alongside the label classifier. The reversal gradient layer multiplies the gradient by negative number during the back-propaagtion.
2 Related work
2.1
Defense methods
A variety of Uheoretically principled (Raghunathan et al.] 2018a; Sinha al. 2017 Raghunathan al.
2018b Wong al., 2018 Wong & Kolter 2018 Gowal et al. 2018 and empirical defense approaches (Bai
et al. 2021 were proposed to enhance robustness since the discovery of adversarial examples. Theoretically principled methods focus on certifying robustness to adversarial perturbations under given norm, using variety of techniques such as randomized smoothing (Cohen et al. 2019). However, empirical defence methods in general, and in particular adversarial training, still yield preferable results.
Among the empirical defence techniques, we can find: adversarial regularizaiinn- adding various regularization terms to the loss functions to enhance robustness (e.g., encouraging logits for clean and adversarial examples
to be similar) (Kurakin et al. 2016a Madry et al. 2017 Zhang et al. 2019b Wang et al. 2019b Kannan et al.
2018 Jin et al. 2022), curriculum-iusel adversarial truining taking incremental approach when learning
PGD adversaries in the goal of improving generalization on clean data while still preserving robustness (e.g.. gradually increasing the number of PGD itcrations to avoid overfitting the adversarial examples) (Cai et al.
2018 Zhang et al. 2020 Wang et al., 2019a), ensemble adversarial training where clean data is augmented with adversarial examples generated from different target models instead of single model (Tramèr al, 2020), adversorial training with adaptive attack budget where we change the perturbation budget to prevent over-confident predictions and achieve better exploration of the manifold
2017 Pang et al.
2019
Yang al.
2020 semi-superriss. and unsupervised adversurial training several methods theoretically and empirically demonstrated how unlabeled data can reduce the sample complexity gap between standard training and adversarial training (Carmon et al.]
Ding et al. 2018: Cheng et al.
2019 Uesato et al. 2019 Zhai et al. 2019)
robust self and pre- tet-ttraning--o other works integrate self-super vised pretraining tasks such as Selfie, Rotation
and Jigsaw together with adversarial examples (Jiang et al.) 2020 Chen et al., 2020). efficient adversarial
training due to the high cost of adversarial training, eflicient methods aim to keep the favorable performance of adversarial training while reducing the computational and time costs (Shafahi et al. 2019 Wong et al. 2020:
Andriushcheuko & Flammarion 2020 Zhang et al. 2019a),, and many other techniques such adversarial training based on feature scatter (Zhang & Wang 2019). adversarially robust distillation (Goldblum et al. 2020), hypersphere embedding (Pang et. al.] (020b), and augmenting adversarial examples by interpolation Lee et al. 2020). In an additional research direction, researchers suggested to add new dedicated building blocks to the network architecture for improved robustness (Xic & Yuille 2019 Xie et al. 2019aa Liu et al.
2020). Liu et al. (2020) hypothesised that different adversaries belong to different domains, and suggested gated batch normalization which trained with multiple perturbation types. Guo et al. (2020) focused
3

---

Published in Transactions on Machine Learning Research (09/2022)
searching robust architectures against adversarial examples. Others works presented improved robustness by combining data augmentation techniques and generated data. (Rebuffi et al. 2021a where the latest is also the current state-of-the-arr in robustness.
Our work belongs to the the family of adversarial regularization techniques, for which we elaborate on common and best performing methods, and highlight the differences compared to our method.
Madry al. (2017) proposed a technique, commonly referred to as Adversarial Training (AT), to minimize the cross entropy loss on adversarial examples generated by PGD (without using the natural examples). Zhang et al.[ (2019b) suggested to decompose the prediction error for adversarial examples as the sum of the natural error and boundary error, and provided differentiable upper bounds on both terms. Motivated by this decomposition, they suggested a technique called TRADES that uses the Kullback-Leibler (KL) divergence as a regularization term that will push the decision boundary away from the data. They do so by applying the KI.-divergence on the logits of clean examples and their adversarial counterparts. [Wang et al. (2019b) suggested that misclassified examples have siguificant impact on final robustness, and proposed a technique called MART that differentiate between correctly classified and classifiedd examples during training by weighting the KL-divergence between the clean and adversarial logits using the probability of the classifier on the correct label.
Another area of research aims at revealing the connection between the loss weight, landscape and adversarial training (Pabbhu et al. 2019 Yu et al.) 2018: Wu et al. 2020). Specifically, Wu et al. (2020) identified a correlation between the flatness of weight loss landscape and robust generalization gap. They proposed the Adversarial Weight. Perturbation (AWP) mechanism that is integrated into existing adversarial training methods and generates adversarial perturbations on both the inputs and the network weights. More recently, this approach was formalized from a theoretical standpoint by Tsai et al. (2021). However, this method forms a double-perturbation mechanism that perturbs both inputs and weights, which may incur a significant increase in calculation overhead. We demonstrate how DIAL improves results also when combined with AWP, named DIALAWP. In Section 3.4 we elaborate about the loss functions of the different compared methods.
A related approach to ours, called ATDA, was presented by [Song et al. (2018). They proposed to add several constrains to the loss function in order to enforce domain adaptation: correlation alignment and maximum mean discrepancy (Borgwardt et al. 2006 Sun & Saenko 2016). While the objective is similar, using ideas from domain adaptation for learning better representation, we address it in two different ways. Our method fundamentally differs from Song et al. (2018) since we do not enforce domain adaptation by adding specific constrains to the loss function. Instead, we let the network learn the domain invariant representation
directly during the optimization process, as suggested by Ganin & Lempitsky (2015); Ganin el al. (2016). Morcover, Song et al. (2018) focusod mainly of Fast Gradient Sign Method (FGSM) attack, which is a one step variant of PGD attack. We empirically demonstrate the superiority of our method in Section 4 In a concurrent work, Qian et al. (2021) utilized the idea of exploiting local and global data information, and suggested to generate the ad adversarial examples by attacking an additional domain classifier.
2.2 Theoretical analysis of robust generalization
Several works investigated the sample complexity requires the ensure adversarial generalization compared to the non-adversarial counterpart. [Schmidt et al. (2018) has shown that there exists a distribution (mixture of Gaussians) where ensuring robust. generalixation necessarily requires more data than standard learning. This has been furthered investigated in a distribution-fee free models via the Rademacher complexity, VC dimension,
and fat-shattering dimension (Yin et al.
2019 Attias ct al. 2019 Khim & Loh 2018 Awasthi et al. 2020
Cullina el al., 2018 Montasser el al. 2019
Tsai el al.] [2021 Attias el al. 2022 Allias & Hanneke] 2022 and
additional settings DDochnoss et al. 2018
Carmon et al.) 2019
3 Domain Invariant Adversarial Learning approach
In this section, we introduce our Domain Invariant Adversarial Learning (DIAL) approach for adversarial training. The source domain is the natural dataset, and the target domain is generated adversarial using attack on the natural domain. We aim to learn model that has low error on the source (natural) task (e.g.,
4

---

Published in Transactions on Machine Learning Research (09/2022)
classification) while ensuring that the internal representation camnot discriminate between the natural and adversarial domains. In this way, we enforce additional regularization on the feature representation, which enhances robustness.
3.1 The benefits of invariant representation to adversarial examples
The motivation behind the proposed method is to enforce an invariant feature representation to adversarial perturbations. Given a natural example I and its adversarial counterpart x', if the domain classifier manages to distinguish between them, this means that the perturbation has induced a significant difference in the feature representation. We impose an additional loss on the natural and adversarial domains in order discourage this behavior.
We demonstrate that the feature representation layer does not discriminate between natural and adversarial examples, namely (y( Cy(x';0,), Figure presents the scaled mean and standard deviation (std) of the absolute differences between the natural examples from test and their corresponding adversarial examples on different features from the feature representation layer. Smaller differences in the mean and std imply a higher domain invariance and indeed, DIAL achieves near-zero differences almost across the board. Morcover, DIAL's fcaturo-level invariance almost consistently outperforms the naturally trained model (model trained without adversarial training), and the model trained using standard adversarial training techniques (Madry al. 2017). We provide additional features visualizations in Appendix H
Recently, other communities also discovered the benefits of adopting analogous architectures to DANN, such as the contrastive learning community which used similar architecture to improve representation learning
Dangovski et al. 2021 Wang et al. 2021
DIAL difference
difference
Clean mean difference
24
Clean std difference
Stendard AT meen difference
Standerd sld difference
0.4
0.3
3.0
0
(a) Mean difference comparison
(b) Standard deviation difference comparison
Figure 2: We visualize the (a) Mean and (b) standard deviation (std) differences comparison between three models: (1) Naturally trained model (without adversarial training), named Clean. (2) Model trained using standard adversarial training, named standard AT, and (3) Model trained using method, DIAL. We visualize five random features from the features layer. Each bar represent the difference between the means/ste of the natural examples and the mean/std of their corresponding adversarial examples on this same feature.
3.2 Model architecture and regularized loss function
Let us define the notation for our domain invariant robust architecture and loss. Let Gy(:0y) the feature extractor neural network with parameters 0y. Let Gy(:;6,) be the label classifier with parameters Oy. and let Ga(: 02) be the domain classifier with parameters 0g. That is, (G)((:))))) is essentially the standard model (e.g., wide residual network (Zagoruyko & Komodakis 2016)), while in addition, we have domain classification layer to enforce domain invariant on the feature representation. An illustration of the architecture is presented in Figure !
Given a training set ((x,,))) the natural loss is defined as:
c = + En
We consider two basic forms of the robust loss. One is the standard cross-entropy (CE) loss between the predicted probabilities and the actual label, which we refer to later DIALCE. The second is the
5

---

Published in Transactions on Machine Learning Research (09/2022)
Kullback-Leible (KL) divergence between the adversarial and natural model outputs (logits), i.., the class probabilities between the natural examples and their adversarial counterparts, which we refer to DIAL&L.
9ob = C 4 = = - KL(G,(Gy((d:))).... (m)
where {22, 34) H= are the generated corresponding adversarial examples. Next, we define source domain label di as 0 (for natural examples) and target domain label d, as (for adversarial examples). Then, the natural and adversarial domain losses are defined as:
L Σ ())
Cd - 1 n - ()
We can now define the full domain invariant robust loss:
DIAL.C = fiat + AL r r( + fldv);
DIALKI. CM + ACKI r r(Chat + fadv).
The goal is to mánémiiz the loss the natural and adversarial classification while mazimizzón the loss for the domains. The reversal-ratio sapee-parameter is inserted into the network layers gradient reversal layer (Ganin & Lempitsky 2015 Ganin et al. 2016) that leaves the input unchanged during forward propagation and reverses the gradient by multiplying it with a negative scalar during the back-propagation. The reversal-ratio parameter is initialized to a small value and is gradually increased to r, as the main objective converges. This enforces a domsin-invariant representation as the training progress: a larger value enforces a higher fidelity the domain. A comprehensive algorithm description can be found in Appendix
3.3 Theoretical Analysis through the Lens Generalization Bounds
Our method refers to the natural and adversarial examples two distinct domains, where the source domain consists of natural examples and the target domain is the adversarially perturbed examples. Given this assumption, we can adapt the generalization bounds from Mansour et al. (2009) to theoretically justify our approach.
Preliminarics. Let H be a set of functions mapping X to Y and let Y x R4 be a loss function over Y. The Lo(S.g) g) loss functional is defined as the expected loss for any two functions S.g X - in H, and any distribution Q over X:
CQls.9) = = (...)(())))((((()))(
We also denote fQ X + as the target function on examples drawn from Q. That is, the error of a function
h E H is defined as (h, fq) = (L-o)(((((=)
We define the discrepancy distance disc between two distributions Q1 and Q2 over X by:
discy (Qs, maxi,N'H (h, h') - (h,
Reduction to domain adaptation. Given source (natural) distribution D, we define the target (adver- sarial) distribution Dadv, such that every pair (I, y) in the support of D is mapped to (2, v), where 2 (x) and Dadv(2, y) = D(x, y).
Let hD = arg min (n(h, be the optimal natural function, and let hi = arg argminken L (h, J' the optimal robust function. Additionally, we assume that the labeling function fDm. is the same as fD-
Under these assumptions, we reduce our problem to a domain adaptation one, which consists of selecting a hypothesis h H with small expected loss according to the target distribution.

---

Published in Transactions on Machine Learning Research (09/2022)
We now adapt Theorem 8 in (Mansour et al. 2009 to case and bound the adversarial loss by:
adv
natural
D (h, fDadr) ) I Darr (ha fDaw)) ) ) CD hi)) + LD Dawe + disci (Dady
(1)
trade-off
discrepancy
The bound on the adversarial loss consists of four terms: first term approximation error of H for adversarial distribution. second term estimation error on natural examples for the output function on the true distribution (compared to the optimal h E H). third term - trade-off error (depends on H, D, Dodv: and not the algorithm output) that represents the fraction of points for which the optimal H on adversarial examples are wrong, compared the optimal Hl on clean examples. forth term the discrepancy distance between D and Dado-
Therefore, the adversarial loss bounded in equation | depends, in addition to the natural and ad advessaiial losses, on the discrepancy distance, discz, between the two distributions. The discrepancy distance will decrease as the two distributions D and Dadv will be similar to one another.
Our adversarial training method, DIAL, minimizes the adversarial and natural losses, and simultaneously learns an invariant feature representation between the adversarial and natural domains. This representation ensures that adversarial examples and their natural counterparts will be invariant to the domain they were taken from, and therefore makes the two distributions D and Dadv similar to each other. Altogether, DIAL also minimizes the discrepancy distance, which in turn leads to minimizing the upper bound on the adversarial error.
3.4 Related work loss function comparison
In Table further illustrate the loss functions of the different methods. For TRADESAWP, the process involves rumning TRADES loss function, and then running weight perturbation as defined in equation (10) in (Wu et al. 2020). These loss functions can be compared to our proposed loss functions DIALCE and DIALKI which are detailed in section 3.2 where the main difference lies in the new natural and adversarial domain losses. We colored in red the unique terms of our method.
4 Experiments
In this section we conduct comprehensive experiments to emphasise the effectiveness of DIAL, including evaluations under white-box and black-box settings, robustness to unforeseen adversarics, robustness to unforeseen corruptions, transfer learning, and ablation studies. Finally, we present a new measurement to test the balance between robustness and natural accuracy, which we named Fi-robust score.
4.1 A case study on SVHN and CIFAR-100
In the first part of our analysis, we conduct a case study experiment on two benchmark datasets: SVHN (Netzer et al. 2011 and CIFAR-100 Krizhevsky et al. (2009). We follow common experiment settings as in Rice al. (2020): Wu al. 2020). We used the PreAct ResNet-18 (He et al., 2016) architecture on which we integrate domain classificaiin layer. The adversarial training is done using 10-step PGD adversary with perturbation size of 0.031 and a step size of 0.003 for SVHN and 0.007 for CIFAR-100. The batch size is 128, weight decay is 7 and the model is trained for 100 epochs. For SVHN, the initial learinnig rate is set to 0.01 and decays by a factor of 10 after 55, 75 and 90 iteration. For CIFAR-100, the initial lcarning rale set to 0.1 and decays by a factor of 10 after 75 and 90 iterations. Results are averaged over 3 restarts while omitting one standard deviation (which is smaller than 0.2% all experiments). As can be seen by the results in Tables 2 and 3 DIAL presents consistent improvement in robustness (c.g., 5.75% improved robustness on SVHN against AA) compared the standard AT while also improving the natural accuracy. More results presented in Appendix B
7

---

Published in Transactions on Machine Learning Research (09/2022)
Table 1: Loss function comparison. Tet (x,y) be the natural example and corresponding label. Let x' be the adversarial example generated from x. CE refers to the cross-entropy loss function, KL refers the KL-ddiergence loss function, and BCE is the boosted cross-entropy loss function. LCORAL and LaMMD correspond to the correlation alignment and maximum mean discrepancy (Borgwardt et al.) 2006 [Snn & Sacnko 2016), respectively. Emargin minimize the intra-class variations and maximize the inter-class variations [Song et al. 2018). À is a hyper parameters to control the ratio between different losses, and r is the reversal ratio hyper parameter. Let Gy(:0) be the ura extractor neural network with parameters Dy. Let (y000)) be the label classifier with parameters oy. and let Ga(:;aa) the domain classifier with parameters 0d. That is, G(=:0) (y(Gy(:;y): By) is essentially the standard model definition (c.g., wide residual network). We define source domain label d as 0 (for natural examples) and target domain label as (for adversarial examples). For convenience, we present the loss function on a single example.
Method
Loss function
AT
CE(G(z'; 0), v)
TRADES
CE(G(=; 0), KT(G((=; II G(z;))
MART
BC G(x; (1 G(a;0),)
ATDA
LMMD + -margin
DIALoB
CCCGG CE(G(a;O). + A KL(G(2':0) ((E (00.
DIALKI.
CE(G,(C,
Table 2: Robustness against white-box, black-box attacks and Auto-Attack (AA) on SVHN. Black-box
attacks are generated using naturally trained surrogate model. Natural represents the naturally trained (non-adversarial) model.
White-box
Black- Black-Box
Defense Model Natural 2
PGD
PGD1000
CW
2GD
PGD 100
PGD
CW
AA
NATURAL
96.85
0
0
AT
89.90
53.23
49.45
49.23
48.25
86.44
86.28
86.18
86.42
45.25
DIALKL (Ours)
90.66
58.91
55.30
55.11
53.67
87.62
87.52
87.41
87.63
51.00
DIALCE (Ours)
92.88
55.26
50.82
50.54
49.66
89.12
89.01
88.74
89.10
46.52
Table 3: Robustness against white-box, black-box attacks and Auto-Attack (AA) on CIFAR100. Black-box attacks are generated using naturally trained surrogate model. Natural represents the naturally trained (non-adversarial) model.
White-box
Black- Black-Box
Defense Model
Natural 2GD
PGD 100
1GD
CW
PGD PCD
300
PGD 1000
CW
AA
NATURAL
79.30
0
0
0
0
0
0
0
0
AT
56.73
29.57
28.45
28.39
26.6
55.52
55.29
55.26
55.40
24.12
DIALKL (Ours)
58.47
31.19
30.50
30.42
26.91
57.16
56.81
56.80
57.00
25.87
DIALcE (Ours)
60.77
27.87
26.66
26.61
25.98
59.48
59.06
58.96
59.20
23.51
4.2 Performance comparison on CIFAR-10
In this part, we evaluate the performance of DIAL compared to other well-known methods on CIFAR-10. We follow the same experiment setups as in Madry al. (2017): Wang et al. (2019b); Zhang al. (019b)). When experiment settings not identical between tested methods, choose the most commonly used settings,
8

---

Published in Transactions on Machine Learning Research (09/2022)
and apply it to all experiments. This way, we keep the comparison as fair as possible and avoid reporting changes in results which are caused by inconsistent experiment settings (Pang et al. 202(a). To show that our results are not caused because of what is referred to as obfuscated gradients (Athalye et al. 2018), we evaluate our mothod with same setup as in our defense model, under strong attacks (c.g., PGD to0r)
in both white-box, black-box settings, Auto-Atacc (Croce & Hein 2020), unforeseen "natural" corruptions (Hendrycks Dictterich 2018), and unforescon adversarios. To make sure that the reported improvements are not caused by adversarial overfitting (Rice et al. 2020), we report best robust results for each method on average of 3 restarts, while omitting one standard deviation (which is smaller than 0.2% in all experiments). Additional results for CTFAR-10 as well as comprehensive evaluation on MNIST can be found in Appendix D aadd
Table 4: Robustness against white-box, black-box attacks and Auto-Attack (AA) CIFAR-10, Black-box attacks are generated using naturally trained surrogate model. Natural represents the naturally trained (non-adversarial) model.
White-box
Black-Box
Defense Model
Natural
PGD
PGD
CW
PGD
PGT
CW
AA
NATURAL
95.43
0
0
0
0
TRADES
84.92
56.60
55.56
54.20
84.08
83.89
83.91
53.08
MART
83.62
58,12
56.48
53.09
82.82
82.52
82.80
51.10
AT
85.10
56.28
54.46
53.99
84.22
84.14
83.92
51.52
ATDA
76.91
43.27
41.13
41.01
75.59
75.37
75.35
40.08
DIALKL (Ours)
85.25
58.43
56.80
55.00
84.30
84.18
84.05
53.75
DIALCE (Ours)
89.59
54.31
51.67
52.04
88.60
88.39
88.44
49.85
DIALAWP (Ours)
85.91
61.10
59.86
57.67
85.13
84.93
85.03
56.78
TRADESAWT
85.36
59.27
59.12
57.07
84.58
84.58
84.59
56.17
CIFAR-10 setup. We use the wide residual network (WRN-34-10) Zagoruyko & Komodakis 2016 architecture. Sidelong this architecture, we integrate a domain classification layer. To generate the adversarial domain dataset, we use perturbation size of 0.031. We apply 10 of inner maximization iterations with perturbation step size of 0.007. Batch size is set to 128, weight decay is set to 7 and the model is trained for 100 epochs. Similar to the other methods, the initial learning rate was set to 0.1, and decays by factor of 10 at iterations 75 and 90. See Appendix C for additional details.
Table 5: Black-box attack using the adversarially trained surrogate models CIFAR-10.
Surrogate (source) model
Target model
robustness %
TRADES
DIALcE TRADES
67. 77
DIALcE
65.75
MART
DIALcE MART
70.30
DIALeR
64.91
AT
DIALCE
65.32
DIALCE
AT
63.54
ATDA
DIALCE 66.77
DIALCE
ATDA
52.56
Thite-box/BBacc-bra robustness. As reported in Table 4 and Appendix our method achieves better robustness compared to the other methods. Specifically, in the white-box settings, our method improves robustness over Madry et al. (2017) and TRADES by 2% while keeping higher natural accuracy. We also observe better natural accuracy 1.65% over MART while also achieving better robustness over all attacks. Moreover, our method presents significant. improvement of up to 15% compared to the the domain invariant.
9

---

Published in Transactions on Machine Learning Research (09/2022)
method suggested by Song al. (2018) (ATDA). When incorporating AWP, method improves the results of TRADESAWP by almost 2%. When tested black-box settings, DIAL.E presents significant. improvement of more than 4.4% over the second-best performing method, and to 13%. In Table 5 we also present the black-box results when the source model is taken from one of the adversarially trained models. addition ion to the improvement black-box robustness, DIALCE also manages to achieve belier clean accuracy of more than 4.5% over the second-best performing method.
4.2.1 Robustness to Unforeseen Attacks and Corruptions
Unforeseen Adversaries. To further demonstrate the effectiveness of our approach, we test our method against various adversarics that were not used during the training process. We attack the model under the white-box settings with f-PPGD l-PGD, eDeeeeooooo and fz-DeepFool @Moosavi-Dezfooli et al. 2016) adversaries using Foolbox (Rauber al., 2017). We applied commonly used attack budget with 20 and 50 iterations for PGD and DeepFool, respectively. Results presented in Table [ As can be seen, approach gains improvement of up to 4.73% over the second best method under the various attack types and an average improvement of 3.7% over all threat models.
Table 6: Robustness on CIFAR-10 against unseen adversaries under white-box settings.
Threat Model
Attack Constraints
DIALKL
DIALCE
AT
TRADES
MART
ATDA
= 0.5
76.05
80.51
76.82
76.57
75.07
66.25
42PGD
= 0.25
80.98
85.38
81.41
81.10
80.04
71.87
12
74.84
80.00
76.17
75.52
75.95
65.76
f--PGD
7.84
78.69
83.62
79.86
79.16
78.55
69.97
eo-Doe
overshoot=0.02
84.53
88.88
84.15
84.23
82.96
76.08
tx DeeppFool
wveshoot-0000
68.43
69.50
67.29
67.60
66.40
57.35
Unforeseen Corruptions. We further demonstrate that our method consistently holds against unforeseen "natural" corruptions, consists of 18 unforeseen diverse corruption types proposed by HHendrycks & Dietterich (2018) on CIFAR-10, which we refer to as CIFAR10-C. The CIFAR10-C benchmark covers noise, blur, weather, and digital calegories. As can be shown in Figure our method gains a significant and consistent. improvement over all the other methods. Our method leads an average improvement of 4.7% with minimum improvement of 3.5% and maximum improvement of 5.9% comparod to the second best method over all unforeseen attacks. See Appendix F for the full experiment results.
zom Mar
Sabau Mar
disti transfem
saturit
MART
TRACES
corthart
micton Mar
ATDA
Figure 3: Accuracy comparison over all unforeseen corruptions.
10

---

Published in Transactions on Machine Learning Research (09/2022)
4.2.2 Transfer Learning
Recent works (Salman et al. 2020 Utrera et al. 2020) suggested that robust models transfer better on standard downstream classification tasks. Table 7 we demonstrate the advantage of our method when applied for transfer learning across CIFAR10 and CIFAR100 using the common linear evaluation protocol. see Appendix G for detailed settings.
Table 7: Transfer learning results comparison.
Target
Source
Defence Model
CIFAR10
CIFAR100
DIAL AT TRADES
28.57
CIFARIO
26.95
25.40
DIAL
73.68
CIFAR100
AT TRADES
71.41
71.42
4.2.3 Modularity and Ablation Studies
We note that the domain classifier a modular component that can be integrated into existing models for further improvemenss. Removing the domain head and related loss components from the different DIAL formulations results in some common adversarial training techniques. For DIALKL, removing the domain and related loss components results in the formulation of TRADES. For DIALCE, removing the domain and related loss components results in the original formulation of the standard adversarial training, and for DIALAwP the removal results in TRADES Awp. Therefore, the ablation studies will demonstrate the effectiveness of combining DIAL on top of different adversarial training methods.
We investigate the contribution of the additional domain head component introduced in our method. Experiment configuration are in 4.2 and robust accuracy is based on white-box PGD CIFAR-10 dataset. We remove the domain head from both DIALKL. DIALAwP, and DIALCE (equivalent to r = 0) and report the natural and robust accuracy. We perform 3 random restarts and omit one standard deviation from the results. Results are presented in Figure 4 All DIAL variants exhibits stable improvements on both natural accuracy and robust accuracy. DIALCE, DIALKL., and DIALAWP present an improvement of 1.82%, 0.33%, and 0.55% on natural accuracy and an improvement of 2.5%, 1.87%, and 0.83% on robust accuracy, respectively. This evaluation empirically demonstrates the benefits of incorporating DIAL on top of different. adversarial training techniques.
DIAL
DIAL AND
65
Accuncc
Figure 4: Ablation studies for DIALKL: DIALgg, and DIALAWP CIFAR-10. Circle represent the robust- natural accuracy without using DIAL, and square represent the robust-natural accuracy when incorporating DIAL.
11

---

Published in Transactions on Machine Learning Research (09/2022)
4.2.4 Visualizing DIAL
further illustrate the superiority of our method, we visualize the model outputs from the different methods on both natural and adversarial lest data. Figure [ shows the embedding received after applying 1-SNE (Van der Maaten & Hinton 2008) with two componente on the model output for our method and for TRADES. DIAL seems to preserve strong separation between classes both natural test data and adversarial test data. Additional illustrations for the other methods are attached in Appendix H
(a) DIAL natural logits
(b) DIAL adversarial logits
(c) TRADES natural logits
(d) TRADES on adversar-
ial logits
Figure 5: t-SNE embedding of model output (logits) into two-dimensional space for DIAL and TRADES
using the CIFAR-10 natural Lest data and the corresponding PGD generated adversarial examples.
4.3 Balanced measurement for robust-natural accuracy
One of the goals of our method is to better balance between robust and natural accuracy under given model. For a balancod metric, we adopt the idea of Fi-score, which is the harmonic mean between the precision and recall. However, rather than using precision and recall, we measure the Fi-score between robustness and natural accuracy, using a measure we call the Fi-robust score.
true Tmme_robust
Fi-robust True__robue + 1 (falsccrobbust robust + false natural)
(2)
where true robust are the adversarial examples that were correctly classified, false robust are the adversarial examples that were miss- classillss,,fed, and false_natural natural are the natural examples that were miss-classified Results are presented in Table X and demonstrate that our method achieves the best Fi-robust score in both settings, which supports our findings from previous sections.
Table 8: F1-robust measurement using 2GD attack in white and black box settings CIFAR-10.
TRADES
MART
AT
ATDA
DIALcE
DIALKL
DIALAwP
TRADESAWP
White-box 0.659
0.666
0.657
0.518
0.660
0.675
0.698 0.854
0.682
Black-box
0.844
0.831
0.845
0.761
0.890
0.847
0.849
5 Conclusion and Future Work
In this paper, investigated the hypothesis that domain invariant representation can be beneficial for robust learning. With this idca in mind, we proposed a new adversarial learning method, called Domain Inveriant Adversarial Learning (DIAL) that incorporates domain adversarial neural net work into the adversarial training process. The proposed method, DIAL, is theoretically motivated by the domain adaptation generalization bounds. DIAL generic and can be combined with any network architecture and any adversarial training technique wide range of tasks. Additionally, since the domain classifier does not require the class labels, argue that additional unlabeled data be leveraged in future work. Our evaluation process included
12

---

Published in Transactions on Machine Learning Research (09/2022)
strong adversaries, unforescen adversaries, unforeseen corruptions, transfer learning tasks, and ablation studies. Using the extensive empirical analysis, we demonstrate the significant and consistent improvement obtained by DIAL in both robustness and natural accuracy.
6 Broader Impact Statement
Adversarial examples illustrate fundamental vulnerability of deep neural networks, and manage to break state-of-the art DNNs in various fields. As these DNNs are deployed in critical systems, such autonomous vehicles and facial recognition systems,, it becomes crucial to build models which are robust against such attacks. These kind of system cannot tolerate attacks that can cost in human lives. For this reason, we proposed DIAL to improve models' robustness against adversarial attacks. We hope that it will help in building more secure models for real-world applications. DIAL is comparable to the state-of-the-ate methods we tested in terms of training times and other resources. That said, this work is not without limitations: adversarial training still computationally expensive procedure that requires extra computations compared to standard training, with the concomitant environmental costs. Even though incorporating our method introduced improved standard accuracy, adversarial training still degrades the standard accuracy. Moreover, models are trained to be robust using well known threat models such as the bounded lp norms. However, once a model is deployed, we cannot, control the type of attacks it faces from sophisticated adversaries. Thus, the general problem is still very far from being fully solved.
References
Rima Alaifari, Giovanni S Alberti, and Tandri Gauksson. Adef: iterative algorithm to construct adversarial deformations. orXiv preprint arxiv: (tm:1804.072 804007729, 2018.
Maksym Andriushchenko and Nicolas Flammarion. Understanding and improving fast adversarial training. arXiv preprint arXic:2007.0661 2020.
Anish Athalye, Nicholas Carlini, and David Wagner. Obfuscated gradients give a false sense of security: Circumventing defenses to adversarial examples. In International Conference on Machine Learning, pp- 274 283. PMLR, 2018
Idan Attias and Steve Hanneke. Adversarially robust learning of real-valued functions. arXiv preprint 127777,20..1997, 2022.
Idan Attias, Aryeh Kontorovich, and Yishay Mansour. Improved generalization bounds for robust learning. PMLR, 2019.
In Algorithmic Learning Theory, pp.
Idan Attias, Steve Hanneke, and Yishay Mansour. A characterization of semi-supervised adversarially-robust pac Icarnability. arXiv preprint arXiv:2202.054 2022.
Pranjal Awasthi, Natalie Frank, and Mehryar Mohri. On the rademacher complexity of linear hypothesis sets. arXiv preprint Xxxx:22007.10445 2020.
Tao Bai, Jinqi Luo, Jun Zhao, Bihan Wen, and Qian Wang. Recent advances in adversarial training for adversarial robustness. arXiv preprint arXiv:2102.0133 2021.
Karsten M Borgwardt, Arthur Gretton, Malte J Rasch, Hans-Peter Kriegel, Bernhard Schölkopf, and Alex J Smola. Integrating structured biological data by kernel maximum mean discrepancy. Bioinformatics, 22 (14):049-657, 2006.
Tom B Brown, Nicholas Carlini, Chiyuan Zhang, Catherine Olsson, Paul Christiano, and lan Goodfellow. Unrestricted adversarial examples. arXiv preprint ar Xr09: 08350, 2018.
Qi-Zhi Cai, Min Du, Chang Liu, and Dawn Song. 1rx 2018.
Currculuu adversarial training. arXiv preprint
13

---

Published in Transactions on Machine Learning Research (09/2022)
Nicholas Carlini and David Wagner. Adversarial examples not easily detected: Bypassing ten detection methods. In Proceedings of the 10th ACM Workshop on Artificial Intelligence and Securily, pp. 3 3-14, 2017a.
Nicholas Carlini and David Wagner. Towards evaluating the robustness of neural networks. In 2017 ieee symposium on security and prisacy (sp), pp. 39-57. IEEE, 2017b.
Yair Carmon, Aditi Raghunathan, Ludwig Schmidt, Percy Liang, and John C Duchi. Unlabeled data improves adversarial robustness. Xrviv preprint or
2019.
Tianlong Chen, Sijia Liu, Shiyu Chang, Yu Cheng, Lisa Amini, and Zhangyang Wang. Adversarial robustness: From supervide pre pre-training to fine-tuning. In Proceedings of the IEEE/CVE Conference on Computer Vision and Pattern Recognátion, pp. 699-708, 2020.
Minhao Cheng, Qi Lei, Pin-Yu Yu Chen, Inderjit Dhillon, and Cho-Jui Hsieh. Cat: Customized adversarial training for improved robustness. arXiv preprint arXiv:2002.06780 2020.
Jeremy Cohen, Elan Rosenfeld, and Zico Kolter. Certified adversarial robustness via randomized smoothing. In International Conference on Machine Learning, pp. 1310-1320. PMLR, 2019.
Francesco Croce and Matthias Hein. Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks. In TCML, 2020.
Daniel Cullina, Arjun Nitin Bhagoji, and Prateek Mittal. Pac-learning in the presence of advessaiies Tn Advances in Neural Information Processing Systems, pp. 230 241, 2018.
Rumen Dangovski, Li Jing, Charlotte Loh, Seungwook Han, Akash Srivastava, Brian Cheung, Pulkit Agrawal, and Marin Soljacic. Equivariant self-supervised learning: Encouraging equivariance in representations. In International Conference on Learning Representations, 2021.
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei- Fei-Fei. Imagenet: A large-scale hierarchical image database. In 2009 IEFE conference on computer vision and pattern recognition, pp. 248-255. Icce, 2009.
Gavin Weiguang Ding, Yash Sharma, Kry Yik Chau Lui, and Ruitong Huang. Mma training: Direct input space margin maximization through adversarial training. arXiv preprint arXic:1812.096: 2018.
Dimitrios Diochnos, Saeed Mahloujifar, and Mohammad Mahmoody, Adversarial risk and robustness: General definitions and implications for the uniform distribution. In Advances in Neural Information Processing Systems, pp. 10359 10368, 2018
Yinpeng Dong, Fangzhou Liao, Tianyu Pang, Hang Su, Jun Zhu, Xiaolin Hu, and Jianguo Li. Boosting adversarial attacks with momentum. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 9185-9193, 9193, 2018.
Logan Engstrom, Brandon Tran, Dimitris Tsipras, Ludwig Schmidt, and Aleksander Madry. A rotation and translation suffice: Fooling cmns with simple transformations. 2018.
Yaroslav Ganin and Victor Lempitsky. Unsupervised domain adaptation by backpropagation. In International conference on machine learning, pp. 1180-1181 PMLR, 2015.
Yaroslav Ganin, Evgeniya Ustinova, Hana Ajakan, Pascal Germain, Hugo Larochelle, François Laviolette, Mario Marchand, and Victor Lempitsky. Donain-adversarial training of neural networks. The journal of machine learning research, 17(1):2096-2033) 2016.
Justin Gilmer, Ryan P Adams, lan Goodfellow, David Andersen, and George E Dahl. Motivating the rules of the game for adversarial example rescarch. arXiv preprint xrXt::1807.06732 2018.
Micah Goldblum, Liam Fowl, Soheil Feizi, and Tom Goldstein. Adversarially robust distillation. In Proceedings of the AAAT Conference on Artificial Intelligence, volume 34, pp. 3996 4003, 2020.
14

---

Published in Transactions on Machine Learning Research (09/2022)
Ian J Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. arviv preprint Xiv: 141..1212222 11..15227777 6572, 2014.
Sven Gowal, Krishnamurthy Dvijotham, Robert Stanforth, Rudy Bunel, Chongli Qin, Jonathan Uesato, Relja Arandjelovic, Timothy Mann, and Pushmeet Kohli. On the effectiveness of interval bound propagation for training verifiably robust models. arXiv preprint arXiv: 12715, 2018.
Minghao Guo, Yuzhe Yang, Rui Xu, Ziwei Liu, and Dahua Lin. When nas meets robustness: In search of robust architectures against adversarial attacks. In Proceedings of the IEEE/CVF Conference on Compuler Vision and Pottern Recognition, pp. 631-640, 2020.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Identity mappings in deep residual networks. In European conference on computer vision, pp. 630-645. Springer, 2016.
Dan Hendrycks and Thomas G Dietterich. Benchmarking neural network robustness to common corruptions and surface variations. ar preprint wxXi:1807.01697 2018.
Ziyu Jiang, Tianlong Chen, Ting Chen, and Zhangyang Wang. Robust pre-training by adversarial contrastive learning. In NewrIPS, 2020.
Gaojie Jin, Xinping Yi, Wei Huang, Sven Schewe, and Xiaowei Huang, Enhancing adversarial training with second-order statistics of weights. In Proceedings of the IEEE/CVE CCF Conference on Computer Vision and Pottern Recognition, pp. 15273-1555 11283,, 2022.
Harini Kannan, Alexey Kurakin, and lan Goodfellow. arXiv:1803.0693 2018.
Adversarial
logit pairing.
arXiv preprint
Justin Khim and Po-Ling Loh. Adversarial risk bounds for binary classification via function transformation. arxiv preprint (ff::1810009519 2, 2018.
Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. 2009.
Alexey Kurakin, Ian Goodfellow, and Samy Bengio. Adversarial machine learning at scale. arXie preprinl arXiv:1611.01936 2016a.
Alexey Kurakin, lan Goodfellow, Sany Bengio, et al. Adversarial examples in the physical world, 2016b.
Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner. Gradient- based learning applied to document recognition. Proceedings of the IEEE, 86(11):2278-232) 1998.
Sachyung Lec, Hyungyu Lee, and Sungroh Yoon. Adversarial vertex mixup: Toward better adversarially robust generalization. In Proceedings of the IEEE/CVF Conference on Computer Vision and Patlern Recognition, pp. 272 272-2812 281, 2020.
Sachyung Lee, Changhwa Park, Hyungyu Lee, Jihun Yi, Jonghyun Lee, and Sungroh Yoon. Removing undesirable feature contributions using out-of-distribution data. arXie preprint or arXi0:2101.06639, 2021.
Aishan Liu, Shiyu Tang, Xianglong Liu, Xinyun Chen, Lei Huang, Zhuozhuo Tu, Dawn Song, and Dacheng Tao. Towards defending multiple adversarial perturbations via gated batch normalization. arXio preprint arXiv:2012.0166 2020.
Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. arXiv preprint crXic:1706. 1706.0608, 2017.
Yishay Mansour, Mehryar Mohri, and Afshin Rostamizadeh. Domain adaptation: Learning bounds and algorithms. arXiv preprint arXiv:0902.3430 2009.
Omar Montasser, Steve Hanneke, and Nathan Srebro. Ve classes are adversarially robustly learnable, but only improperly. In Conference on Learning Theory, pp. 2512 2512-2530, 2530, 2019.
15

---

Published in Transactions on Machine Learning Research (09/2022)
Seyed-Mohsen Moosavi-Dezfooli, Alhussein Fawzi, and Pascal Frossard. Deepfool: simple and accurate method to fool deep neural networks. In Proceedings of the IEEE conference on compuler vision and pattern recognition, pp. 2574 2582, 2016.
Yuval Netzer, Tao Wang, Adam Coates, Alessandro Bissacco, Bo Wu, and Andrew Ng. Reading digits in natural images with unsupervised feature learning. 2011.
Tianyu Pang, Kun Xu, Chao Du, Ning Chen, and Jun Zhu. Improving adversarial robustness via promoting ensemble diversity. In International Conference on Machine Learning, pp. 4970 4979. PMLR, 2019.
Tianyu Pang, Xiao Yang, Yinpeng Dong, Hang Su, and Jun Zhu. Bag of tricks for adversarial training, ar preprinl ar Xiv:20000 00467, 2020a.
Tianyu Pang, Xiao Yang, Yinpeng Dong, Kun Xu, Jun Zhu, and Hang Su. Boosting adversarial training with hypersphere embedding. arXiv preprint arXiv:2002. 08619, 2020b.
Vinay Uday Prabhu, Dian Ang Yap, Joyce Xu, and John Whaley, Understanding adversarial robustness through loss landscape geometries. arXiv preprint arxi:: 1907. 00070, 2019.
Zhuang Qian, Shufei Zhang, Kaizhu Huang, Qiufeng Wang, Rui Zhang, and Xinping Yi. Improving model robustness with latent distribution locally and globally, arXie preprint arXiv:2107.04401 2021.
Aditi Raghunathan, Jacob Steinhardt, and Percy Liang. Certified defenses against adversarial examples. 2018a.
arxiv preprinl
Aditi Raghunathan, Jacob Steinhardt, and Percy Liang. Semidefinite relaxations for certifying robustness to adversarial examples. arXiv preprint arXiv:181100101 2018b.
Jonas Rauber, Wieland Brendel, and Matthias Bethge. Foolbox: A python toolbox to benchmark the robustness of machine learning models. In Reliable Machine Learning in the Wild Workshop, 34th International Conference on Machine Learning, 2017. URL http:///axiv. org. http://bb1177.....
Allves Rebuffi, Sven Gowal, Dan A Calian, Florian Stimberg, Olivia Wiles, and Timothy Mann. Fixing data augmentation to improve adversarial robustness. arXiv preprint ar Xixx:2103.01946, 2021a.
Sylvestre-Alue Alvise Rebuffi, Sven Gowal, Dan Andrei Calian, Florian Stimberg, Olivia Wiles, and Timothy A Mann. Data augmentation can improve robustness. Advonces in Neural Information Processing Systems, 34, 2021b.
Leslie Rice, Eric Wong, and Zico Kolter. Overfitting in adversarially robust deep learning. In International Conference on Machine Learning, pp. 8093 8104. PMLR, 2020.
Jérôme Rony, Luiz Hafemann, Luiz S Oliveira, Ismail Ben Ayed, Robert Sabourin, and Eric Granger. Decoupling direction and norm for efficient tradient-based 12 adversarial attacks and defenses. In Proceedings of the IEEE/CVE Conference on Computer Vision and Pattern Recognition, pp. 4322 4322-4330, 2019.
Hadi Salman, Andrew Ilyas, Logan Engstrom, Ashish Kapoor, and Aleksander Madry. Do adversarially robust imagenet models transfer better? arXiv preprint or 20X00:2007.084499 2020.
Ludwig Schmidt, Shibani Santurkar, Dimitris Tsipras, Kunal Talwar, and Aleksander Madry. Adversarially robust generalization requires more data. arXie preprint 1xX 2018.
Ali Shafahi, Mahyar Najibi, Amin Ghiasi, Zheng Xu, John Dickerson, Christoph Studer, Larry Davis, Gavin Taylor, and Tom Goldstein. Adversarial training for free! arXiv preprint arXiv:1904.1285 12845, 2019.
Aman Sinha, Hongscok Namkoong, and John Duchi. Certifiable distributional robustness with principled adversarial training. arXiv preprint arXiv:1710.1 10571, 2, 2017.
Chuanbiao Song, Kun He, Liwei Wang, and John Hoperoft. Improving the generalization of adversarial training with domain adaptation. arXiv preprint ar
2018.
16

---

Published in Transactions on Machine Learning Research (09/2022)
Baochen Sun and Kate Saenko. Deep coral: Correlation alignment for deep domain adaptation. In European conference on computer vision, pp. 443-450 Springer, 2016.
Christian Szegedy, Wojciech Zaremba, Πya Sutskever, Joan Bruna, Dumitru Erhan, Tan Goodfellww, and Rob Fergus. Intriguing properties of neural networks. arXiv preprint arXiv:1312.6190 6199,, 2013.
Pedro Tabacof and Eduardo Valle. Exploring the space of adversarial images. In 2016 International Joint Conference Neural Networks (IJCNN), pp. 426-433. IEEE, 2016.
Florian Tramèr, Alexey Kurakin, Nicolas Papernot, Ian Goodfellow, Dan Bonch, and Patrick McDaniel. Ensemble adversarial training: Attacks and defenses. arXiv preprinl 1ixx:1705.077 2017.
Yu-Lin Tsai, Chia-Yi Yi Hsu, Chia-Mu Yu, and Pin-Yu Chen. Formalizing generalization and robustness of neural networks to weight perturbations. arXiv preprint orXiv:2133002000 2021.
Dimitris Tsipras, Shibani Santurkar, Logan Engstrom, Alexander Turner, and Aleksander Madry. Robustness may be at odds with accuracy. arXiv preprint arXiv: 12152, 2018.
Jonathan Uesato, Jean- cann-Bapisste Alayrac, Po-Sen Huang, Robert Stanforth, Alhussein Fawzi, and Pushmeet Kohli. Are labels required for improving adversarial robustness? arXiv preprint arxi 100:1990.110 19011199557, 2019.
Francisco Utrera, Fvan Kravitz, N Benjamin Frichson, Rajiv Khanna, and Michael W Mahoney. Adversarially- trained deep nets transfer better. arXiv preprint arXiv:2007.05869 2020.
Laurens Van der Maaten and Geoffrey Hinton. Visualizing data using t-sne. Journal of machine learning research, (11), 2008.
Yifei Wang, Zhengyang Geng, Feng Jiang, Chuming Li, Yisen Wang, Jiansheng Yang, and Zhouchen Lin. Residual relaxation for multi-view representation learning. Advances in Neural Information Processing Systems, 34:12104-12115, 2021.
Yisen Wang, Xingjun Ma, James Bailey, Jinfeng Yi, Bowen Zhou, and Quanquan Gu. On the convergence and robustness of adversarial training. In ICML, volume 1, pp. 2, 2019a.
Yisen Wang, Difan Zou, Jinfeng Yi, James Bailey, Xingjun Ma, and Quanquan Gu. Improving adversarial ro- bustness requires revisiting misclassified examples. In International Conference on Learning Representations, 2019b.
Eric Wong and Zico Kolter. Provable defenses against adversarial examples via the convex outer adversarial polytope. In International Conference on Machine Learning, pp. 5286 5295. PMLR, 2018.
Eric Wong, Frank R Schmidt, Jan Hendrik Metzen, and J Zico Kolter. Scaling provable adversarial defenses. arXiv preprint arXiv: 10551144 2018
Eric Wong, Leslie Rice, and Zico Kolter. Fast is better than free: Revisiting adversarial training. arXiv preprint ar Xrx::20001 089994, 2020.
Dongxian Wu, Shu-Taa Xia, and Yisen Wang. Adversarial weight perturbation helps robust generalization. Advances in Neural Information Processing Systems, 33, 2020.
Chaowei Xiao, Jun-Yan Zhu, Bo Li, Warren He, Mingyan Liu, and Dawn Song. Spatially transformed 2018.
adversarial examples. arXiu preprint
Cihang Xie and Alan Yuille. Intriguing properties of adversarial training at scale. arXiv preprint Nkk::1906.03787 2019.
Cihang Xie, Yuxin Wu, Laurens van der Maaten, Alan Yuille, and Kaiming Hc. Feature denoising for improving adversarial robustness. In Proceedings of the IEEE/CVF Conference on Computer Vision and Paltern Recognation, pp. 501-509, 2019a.
17

---

Published in Transactions on Machine Learning Research (09/2022)
Cihang Xie, Zhishuai Zhang, Yuyin Zhou, Song Bai, Jianyu Wang, Zhou Ren, and Alan L Yuill.. Improving transferability of adversarial examples with input diversity. In Proceedings of the IEFE/CVE Conference on Computer Vision and Pottern Recognition, pp. 2730 2739, 2019b.
Huanrui Yang, Jingyang Zhang, Hongliang Dong, Nathan Inkawhich, Andrew Gardner, Andrew Touchet, Wesley Wilkes, Heath Berry, and Hai Li. Dverge: diversifying vulnerabilities for enhanced robust generation of ensembles. arviv preprind or Xiv:20000 14720, 2020.
Dong Yin, Ramchandran Kannan, and Peter Bartlett. Rademacher complexity for adversarially robust generalization. In International Conference on Machine Learning, pp. 7085 7094. PMLR, 2019.
Fuxun Yu, Chenchen Liu, Yanzhi Wang, Liang Zhao, and Xiang Chen. Interpreting adversarial robustness: A view from decision surface in imput space. arXiv preprint arXiv:1810.0014, 2018.
Sergey Zagoruyko and Nikos Komodakis. Wide residual networks. ar preprint arXie: 07000 2016.
Runtian Zhai, Tianle Cai, He, Chen Dan, Kun He, John Hoperoft, and Liwei Wang. Adversarially robust. generalization just requires more unlabeled data. arXiv preprint arxic:1906 00555,, 2019.
Dinghuai Zhang, Tianyuan Zhang, Yiping Lu, Zhanxing Zhu, and Bin Dong. You only propagate once: Accelerating adversarial training via maximal principle. arXiv preprint Miii:1905.00877 2019a.
Haichao Zhang and Jianyu Wang. Defense against adversarial attacks using feature scattering- adversarial training. Advances in Neural Information Processing Systems, 32:1831-1841, 2019.
Hongyang Zhang, Yaodong Yu, Jiantao Jiao, Eric Xing, Laurent El Ghaoui, and Michael Jordan. Theoretically principled tradc-off between robustness and accuracy. In International Conference Machine Learning, pp. 7482-7482. PMLR, 2019b.
Jingleng Zhang, Xilie Xu, Bo Han, Gang Niu, Lizhen Cui, Masashi Sugiyama, and Mohan Kankanhalli. Attacks which do not kill training make adversarial learning stronger. In International Conference on Machine Learning, pp. 11278- 1287. PMLR, 2020.
18

---

Published in Transactions on Machine Learning Research (09/2022)
Algorithm 1 Domain Invariant Adversarial Learning
Input: Source data S ((ss yi) and network architecture Gy, Gy. Ga Parameters: Batch size m, perturbation size e, pgd attack step size T, adversarial trade-off X, initial reversal ratio r, and step size a Init: Y, and Y1 source and target domain vectors filled with and respectively
Output: Robust network G (G Gy. Ga) parameterized by θ (0y, fy. 0a) respectively repeat
Fetch mini-batch X, = {エ Y. = {
# Generate adversarial target domain batch X for j I,..., m (in parallel) do r/ + (GD(()),b,,,7)
X, f X, x
end for
2y C -- (X.), (X)),
e, 4 + (X.),, Yo), (X)))(G)(X)) K)
{t f4 + N r(E: + +
0 8 ---0-- al
until stopping criterion is not met
A Domain Invariant Adversarial Learning Algorithm
Algorithm [ describes pseudo-code of our proposed DIALCE variant. As can be seen, target domain batch is not given in advance with standard domain-adaptation task. Instead, for each natural batch we generate target batch using adversarial training. The loss function is composed of natural and adversarial losses with respect to the main task (e.g., classification), and from natural and adversarial domain losses. By maximizing the losses on the domain we aim to learn a feature representation which is invariant to the natural and adversarial domain, and therefore more robust.
B additional results on CIFAR-100 and SVHN
Table Robustness against white-box, black-box attacks and Auto-Attack (AA) on SVHN. Black-box attacks are generated using naturally trained surrogate model and applied to the best performing robust models.
White-box
Black-Box
Defense Model
Natural
2GD
PGD
100
PCD1000
CW
PGD
PGD 100
PGD 1000
CW
AA
TRADES
90.35
57.10
54.13
54.08
52.19
86.89
86.73
86.57
86.70
49.5
DIALet (Ours)
90.66
58.91
55.30
55.11
53.67
87.62
87.52
87.41
87.63
51.00
DIALcE (Ours)
92.88
55.26
50.82
50.54
49.66
89.12
89.01
88.74
89.10
46.52
Table 10: Robustness against white-box, black-box allacks and Auto-Attack (AA) CIFARI00. Black-box attacks are generated using naturally trained surrogate model and applied to the best performing robust models.
White-box
Black-Bon
Defense Model
Natural
20
1GD
1GD1000
2GD
PGD
1GD1000
CW
AA
TRADES
58.24
30.1
29.66
29.64
25.97
57.05
56.71
56.67
56.77
24.92
DIAL (Ours)
58.47
31.19
30.50
30.42
26.91
57.16
56.81
56.80
57.00
25.87
DIALcE (Ours)
60.77
27.87
26.66
26.61
25.98
59.48
59.06
58.96
59.20
23.51
19

---

Published in Transactions on Machine Learning Research (09/2022)
C CIFAR-10 Additional Experimental Setup details
Additional defence setup. For being consistent with other methods, the natural images are padded with 4-pixel padding with 32-random crop and random horizontal flip. Purthermore, all methods are trained using SGD with momentum 0.9. For DIATKL, we balance the robust loss with À = 6 and the domains loss with y == 4. For DIALCE, we balance the robust loss with À and the domains loss with r = 2. For DIAL-AWP, we used the same learning rate schedule used in Wu et al. (2020). where the initial 0.1 learning rate decays by a factor of 10 after 100 and 150 iterations. For black-box attacks, we used two types of surrogate models (1) naturally trained surrogate model, with natural accuracy of 95.61% and (2) surrogate model trained using one of the adversarial training methods.
D Benchmarking the State-of-the-ar on MNIST
Defence setup. We use the same CNN architecture used in Zhang et al. (2019b) which consists of four convolutional layers and three fully-connected layers. Sidelong this architecture, we integrate domain classification layer. To generate the adversarial domain dataset, we use perturbation size of e = 0.3. We apply 40 iterations of inner maximization with perturbation step size of 0.01. Batch size set to 128 and the model is trained for 100 epochs. Similar to the other methods, the initial learning rate was set to 0.01, and decays by a factor of 10 after 55 iterations, 75 and 90 iterations. All the models in the experiment are trained using SGD with momentum 0.9. For our method, we balance the robust loss with X = 6 and the domains loss with 0.1.
robustness. We evaluate all defense models using 4GD40 PGD100 1GD
and CW (exo version of Carlini & Wagner 2017b) attack optimized by PGD-100) with step size 0.01. We constrain all attacks by the same perturbation c = 0.3. For our black-box setting, we use a naturally trained surrogate model with natural accuracy of 99.51%. As reported in Table 11 our method achieves improved robustness over the other methods under the different attack types, while preserving the same level of natural accuracy, and even surpassing the naturally trained model. We should note that general, the improvement margin MNIST is more moderate compared to CIFAR-10, since MNIST is an casier task than CIFAR-10 and the robustness range is already high to begin with. Additional results are available in Appendix D
Table 11: Robustness against white-box, black-box attacks and Auto-Attach (AA) on MNIST. Black- attacks are generated using naturally trained surrogate model and applied to the best performing robust. models.
White-box
Black-Boo
Defense Model
Natural
4GD
IGD
CW
PGD
PGD 100
CW
AA
TRADES
99.48
96.07
95.52
95.69
98.12
97.86
98.21
92.79
MART
99.38
96.99
96.11
95.98
98.16
97.96
98.28
93.30
AT
99.41
96.01
95.49
95.78
98.05
97.73
98.20
88.50
ATDA
98.72
96.82
96.26
96.31
97.74
97.28
97.76
93.31
DIALKL (Ours)
99.46
97.05
96.06
96.17
98.14
97.83
98.14
93.68
DIALCE (Ours)
99.52
97.61
96.91
97.00
98.41
98.12
98.48
93.43
E Additional Results on MNIST and CIFAR-10
present additional results using the 1GD1 threat model. We use step size of 0.003 and constrain the altacks by the same perturbation
In Table 12
0.031. Table 13 presents comparison of our method combined with AWP to other the variants of AWP that were presented in Wu et al. (2020). In addition, in Table 14 we add the Fj-robust scores for different variants of AWP.
20

---

Published in Transactions on Machine Learning Research (09/2022)
Table 12: PGD 1000 attack on MNIST and CIFAR-10 on white-box and black-box settings.
MNIST
CIFAR-10
Defense Model
White-box
Black-box
White-box
Black-box
TRADES
95.22
97.81
56.43
83.80
MART
95.74
97.89
56.55
82.47
AT
95.36
97.78
54.40
83.96
ATDA
96.20
97.34
41.02
75.11
DIALCE (Ours)
96.78
98.10
51.57
88.22
DIALKL (Ours)
95.99
97.89
56.73
84.00
Table 13: Robustnoss comparison of DIAL-AWP and other variants of AWP that do not require additional data under the Loo. threat model.
Defense Model
Natural
PGD
PGD100
CW no
AA
DIAL-AWP (Ours)
85.91
61.10
59.86
57.67
56.78
TRADES-AWP (Wu et al. 2020)
85.36
59.27
59.12
57.07
56.17
MART-AWP (Wu et al. 2020
84.43
60.68
59.32
56.37
54.23
AT-AWP (Wu al. 2020
85.57
58.14
57.94
55.96
54.04
Table 14: Fy-robust measurement on AWP variants based on white-box attack.
Defense Model
Fi-robust
DWAL-AWP (Ours)
0.69753
TRADES-AWE (Wu et al.
2020)
0.68162
MART-AWP Wu el al. 2020)
0.68857
AT-AWP (Wu et al. 2020)
0.67381
F Extened results on Unforeseen Corruptions
We present full accuracy results against unforeseen corruptions in Tables 15 and 16
Table 15: Accuracy (%) against unforeseen corruptions.
Defanee Model
brightteea
detscuo blue
Fog
cluss blu
Jack compeeaiion
motion blut
saturate
SLoW
speckle uoise
TRADES
80.04
50.68
MART
76
75.62
58
78.02
AT
83.30
80.42
77. 77.99
73
78.66 64
82,
80.37
60.74
ATDA
72.07
69.38
45.52
64 08.88
73.22
63.47
72.07
86.76
72.27
DIAL (Ouza)
87.14
84.84
06.08
81.82
87.0Y
81.20
80.45
84.18
84,04
Table 16: Accuracy (%) against unforeseen corruptions.
Defense Model
transform
abot
bluc
TRADES
43.11
70. 11
76.45 45-45
78 21
73.72
$2. 03.77
50 -1212
50.72
75.97
MART
41.22
77.77
75.07
74.97
$1.51
70. 66
79.25
77.8
AT
43.30
56
77.
73.76
$2.76
8G
40
73.58
ATDA
36.06
07. 86
02 56
G4. 68
73. 4G
73. 28
70.50
51
(Ounz)
48.54
84.15
81.76
83.76
78.28
21

---

Published in Transactions on Machine Learning Research (09/2022)
G Transfer Learning Settings
The models used are the same models from previous experiments. We follow the common procedure of "fixed-feature" setting, where only a linear layer on top of the pre-trained network is trained. We train a linear classifier on CIFAR-100 on top of the pro-trained network which was trained on CIFAR-10. We also train a linear classifier on CIFAR-10 10 on top of the pre-trained nwork which was trained on CIFAR-100. We train the linear classifier for 100 epochs, and an initial learning rate of 0.1 which is decayed by a factor of 10 at epochs 50 and 75. We used SGD optimizer with momentum 0.9.
H Extended visualizations
In Figure we provide additional visualizations of the different adversarial training methods presented above. We visualize the models outputs (logits) using L-SNE with two components on the natural Lest data and the corresponding adversarial test data generated using PGD20 white-box attack with step size 0.003 and
c c= 0.031 on CIFAR-10.
In Figure we visualize statistical differences between natural and adversarial examples in the fea representation layer. Specifically, we show the differences in mean and std on thirty random feature values from the feature representation layer as we pass through network the natural test examples and their corresponding adversarial examples. We present the results on same network architecture (WRN-34-10), trained using three different training procedures: naturally trained network, network trained using standard adversarial training (AT) Madry et al. (2017), and DIAL on the CIFAR-10 dataset. When the statistical characteristics of each feature differ from each other, it implics that the features layer is less domain invariant.
That is, smaller differences in mean/std yields a better invariance to adversarial examples. One can observe that for DIAL, there is almost no differences between the mean/std of natural examples and their corresponding adversarial examples. Moreover, for the vast majority of the features, DIAL present smaller differences compared to the naturally trained model and the model trained with standard adversarial training. Best viewed in colors.
(a) Mean difference comparison
(b) Standard deviation difference comparison
Figure 6: Mean and std differences comparison between DIAL, naturally trained model and model trained using standard adversarial training on thirty random features from the features layer on the CIFAR-10 dataset. with WRN-34-100 10 architecture. Each bar represents the absolute difference between the means/ste /std of the natural examples and the mcan/std of their corresponding adversarial examples on this same feature.
22

---

Published in Transactions on Machine Learning Research (09/2022)
(a) MART embedded model output (logits) (b) MART embedded model output (logits)
on natural test data
on adversarial test
100.09000
(c) AT embedded model output (logits) on (d) AT embedded model output (logits) on natural test data
adversarial test data
(c) ATDA embedded model output (logits) (f) ATDA embedded model output (logits) adversarial test data
natural test data
Figure 7: I-SNE embedding of model output, logits, in two-dimensional space for MART, AT, and ATDA under natural and adversarial test data from CIFAR-10.
23

---

Published in Transactions on Machine Learning Research (09/2022)
In-depth Analysis
Additionally, we present two additional results and visualizations that can show our performance difference is solid. First, in Figure we present a 2d T-SNE plot on the features layer (unlike the logits layer, which is more related to the quantitative results), to further demonstrate that our method indeed learns a domain invariant feature representation better than the other methods, and compared it to TRADES, Second, we wish demonstrate that our performance gain is not a due specific aspects of the domain (e.g., improvement only specific classes). To do so, we visualize in figures 10 the classification improvement obtained by our method each class in CIFAR-10. As be seen, the natural examples, method improved the accuracy on all classes, and on the adversarial examples, our method improves robustness 9 out of 10 classes. This further demonstrates the generalization of our approach.
(a) DIAL embedded features on natural (b) DIAL embedded features on adversarial test data
test data
TRADES embedded features natu (d) TRADES embedded features adver- sarial data
data
Figure 8: Visualizing the two-dimensional T-SNE feature space embedding of DIAL and TRADES for (1)
natural test data, and (2) adversarial test data from CIFAR-10.
24

---

Published in Transactions on Machine Learning Research (09/2022)
80
DIAL
+48. 8
+0.4
Robustness (%)
60
50
+0.7
+3.9
+2.1
30
+2.0
plane
car
bird
cat
deer
dog
frog
horse
ship
truck
Class Type
Figure DIATKL and TRADES Robustness (%) for each class CIFAR-10. Adversarial examples
generated using PGD-20. Our method manages to improve robustness over TRADES 9 out of 10 classes. Green annotation presented the difference percentage improvenent.
100
TRADES
+0.1
DIAL
+5,7
+5.8
+2.1
+1.7
+0.8
Natural Accuracy (%)
80
+8.0
60
50
plane
car
bird
cat
deer
dog Type
frog
horse
ship
truck
Figure 10: DIALCE and TRADES natural accuracy (%) for each class CIFAR-10. Our method manages to improve natural accuracy on all 10 classes. Green annotation presented the difference percentage improvement.
25


Reviews:
[{"review_id": "Y9izZw8uTX", "reviewer": "Reviewer_HPx8", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "This paper has a simple and general framework to increase robustness. Some expriments are conducted to verify its effectiveness.\n\nI have some concerns over the writing of the motivation, as chapter 3.1 introduces lots of notations before presenting the objective of the paper. Some rigorous definition could be saved for later.  I also cannot find too much information from Eq (1), as there was no non-trivial inequality in this derivation, so I highly doubt it would lead to non-trivial results.\n\nThe KL divergence betwen adverarial and natural features should not be trivially put here. As it could very complicated distribution, unless it is simplified to Gaussian or something, but this reduces to simpler distance function rather than KL.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper proposes a method to increase adversarial invariance of machine learning models, by adding another objective function that minimizes the feature difference between original data features and adversarial data features.", "claims_and_evidence_yn": "", "claims_explanation": "", "audience_yn": "", "audience_explanation": "", "requested_changes": "The writing feels rather unnecessarily complicated and maybe not in easy-reading order. It is hard to say what exactly to do, but I just mention it here.\n\nI think a detailed explanation of KL between features should be provided.", "broader_impact_concerns": "none", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "SF9t5rciS1", "reviewer": "Reviewer_K348", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Pros\n- The main idea of the paper is easy to follow.\n- The way how the loss is grounded and motivated by deriving equation (1) is a big plus to me.\n\nCons\n- The related work is hard to understood by readers who are not familiar with these papers.\n  - Beginning of section 2.1: Simply listing a long list of papers is not so helpful to readers that are not familiar with the field. I suggest you expand the beginning of 2.1 a bit. You can summarise the key idea of each category using 1 or 2 sentences while listing related work in that category. I believe you do have a lot more space to spent but still keep you draft $\\leq$ 12 pages.\n  - Second to last paragraph of section 2.1: FGSM is not explained.\n  - The second half of section 2.1 needs some more work: Currently it's not very clear what each previous method is different from the proposed one by simply reading it. You may want to move it in the end if that helps, or you need to improve the presentation (either in writing or with the help of some illustration) to not make people confused after finishing section 2.1 and to make understand the experimental comparisons easier.\n  - Title of section 2.2: Should it be theoretical analysis or something?\n  - Are max-margin type of methods also related here in general?\n\nQuestions\n- The first unlabelled equation: $\\ell$ is used to denote loss here while later $L$ is used. Is this intended?\n- List of SOTA methods in the paragraph above figure 1: the list stops at 2020---are they actually SOTA or more like popular benchmark targets?\n- Figure 1 and related work: It looks like the contrastive learning community also found using similar architecture improves representation learning; see [1,2].\n- \"the natural and adversarial losses\" below equation (1): which terms do you refer to by these names (there are 3 terms other than the discrepancy term)? Maybe consider annotating equation (1).\n\n[1] Equivariant Self-Supervised Learning: Encouraging Equivariance in Representations, ICLR 2022\n[2] Residual Relaxation for Multi-view Representation Learning\n, NeurIPS 2021", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper argues that by ensuring the internal representation of a classifier to be adversarial invariant could lead to a better trade-off between robustness and accuracy.\nIt motivates this idea by recasting this problem as learning domain invariant features where the two domains are the natural domain (original data) and the adversarial domain (adversarially perturbed data).\nIt proposes an architecture with an extra head for domain classification with an extra loss term to encourage the feature to be domain invariant.\nResults not only show that models learned by the proposed method are more robust than state-of-the-arts but also have a better robustness-accuracy trade-off, using a new-proposed metric.", "claims_and_evidence_yn": "", "claims_explanation": "", "audience_yn": "", "audience_explanation": "", "requested_changes": "Critical\n- Improve related work section as I suggest above.\n- Figure 2: Labels and legends are a bit to small.\n- Figure 2: Is \"Clean\" refer to a normally trained model? Please define it in the text.\n- The semi-supervised point in section 4.2.3: Either demonstrate it empirically or put it to discussion / future work.\n- Typo in the third to the last line of section 5: \"unforeseen adversaries ,unforeseen corruptions\" -> \"unforeseen adversaries, unforeseen corruptions\".\n  - Also it looks to me that you have an extra space after the last word \"adversaries\" in the previous line.\n\nGood to have\n- Figure 2: For baselines, features also have large variance---meaning that some examples have pretty much invariant features while some have very different features. It would be interesting to do a visual inspection to see what are those examples.\n- Table 1 and 2: The locations of them are a bit weird.\n- Structure of section 4: I suggest to structure the experiments by tasks instead of datasets.", "broader_impact_concerns": "Discussions on security related benefits could be expanded---currently only a single sentence (\"We hope that it will help in building more secure models for real-world applications.\").", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "XG9OuvNijH", "reviewer": "Reviewer_y8NQ", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "## Paper strengths and contributions\n**Motivation and novelty**\n\n- The motivation for optimizing the trade-off between robust and standard accuracy is convincing.\n- The idea of learning a feature representation that is invariant across natural samples and adversarially generated samples and can be used for classification is intuitive and convincing. This paper presents an effective way to implement this idea.\n\n**Experimental results**\n\n- The presentation of the experimental results is clear.\n- The experimental results show that the proposed method outperforms existing baselines on datasets including CIFRA100, SVHN, and MNIST.\n- The experimental results show that the proposed method performs better on unforeseen corruptions.\n- The t-SNE plots demonstrate that the proposed method learns clear clusters of classes.\n\n**Reproducibility**\n\nThe code is provided, which helps understand the details of the proposed framework.\n\n## Paper weaknesses and questions\n\n**Small-scale datasets**\n\nDespite the convincing experimental results, the experiments are only conducted on small-scale datasets such as CIFAR100 and SVHN. It would be a lot more convincing to demonstrate that the success of the proposed method can apply to larger and more realistic datasets such as ImageNet.\n\n**Grammar errors**\n\nThere are many grammar errors (e.g. \"let $G_d(·;\\theta_d)$ the domain classifier\") making this paper less enjoyable to read. The authors should carefully proofread the paper and fix these errors.\n\n**Unclear figure caption**\n\nThe caption of Figure 2 states \"DIAL, naturally trained model and model trained using standard adversarial training on five random features from the features layer\" without introducing the legend (i.e. DIAL mean difference, Clean mean difference, Standard AT mean difference) on Figure 2(a) and Figure 2(b).\n\n**Standard deviation**\n\nGiven the small gaps among the performance of different methods, it would make sense to provide the standard deviation of the accuracy.\n\n**Qualitative results**\n\nIt would make sense to provide some qualitative comparisons to give readers a better idea of the performance difference between the proposed method and the baselines.\n\n**Incorporate two losses**\n\nCan $DIAL_{CE}$ and $DIAL_{KL}$ be integrated and yield even better performance?\n\n**State-of-the-art image classifier result**\n\nIt would be more informative if the performance of the state-of-the-art methods on image classification on each dataset and their attacked performance are shown. This way, readers understand the performance gap and the trade-off.\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper addresses the problem of learning a classifier that is robust to adversarial attacks while maintaining the classification performance. To this end, the paper proposes a method, Domain Invariant Adversarial Learning (DIAL), that learns a feature representation that is invariant across natural samples and adversarially generated samples and can be used for classification. The experiments on image classification datasets, including CIFRA100, SVHN, and MNIST, show that the proposed method outperforms existing baselines. In sum, I believe this work studies an important research direction and proposes a method that seems simple yet effective. However, I am still concerned about the clarity of the paper and the insufficient evaluation of larger-scale datasets. As a side note, this topic does not align well with my research directions and I have not been paying much attention to the progress in this field. Therefore my opinions probably should be taken lightly.", "claims_and_evidence_yn": "", "claims_explanation": "", "audience_yn": "", "audience_explanation": "", "requested_changes": "Details are stated in the Strengths And Weaknesses section.", "broader_impact_concerns": "N/A", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper tackles an important problem - making an accurate across natural images classifier that is robust to attacks, propose a method that learns such a classifier. The method outperforms baselines. The reviewers agree on that. The authors have answered the concerns of the reviewers and updated the version. I don't think any additional revision is needed.
Decision: Accept (poster)