Improving Out-of-Distribution Generalization by Adversarial Training with Structured Priors
Authors: Qixun Wang
Yifei Wang
Hong Zhu
Yisen Wang
Venue: NeurIPS 2022 Accept (2022)

Abstract: Deep models often fail to generalize well in test domains when the data distribution differs from that in the training domain. Among numerous approaches to address this Out-of-Distribution (OOD) generalization problem, there has been a growing surge of interest in exploiting Adversarial Training (AT) to improve OOD performance. Recent works have revealed that the robust model obtained by conducting sample-wise AT also retains transferability to biased test domains. In this paper, we empirically show that sample-wise AT has limited improvement on OOD performance. Specifically, we find that AT can only maintain performance at smaller scales of perturbation while Universal AT (UAT) is more robust to larger-scale perturbations. This provides us with clues that adversarial perturbations with universal (low dimensional) structures can enhance the robustness against large data distribution shifts that are common in OOD scenarios. Inspired by this, we propose two AT variants with low-rank structures to train OOD-robust models. Extensive experiments on DomainBed benchmark show that our proposed approaches outperform Empirical Risk Minimization (ERM) and sample-wise AT. Our code is available at https://github.com/NOVAglow646/NIPS22-MAT-and-LDAT-for-OOD.

Improving Out-of-Distribution Generalization by Adversarial Training with Structured Priors
Qixun Wang!"
Yifei Wang 2- Hong Zhu
Yisen Wang" 1.4t
Key Lab. of Machine Perception (MoE), School of Intelligence Science and Technology. Peking University
2 School of Mathematical Sciences, Peking University Huawei Noah's 's Ark Lab Institute for Artificial Intelligence, Peking University
Abstract
Deep models often fail to generalize well in test domains when the data distribution differs from that in the training domain. Among numerous approaches to address (OOD) generalization problem, there has been growing surge of interest in exploiting Adversarial Training (AT) to improve OOD perfor- mance. Recent works have revealed that the robust model obtained by conducting sample-wise AT also retains transferability to biased test domains. In this paper, OOD per-
this
empirically show that sample- sample-wise wise AT has limited improvement formance. Specifically, we find that AT can only maintain performance at smaller scales of perturbation while Universal AT (UAT) is more robust larger-scale per- turbations. This provides us with clues that adversarial perturbations with universal (low dimensional) structures can enhance the robustness against large data distribu- tion shifts that are common in OOD scenarios. Inspired by this, we propose two AT variants with low-rank structures train OOD-robust models. Extensive exper- iments on Bed benchmark show that our proposed approaches outperform Empirical Risk Minimization (ERM) and sample-wise AT. Our code available https: :/ /et:::/ggttttt giighu..bc... Siaabb.cca///V4/ggooo66///11222 MAT and-LDAn LDAT for- 00D)
1 Introduction
Existing deep learning methods have achieved good performance on visual classification tasks under the same distribution of training sets and test sets. However, when the data distribution of the test set different from that the training set, the classification performance of the deep neural networks (DNNs) may decrease sharply (I. This is mainly because DNNs may capture spurious features such as the background and style information to assist the fast fitting during the training process (2]. However, real-world scenarios, test data may differ from training data in the background and style information, thus DNNs that rely on unstable spurious features to make predictions will fail. Solving the above problem is known the out-of-distribution (OOD) generalization.
Another scenario where DNNs may fail is that they are often vulnerable to adversarial examples [ Adversarial training (AT) is originally proposed as effective way to defend against adversarial attacks [4] Moreover, there is work showing that adversarial training helps to solve the OOD generalization problem because OOD data be seen stronger perturbations to extent (5). The reason why AT can defend against adversarial attacks meanwhile benefit OOD generalization is that it can make DNNs robust the interference of spurious features, such randomly injected noise
Equual Contribution. Cooreesponding author: Yisen Wang (yisen.wang @ pku.edu.cn).
36th Conference on Neural Information Processing Systems (NcurIPS 2022).

---

(in adversarial examples) or the spurious correlation between labels and background information (in OOD generalization). In other words, AT enables DNNs make predictions using intrinsic features rather than spurious features.
A potential problem, however, is that existing AT methods ignore the specific design of perturbations when used for solving OOD generalization problems. They usually simply conduct sample-wise AT [6]. which only brings limited performance improvement to OOD generalization. The essential reason for the failure of this type of approach is that the perturbations it uses cannot distinguish invariant and spurious features. As result, it improves the robustness at the expense of the decreasing standard accuracy 77 Moreover, we empirically find that when adapting Universal AT (UAT (8] to OOD problems, i.e., conducting AT with domain-wise perturbations, shows stronger when facing larger-scale perturbations compared to the sample-wise AT (see Section 3.2). Since
the sample injected with large-scale perturbations can be regarded as OOD samples ( we draw inspiration from this phenomenon that AT with universal (low-ding (ow-dimensional) structures can be the key to solving OOD generalization. Therefore, we propose to use structured low-rank perturbations related to domain information in AT, which can help the model to filter out background and style information, thus benefiting OOD generalization. We make the following contributions in our work:
We identify the limitations of sample-wise AT on OOD generalization through a series of experiments. To alleviate this problem, we further propose two simple but effective AT variants with structured priors to improve OOD performances.
We theoretically prove that our proposed structured AT approach can accelerate the conver- gence of reliance on spurious features to 0 when using finite-time hinte-time-stopped gradient descent, thus enhancing the robustness of the model against spurious correlations.
By conducting experiments on the DomainBed benchmark 9 we demonstrate that our methods outperform ERM and sample-wise AT on various OOD datasets.
2
Related Work
Solving OOD Generalization with AT. According to [.. the performance of deep models is susceptible small-scale perturbations injected in the input images, even if these perturbations are imperceptible to humans. Adversarial training (AT) is an effective approach to improve the robustness to input perturbations [4] 10 [1]. However, many recent works have begun to focus on the connection between AT and OOD due to the fact that OOD data can be regarded as one kind of large-scale perturbation. These works seek exploit the robustness provided by AT improve OOD generalization. For instance, [6] applied sample-wise AT to OOD generalization. They theoretically found that if a model robust input perturbation on training samples, it also generalizes well on OOD data. (5) theoretically established a link between the objective of AT and the OOD robustness. They revealed that the AT procedure can be regarded as a heuristic solution to the worst-case problem around the training domain distribution. Nevertheless, the discussion of (6) and (5) is restricted to the framework of using Wasserstein distance to measure the distribution shift, which is less practical for the real-world OOD setting where domain shifts are diverse. Additionally, they only studied the case of sample-wise AT and did not further investigate the effect of different forms of AT (not sample-wise) on OOD performance. Other works such as [12) focus on the structure design of the perturbations. They used multi-scale perturbations within one sample, but they did not exploit the universal information within one training domain. In our work, we focus on real- world OOD scenarios where there are additional clues lying in the distribution shifts, i.c, the low-rank structures in the spurious features (such as background and style information) across one domain. We further design a low-rank structure in the perturbttioss to specifically eliminate such low-rank spurious correlations.
OOD Evaluation Benchmark. The DomainBed benchmark | provides a fair way of evaluating different state-of-the-et OOD methods, which has been widely accepted by the community. By conducting rigorous experiments in a consistent setting, they revealed that many algorithms that claim to outperform previous methods cannot even outperform ERM. Unlike previous works using AT to address OOD generalization, such as 6 and [5), we adopt the Domainhed benchmark for a fair comparison of our approach with existing oftt--ff-the-att methods in this paper.
2

---

Weakness of Sample-wise AT for OOD Generalization
3.1 Preliminaries
cut-of-distribution (OOD) Generalization. Assuming E A as the random data in the input space X and y WE y as the target random data in the label space we have the predictor w 6(x) where Φ X + 2 denotes the feature extractor and w 2 + )' denotes the classifier.
Now we give the formal definition of the OOD generalization problem. We have of m training domains & {Ei, ..., Em J. where each domain E. is characterized by input dataset E. == {(1 )) containing ne i.idd input samples drawn from the distribution of P.. and test domain Em+1 with data following the distribution of Ptc. where Pte * Pi, = 1, 1,2,... m. f:: X RR+ denotes the loss function. The ultimate goal of OOD generalization find optimal predictor f that minimizes the risk on the unseen test domain:
min P(s.)) (a (z(f y)]-
(1)
Adversarial Training (AT According to 4], AT can expressed as the following optimization problem:
min ((,.))) mmxx C(f(2 + 6), s.t. ||ó||p <, SES
where is the random injected perturbation with 1p norm bounded by e. The inner maximization problem can be optimized by fast gradient sign method (FGSM [3]),, simple one-step scheme:
x = x + csgn V ())(
where sgn(.) the sign function, or by projected gradient descent (PGD (), a more powerful multi-step variant:
II (a' + (
where [] is the projection operator onto the set S, 7 is the step size and denotes the iteration.
3.2 Weakness of AT for OOD Generalization
We now highlight some weaknesses of sample-wise AT for OOD generalization based on a series of empirical evidence. We first conduct toy experiment on the DomainBed benchmark [9] to evaluate the OOD performance AT, We run ERM and AT on four OOD datasets: PACS [1,, OfficeHome [1], VLCS [16], and NICO 17 with fixed set of hyperparameters (detailed experimental settings can be found in Appendix C.I The results are shown Table [ We can see that the improvement
of OOD performance by AT is limited with an average improvement of only 0.1%.
Table 1: Test accuracy (%) on four OOD datasets on DomainBed benchmark with a fixed set of hyperparameters. The improvement of AT is marginal.
Datasets
Algorithm
PACS
OfficeHome
VLCS
NICO
avg
ERM
79.7 + 0.0
59.6
0.0
74.4 + 1.0
70.7 1.0
71.1
AT
81.5 + 0.4
59.9
0.4
75.3 + 0.7
68.2 + 2.2
71.2
We further investigate the reason behind the limitations of performance improvements on OOD datasets of AT. Although previous works have revealed that the robust features obtained by AT can improve OOD generalization (66 [ (8)), ) we find that sample-wise AT' only tolerates small-scale perturbations. Thus, we design an experiment on NICO dataset with multiple scales of perturbations. The scale is calculated with the la norm of the perturbation matrix (experiment details are shown in Appendix C.I). As shown in Figure [ AT suffers severe performance degradation when using large perturbations. This provides clues to understanding the failure of AT in OOD scenarios. The
For simplicity, denote 'AT' for sample-wisg wise AT by default in the rest of the paper.

---

distribution shifts in OOD data usually have much larger scales than the invisible perturbations commonly used in AT. Hence, AT methods designed for small perturbations cannot handle these large-scale domain shifts that often appear in OOD data. However, our experiment shows that this problem can be alleviated by adapting universal AT (UAT [8) to the OOD setting, i.e., using a perturbation for each domain.
Figure 1 shows that UAT has better generalization performance than AT when the perturbation norm is large. There are two empirical explanations for this: First, the background and style information usually have low-rank structure, such as the grassland and snowfield that have recurring parts, Second, similar spurious features often appear within one specific domain, such as PACS [14] and VI.CS [16] datasets. As stated in & the universal perturbation lies in a low dimensional space. Hence using universal (domain-wiss - perturbations will help to resist such low-rank shifts and improve the robustness of the model.
Tel
88xx
19M norm
Figure 1: Test accuracy (%) of AT, UAT, and ERM on NICO dataset. lxc norm used here.
Inspired by this, we proposed two new AT variants with more sophisticated low-rank structures on dif- ferent dimensions to improve OOD generalization in the next section.
4 The Proposed Structured AT Method
In order to construct low-rank structured perturbations, we start by analyzing the structure of sample- wise perturbations. Assume that each input data has shape of N x N x C. N is the size of the input image and C is the number of channels. For simplicity, we assume C = 3. We reparameterize the sample-wise perturbations
a series of 2-D matrices (DI, D22- --.. D ). {Df, .... D {D1, D D where D E R XN² XN* denotes the perturbations in the e-th domain for the input channel ne is the number of the samples in the domain Eg, and is the number of domains. The i-th row of D. represents the e-th channel of the i-th sample in the domain Ee (see the first column in Figure 2 for illustration). By such reparameterization, it is natural to find that there are two orientations to reduce the rank of the perturbations:
1. Along the dimension of the number of samples (along the red arrow in the upper left corner of Figure 2 This corresponds reducing the number of the perturbations used within one domain.
2. Along the dimension of the input scale (along the blue arrow in the upper left corner of Figure 2). This corresponds to reducing the rank of the perturbation used for a specific input sample.
In the following parts, we propose two AT variants with structured priors that reduce the rank in these two directions.
4.1 MAT: Adversarial Training with Combinations of Multiple Perturbations
In this part, propose domain-wise Multiple-perturbatien
Adversarial Training (MAT). It aims to conduct rank minimization along the dimension of the number of samples. Instead of using sample-wise perturbations, MAT constructs a combination of multiple perturbations and shares this mixed perturbation within domain. Specifically, we choose to train the linear combination of k perturbations for each domain E, conduct AT. Here k is hyperparameter and k is far less than the number of samples in domain Ec. The optimization problem can be reformulated as:
min
((ey) (x.)) (C( 8)),)),
(5)
S.L. 80
a
85 110 lp Se,
Σ a = a. 2 for 1, .,,k,
(6)
1=1

---

Sample-wise AT
MAT
LDAT
Niy Niranco
N% Niakke
Domain E
11
Domain
Domain
D3
Figure 2: Illustration of how our proposed structured AT reduces the rank of the perturbations comparing to sample-wise The left column shows how we reparameterize sample-wise AT. The perturbations segmented by domains. A block represents the ne perturbations injected in channel of the samples domain E. This figure shows the case where the input image has three channels (RGB). The red and the blue arrow the upper left shows the two orientations reduce the rank of the perturbations, i.e, along the dimension of the total number of the perturbations and along the dimension of the rank of single perturbation, respectively. The mid column illustrates that MAT reduces the number of the perturbations used for domain E. from ne to & (k < m2). The right column shows that LDAT further reduces the number of the perturbations from & to 1. Moreover, reduces the rank of specific perturbation from N to I < N).
where
k
a²", 8:"
((8%),))
at,
Here denotes the subscript of training domain and of is the weight that can be learned for each perturbation 85. The detailed training procedure of MAT perturbations of and their correspondent coefficients at for each training domain with Gaussian noise. Then we transform &f and of to make sure Σ of 1, of 2 and || S c. For the inner maximization, we conduct one-step gradient ascent to optimize 8; and a:
Algorithm [ We first initialize
MAT works low-rank version of sample-wise AT. In sample-wise AT, we maintain n4, pertur- bations for each domain E., where ne is the number of training samples in domain E.- As for MAT, reduces the number of perturbations available samples from n. to k and obtains low-rank structures (see the third column in Figure for illustration). Therefore, it fulfills rank reduction along the sample-number dimension.
4.2 LDAT: Adversarial Training with Low-rank Decomposed Perturbations
Based on MAT, we further propose Adversarial Training with Low-rank Decomposed perturbations (LDAT). Analogous to MAT, LDAT still shares one perturbation in specific domain. Moreover, LDAT imposes low-rank constraint on the perturbation itself, which corresponds the dimension of the input scale. Technically, we obtain the domain-wise low-rank perturbation matrix å E R N Nxx x by multiplying two matrices: å AB. Here E R and E Rxx where hyperparameter and / & N. Since rank(A < rank(A) (A) and rank(AB) < rank(B) hold for arbitrary matrices B, we have rank(ø) Therefore LDAT reduces the rank of the perturbation
from large value N to relatively small value (see the last column in Figure 2 for illustration).
5

---

Algorithm Detailed Training Procedure of MAT
Input:
Labeled training data m domains E Em. where E, == { {xf yl FE
number of the perturbations to be combined k, perturbation weight learning rate 7
FGSM step size y,. perturbation radius e, number of training epochs T, learning rate for model parameters r, batch size b.
Output:
Updated model Se with parameter 0.
1: Randomly initiate 8. perturbation of. weight of such that E of 1, of 2 2.,110 S Vi E {l,..., } and Ve (1,....m.)
2: for iterations in 1.2,... 2.2,.... ..., do
do 4: Randomly select batch 8 = {(= y -- from domain Ee.
3: for e in 1,2,
5:
Compute the adversarial sample: x = x Ej= a5z Vu E {I.....bb b}
6:
Update 8" by 0 + of + + E
V (( v Vie [I,..., Yu E {l, b}.
7:
Update of by of f a + すサ E._ Project of to the l2 ball of radius E.
Vat (f 11)) vi E {I,.., kj. Yu {I,..., 6}.
8:
Compute the adversarial sample: z = x Σj= 2585, Vu [I.....b} b}
9:
10: Update model parameter: 0 + - r En VoL(Jo(z va). Vu {I,..,b)
11: end for
end for
The formal definition of the LDAT objective is:
min (L s.t. - A Be*0111011 ,10011 Se
(8)
where
A B
(xa)) + (')), A NxIxC B E RIxNxC B
(9)
We provide the detailed training procedure of LDAT in Appendix due to the space limitation of the main text. In comparison MAT, LDAT reduces the number of perturbations available the samples in a domain from & to In addition, it reduces the rank of the perturbation for a single channel of sample from N to l.
4.3 Theoretical Analysis
In this part, we theoretically explain why the domain-wise perturbation proposed MAT and LDAT can help to improve the robustness of the model against spurious correlations following 2 and [1)]. In general, we prove that MAT and LDAT can prevent the model from relying more on spurious features make predictions the spurious correlations in the training data increase. Consequently, the model trained with MAT or LDAT will generalize better on OOD data.
Notations. Let X' denote the random data in the input space X' and let y E y denote the target random data in label space y. For simplicity, let y E (1,-1} in this section. Let denote an underlying class of distributions over X' x ) Let Vine and Tep denote the invariant features and the spurious features respectively. Also for simplicity, assume that there exists an identity mapping &: Xin Xap + X such that each D e ID is induced by distribution over Xmw Xap (so I can be denoted as x = (Nines x)))). Let Tap take values in {+3,- -B} for some > 0.
A Simple OOD Task. Consider a simple OOD task where we have two training domains representing the grass and desert backgrounds respectively. Both domains have two classes: the cow class and the camel class. In the grass/desert domain, the cow/camel class predominates. During test time, the correlation between the labels and the background flips. We can abstract this cow-camel dataset into the following model: training dataset S with four groups of data points drawn from the four
quadrants of the feature space +1) x {-8, +33 respectively (shown in Figure 3). We set the invariant features Zine y and the spurious features sa to be y with probability p [0.5, 1) and

---

-y with probability 1 Note that measures the intensity of spurious correlations in a certain environment. When p = 0.5, there are no correlations between the labels and the spurious features.
Consider a linear classifier 1(x) = Wing + Wpp Following 19). let consider MAT/LDDAT trained with gradient descent algorithm stopped in finite time t. In order characterize the dependence of the model on spurious fea- tures during the training process, we investigate the conver- (1)
Spurious
Grass
gence rate of We'ine (1)).e))) to 0 on the above dataset, which denotes the ratio between the output of the spurious compo- nent to that of the invariant component. We prove that after adding the domain-wise perturbations in finite-time- time-stopped gradient descent, the lower bound of the convergence rate of this ratio does not increase monotonically with p. Hence, the model will not learn a large prediction weight based on spurious features even if the spurious correlation strong (p is large).
Desert +
Invariant
Camels
Cows
Figure 3: Illustration of the simple OOD task.
In the following theorem, we denote the domain-wise per- turbation in MAT/LDAT as 8. Theorem 4.] applliss to both MAT and LDAT since they both use domain-wiss -wise perturbations. See Appendix A for a formal statement and full proof of Theorem 1.1
Theorem 4.1. (informal) Let H be the set of linear classifiers h(z) = Wine (t) + Wsp(t) Consider the above 2-D OOD dataset S. Assume that the empirical distribution of Iane given Esp > 0 is identical to the empirical distribution of Tine given Tsp < 0. is the optimal perturbation obtained by optimizing the object in Eq. 7 or Eq. (9. Let Wine (t)areee + Wsp Wsp(L)o (1) be initialized to the origin, and trained with MAT/LDAT to minimize the exponential loss on S. Then, for any (x,y) E S, we have:
In
B+oy
02 p2 "1 (1-p))
Wap (t)
((
S
(10)
M In(t +1)
Wine (f)
where == by is real number close to O,
2(2M(1+6)-1)
2(2M(1+5)11)
(84
C2 88 ((--8))*
M max d x denotes the maximum value of the margin of the max-margin classifier ú on zES
S. ((() is the lower bound of a given function within a constant factor. Therefore, the lower bound of the convergence rate does not increase monotonically with p under the condition that 2cc1 + C2 + + 3 O.
To sum up, since can prevent this lower bound from growing monotonically with p, we accelerate (t)8
the convergence rate of Twnn (C)asne]
to 0 when there is stronger spurious correlation (larger p). Recall
that the ratio (f) reflects the degree of reliance on spurious features. Therefore, faster convergence of this ratio to (smaller lower bound) means that the model will end up relying less on the spurious correlations within finite training time. In other words, the OOD robustness can be enhanced by using domain-wise perturbations.
(1)
Remark. Here, we demonstrate that MAT and LDAT show stronger OOD robustness compared
to ERM. We compare the result in Theorem 4. to that in Theorem 2 of 22 The full statement of the Theorem [2] is in Appendix B According to the Theorem in 22, even if the max-margin classifier does not rely on Tsp for any level of spurious correlation E [0.5, 1), ERM trained by gradient descent stopped in finite time still fails to avoid using spurious features. Moreover, when conducting ERM with finite-iime-sopper gradient descent, the lower bound of the convergence rate
of (t)
to MFrnco (
In Ω
C+V p(I-p)
Wsp (1)
(11)
M Int
Wwnee Finel
where :: 2(2M 1) and M follows the definition in Theorem 4.1 This lower bound grows 3" monotonically with p. thus ERM will have slower convergence for larger spurious correlations.
7

---

However, with domain-wise perturbations, we can modify the lower bound so that it does not increase monotonically with the spurious correlation p. Thus, we can draw the conclusion that using perturbation for each domain is helpful to reduce dependence on spurious features compared to ERM.
5
Experiments
5.1 Experimental Setup
We conduct experiments on the DomainBed benchmark 19] a testbed for OOD generalization that implements consistent experimental protocols across various approaches ensure fair comparisons. We evaluate on PACS [14,, OfficeHome [[5], VLCS [[)), NICO [[7]. and Colored MNIST M.
There are several changes our experimentation setting comparing DomainBed:
1. Backhone Network. We use R8 20 as our backbone for datasets excluding Colored MNIST instead of ResNet-50 used in 2 for efficiency.
2. Hyperparameter Search Space. We use smaller hyperparameter search space than g We conduct random search of trials for PACS, OfficeHome, and VLCS while 6 trials for NICO and Colored MNIST in the hyperparameter search space, instead of 20 trials adopted in 9 for fcasibility. See Appendix C.2 for more details.
Model Selection Strategy. Since hyperparameter choice has a significant impact on the OOD performance, is critical to use appropriate model selection method. For PACS, OfficeHome, and VLCS datasets, we use training-domain validation proposed in 9 since it is more in line with the OOD scenario. For NICO, we adopt OOD validation following 21 For Colored MNIST, we use test-domain validation [ since it can enlarge the gaps in OOD performance among the algorithms while the gap induced by training-domait validation on Colored MNIST is marginal.
Hyperparameters for MAT and LDAT. To retain low-rank structures in perturbations, we set the upper bound of the search space of the perturbation number in MAT to be 20. Similarly, the upper bound of the rank of the perturbation used in LDAT / is 20. Specifically, the search space of k and / f5, 10, 15, 20} (except on CMNIST, where [5, 20] and e [10, 20)). The complete setup of the hyperparameters for MAT and LDAT is provided in Appendix C.2
5.2 OOD Performance on Benchmark datasets.
Table summarizee the results the five OOD datasets. The results of other approaches for PACS, OfficeHome, NICO, and Colored MNIST datasets are adopted from [21]. The results on VLCS of other algorithms missing (denoted "-") because [2]] does not experiment this dataset.
Comparison with ERM and Sample-wise AT. From Table we observe that both MAT and LDAT outperform ERM (on both runs and the results in (11 and AT on average. In particular, MAT achicves consistently better results than ERM on all five datasets. Additionally, the average performance of AT is worse than ERM, which is consistent with our observations in Section 3.2
Comparison with Existing State-of-the-AAt Approaches. Although the results from 21 use different training protocol from ours: they use larger search space and 20 random search for the hyperparameter combinations, the comparison between ERM ((II) and ERM (our runs) indicates that their corresponding performances are close. A similar comparison has been made in (22]. We find that MAT outperforms all previous algorithms and LDAT ranked fourth among all methods, merely after VREx 23 and IRM [1] (see avg' in Table 2}. And even when excluding Colored MNIST (toy example), our methods still outperform ERM by 0.4 ~ 1.4%, whereas other methods show improvement over ERM (see avg"). From these results, our proposed methods higher than the other works, and our methods clearly outperform ERM. We also extend our evaluation to compare with adversarial augmentation based method [5] in Appendix Single-training domain generalization experiments are shown in Appendix C.4 which shows that methods can maintain OOD performance without the reliance multi-source training data.
see that the promotion of
Comparison between MAT and LDAT. From Table 2 we can see that MAT outperforms LDAT on average. Since LDAT reduces the number of the perturbations used in domain from & to (shown in Figure 2 LDAT can be regarded as a low-rank version of MAT, This indicates that the oversimplified perturbations may be less effective than the ones maintaining some flexibility.
8

---

Table 2: Test accuracy (%) on OOD datasets within DomainBed benchmark using ResNet-18. Here "avg"" denotes the average accuracy on PACS, OfficeHome, NICO, CMNIST datasets and "avg?" denotes the average accuracy on all five datasets. "avg"" denotes the average accuracy on the other three datasets except for CMNIST and VLCS. "avg*" denotes the average accuracy on the other four datasets except for CMNIST. The best results are in bold.
Datasets
Algorithm
PACS
OfficeHome
VLCS
NICO
CMNIST
avg'
avg"
avg"
avg*
ERM (Our runs)
81.7 + 0.3
62.1 + 0.1 62.1
74.4 = 1.0
73.2 1.9
28.1 + 1.5
61.3
63.9
72.3
72.9
AT (Our runs)
81.5 + 0.3
0.3
76,2 0.3
69.7 +
29.1 1.5
60.6
63.7
71.1
72.4
ERM
0.0
63.3 =
71.4
0.1
61.5
72.1
2SC
1..84.0.4
62.9
0.3
2.66
61.0
71.8
MMD
0.2
6.1
6.
5..7
SagNei
0.
1.0
3.77
CORAL
63.8 8.66
68.3
3..0
610
71.2
R
0.3
0.2
67.6
60,2
68.0
70.6
VREs 23
81.8
0.1
71.0 1.3
1.
68.2
72.1
GroupDRO 28
8.34
632
7.8
32.5
62.0
71.8
DANN
8.41
62.9
68.6+1JJ
24.5
59.3
70.9
MTL RD
8.
0.2
0.6
0.1
Mixup 31
0.6
0.5
6.
2.8
69.9
ANDMask 32
0.0
0.3
7.2
27.2 1.4
60.2
71.2
MLDG
0.4
52.4
51.6 6.1
32.7 1.1
52.4
59.0
MAT (Our work)
0.5
64.5
1 2.1
74.6 L 0.8
65.418.. 8.1
71.6
72.2
73.7
73.9
LDAT (Our work)
82.6 1.6.0.5
61.0 + 0.9
75.3 +
0.3
74.4 1.6
52.5 1..5::5. 5.4
67.6
69.1
72.7
73.3
Origin
ERM
AT
MAT
LDAT
Figure 4: The pixel attention heatmap of ERM, AT, MAT and LDAT on NICO dataset. The redder part indicates that the model relies more this part make predictions.
Since both MAT and LDAT outperform most existing state-of-the-an methods and they both exploit low-rank structures, these two methods mutually corroborate the effectiveness of low-rank structure OOD generalization. The respective advantages of the two methods are as follows:
MAT is a complex and high-rank version of LDAT, which has stronger ability to describe more complex spurious background information. As shown by the attention heatmap in Figure MAT can better capture the object than LDAT when faced with a more complex background (the example in the second row of Figure
LDAT costs less memory than MAT during the training process, although there is no significant difference in training time between the two methods. When the memory is limited, LDAT preferred.

---

5.3 Empirical Understanding
Visualization. To empirically show that MAT and LDAT can reduce the reliance on spurious features. we visualize the pixel attention heatmap of ERM, AT. MAT, and LDAT on NICO dataset using GradCam [34]. reflects the contribution of different components of the feature map to the prediction results. We pick the model with the best performance for each method. The results in Figure 4 indicate that the model trained by MAT and LDAT focuses more on the object itself, while ERM and AT adopt the background information that spuriously correlates to the class to make predictions.
Parameter Analysis. The number of the perturbations k used in domain in MAT and the rank of the perturbation in LDAT are two key hyperparameters. We conduct further experiments to analyze the impact on the performances of k and l. We adopt a fixed set of parameters except for and and evaluate on PACS dataset. The results in Figure 5 show that MAT and LDAT are able to keep their performances over ERM as long lt and are far less than the number of the samples N. Additionally, we can observe from the trend that when k and I are too small (= 5), the performances degenerate. This implies the oversimplified structures of the perturbations can less effective for generalization. When k and I take larger value (about 1000), the performance will drop (see Table 3 For the selection of optimal parameters (range), we observe that the parameters that are good on one dataset also work well on others (see Table o the appendix), so in practice we adopt the strategy of searching for the optimal parameters roughly on one dataset and then applying them to other datasets. Additional analysis the impact of the learning rate for the perturbations is in Appendix C.2
M/0
number perturattooe. ((ATT) mak DAT)
Figure 5: The performance different values of the number of perturbations to combined within one domain l: (in MAT) and the rank of the perturbation in a domain (in LDAT).
Table 3: The test accuracy (%) CMNIST with different k and I.
Algorithm
k E [5, 20),7 / [10,20]
or 200
or 500
or 1000
MAT
65.4 + 8.1
34.9 20.2
25.6 8.5
23.4 + 10.8
LDAT
52.5 + 5.4
24.9 + 8.9
19.0 + 6.6
10.3 + 0.1
6 Conclusion
In this work, we empirically revealed the limitations of sample-wise AT on OOD tasks. Due to the lack of constraints on the perturbation and the utilization of domain features, sample-wise AT fails to generalize well when facing large-scale perturbations which is close to the real-world OOD scenarios. We further proposed two AT variants with structured priors, named MAT and LDAT, which add low-rank perturbations to improve model's robustness against the distribution shift of spurious correlations. We theoretically proved the domain- wise perturbations used in MAT and LDAT can benefit OOD generalization, and validated the effectiveness of the proposed methods on OOD tasks through a series of experiments on Domainbed benchmark.
Acknowledgment
Qixun Wang partially supported by the State Key Development Program Grand (No. 2020YFB CFFFB17080 1080022). Yisen Wang is partially supported by the NSF China (No. 62006153). Project 2020BD006 supported by PKU-Baidu Fund, Open Research Projects of Zhejiang Lab (No. 2022RC0AB05), and Huawei Technologies Inc
10

---

References
[1] Martin Arjovsky, Léon Bottou, Ishaan Guirajani, and David Lopez-Paz. Invariant risk mini- mization. In ICML, 2020.
[2] Vaishnavh Nagarajan, Anders Andreassen, and Behnam Neyshabur. Understanding the failure modes of out-of-distribttion generalization. arXiv preprint arXiv:2010. 15775, 2020.
[3] Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, lan Goodfel- low, and Rob Fergus. Intriguing properties of neural networks. arXiv preprint 131261999 2013.
[4] Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. arXiv preprint arXiv: (ttl::706.005555 2017.
[5] Riccardo Volpi, Hongseok Namkoong. Ozan Sener, John Duchi, Vittorio Murino, and Silvio Savarese. Generalizing to unseen domains via adversarial data augmentation. arXiv preprint arXiv:1805.1200 12018, 2018.
[6] Mingyang Yi, Lu Hou, Jiacheng Sun, Lifeng Shang, Xin Jiang, Qun Liu, and Zhiming Ma. Improved ood generalization via adversarial training and pretraing. In ICML, 2021.
[7] Ludwig Schmidt, Shibani Santurkar, Dimitris Tsipras, Kunal Talwar, and Aleksander Madry. Adversarially robust generalization requires more data. In NeurIPS, 2018.
[8] Seyed-Mohsen Moosavi-Dezfooli, Alhussein Fawzi, Omar Fawzi, and Pascal Frossard. Univer- sal adversarial perturbations. In CVPR, 2017.
[9] Ishaan Gulrajani and David Lopez-Paz. In search lost domain generalization. arXiv preprint arXiv:2007.01434, 2020.
[10] Yisen Wang, Xingjun Ma, James Bailey, Jinfeng Yi, Bowen Zhou, and Quanquan Gu. On the convergence and robustness adversarial training. In ICML, 2019.
[11] Dongxian Wu, Shu-Tao Xia, and Yisen Wang. Adversarial weight perturbation helps robust generalization. In NeurlPS, 2020.
[12] Charles Herrmann, Kyle Sargent, Lu Jiang, Ramin Zabih, Huiwen Chang, Ce Liu, Dilip Krishnan, and Deqing Sun. Pyramid adversarial training improves vit performance. arXiv preprint arXiv:21 2021.
[13] J. Goodfellow, J. Shlens, and C. Szegedy. Explaining and harnessing adversarial examples. Computer Science, 2014.
[14] Da Li. Yongxin Yang, Yi-Zhe Song, and Timothy M Hospedales. Deeper, broader and artier domain generalization. In ICCV, 2017.
[15] Hemanth Venkateswara. Jose Eusebio, Shayok Chakraborty, and Sethuraman Panchanathan.
Deep hashing network for unsupervised domain adaptation. In CVPR, 2017.
[16] Chen Fang, Ye Xu, and Daniel N Rockmore. Unbiased metric learning: On the utilization of multiple datasets and web images for softening bias. In ICCV, 2013.
[17] Yue He, Zheyan Shen, and Peng Cui. Towards non-iid image classification: A dataset and baselines. Pattern Recognition, 110:107383, 2021.
[18] Klim Kireev, Maksym Andriushchenko, and Nicolas Flammarion. On the effectiveness of adversarial training against common corruptions. arXiv preprint arXiv:2103.02325, 2021.
[19] Daniel Soudry, Elad Hoffer, Mor Shpigel Nacson, Suriya Gunasekar, and Nathan Srebro. The implicit bias of gradient descent separable data. The Journal of Machine Learning Research, 2018.
[20] Shaoqing Ren, Jian Sun, He, and Zhang. Deep residual learning for image recognition. In CVPR, 2016.
11

---

[21] Nanyang Ye, Kaican Li. Lanqing Hong, Haoyue Bai, Yiting Chen, Fengwei Zhou, and Zhenguo Li. Ood-bench: Benchmarking and understanding out-of-distributoo generalization datasets 2021.
and algorithms. arXiv preprint arXiv:2106.03333
[22] Devansh Arpit, Huan Wang, Yingbo Zhou, and Caiming Xiong. Ensemble of averages: Im- proving model selection and boosting performance in domain generalization. arXiv preprint 11Xi:22110110822 2021.
[23] David Krueger, Ethan Caballero, foem-Henrik Jacobsen, Amy Zhang, Jonathan Binas, Dinghuai Zhang, Remi Le Priol, and Aaron Courville. Out-of-distribttion generalization via risk extrapo- lation (rex). In ICML, 2021.
[24] Zeyi Huang, Haohan Wang, Eric P Xing, and Dong Huang. Self-challenging improves cross- domain generalization. In ECCV, 2020.
[25] Haoliang Li, Sinno Jialin Pan, Shiqi Wang, and Alex C Kot. Domain generalization with adversarial feature learning. In CVPR, 2018.
[26] Hyconseob Nam, HyunJae Lee, Jongchan Park, Wonjun Yoon, and Donggeun Yoo. Reducing domain gap via style-agnostic networks. arXiv preprint 1rXir:1910..1645, 2(7):8, 2019.
[27] Baochen Sun and Kate Saenko. Deep coral: Correlation alignment for deep domain adaptation. In ECCV, 2016.
[28] Shiori Sagawa, Pang Wei Koh, Tatsunori B Hashimoto, and Percy Liang. Distributionally robust neural networks. In ICLR, 2019.
[29] Yaroslav Ganin, Evgeniya Ustinova, Hana Ajakan, Pascal Germain, Hugo Larochelle, François Laviolette, Mario Marchand, and Victor Lempitsky. Domain-adversaria training of neural networks. The Journal of Machine Learning Research, 17(1):209--0030 2016.
[30] Gilles Blanchard, Aniket Anand Deshmukh, Urun Dogan, Gyemin Lee, and Clayton Scott. Domain generalization by marginal transfer learning. arXiv preprint arXiv:1711.07910. 2017.
[31] Shen Yan, Huan Song, Nanxiang Li, Lincan Zou, and Liu Ren. Improve unsupervised domain adaptation with mixup training. arXiv preprint arXiv:2001.0066 2020.
[32] Giambattista Parascandolo, Alexander Neitz, Antonio Orvieto, Luigi Gresele, and Bernhard Schölkopf. Learning explanations that are hard to vary. arXiv preprint arXiv:2009.00329, 2020.
[33] Da Li, Yongxin Yang. Yi-Zhe Song, and Timothy M Hospedales. Learning to generalize: Meta-learning for domain generalization. In AAAI, 2018.
[34] Jacob Gildenblat and contributors. Pytorch library for cam methods. https://github.cm com/ jacobgil/pytorch grad cam 2021.
[35] Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, al. Imagenet large scale visual recognition challenge. International Journal of Computer Vision, 115(3):211-255 2015.
[36] Chris Xing Tian, Haoliang Li, Xiaofei Xie, Yang Liu, and Shiqi Wang. Neuron coverage -uideddd domain generalization. JEEE Transactions on Pattern Analysis and Machine Intelligence, 2022.
12

---

Checklist
The checklist follows the references. Please read the checklist guidelines carefully for information on how to answer these questions. For each question, change the default [TODO] to [Yes], [No], , or [N/A] You strongly encouraged to include justification to your answer, either by referencing the appropriate section of your paper or providing a brief inline description. For example:
Did you include the license to the code and datasets? [Yes] See Section.
Did you include the license to the code and datasets? [No] The code and the data are proprietary.
Did you include the license to the code and datasets? [N/A]
Please do not modify the questions and only use the provided macros for your answers. Note that the Checklist section does not count towards the page limit. In your paper, please delete this instructions block and only keep the Checklist section heading above along with the questions/answers below.
1. For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope? [Yes] See Section [
(b) Did you describe the limitations of your work? [Yes] See Section 5.2 We mention that the oversimplified perturbation (LDAT) will be less effective than the ones with more flexibility (MAT).
(c) Did you discuss any potential negative societal impacts of your work? [No]
(d) Have you read the ethics review guidelines and ensured that your paper conforms to them? [Yes]
2. If you including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results? [Yes] Full assump- tions
Appendix [A]
(b) Did you include complete proofs of all theoretical results? [Yes] See Appendix A
3. If you ran experiments..
(a) Did you include the code, data, and instructions needed to reproduce the main experi- mental results (either in the supplemental material or as URL)? [No] It will release
upon acceptance.
(b) Did you specify all the training details (e.g.. data splits, hyperparameters, how they were chosen)? [Yes] See Section C.I and C.2
(c) Did you report error bars (c.g., with respect to the random seed after running experi-
ments multiple times)? [Yes] See Table 2 for an example.
(d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)? [No]
4. If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...
(a) If your work uses existing assets, did you cite the creators? [Yes] We use DomainBed benchmark and GradCam and cite their creators.
(b) Did you mention the license of the assets? [No] All assets we use are open source.
(c) Did you include any new assets either the supplemental material or as URL.? [No]
(d) Did you discuss whether and how consent was obtained from people whose data you' 're
using/curating? [No] All datasets we use are open source.
(c) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content? [No]
5. If you used crowdsourcing conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, applicable? [No]
(b) Did you describe any potential participant risks, with links to Institutional Review
Board (IRB) approvals, applicable? [No]
(c) Did you include the estimated hourly wage paid participants and the total amount
spent participant compensation? [No]
13


Reviews:
[{"review_id": "zLVvDOsZFp3", "reviewer": "Reviewer_mRzf", "summary": "This paper proposes to utilize adversarial training to improve the test accuracy on the OOD dataset. They present the low-rank perturbations to perform adversarial training, inspired by the  better performance of universal adversarial training. Experiments on benchmark datasets veriy the effectiveness.", "questions": "The aussmption of low-rank structure should be detailed discussed in the rebuttal. ", "limitations": "The aussmption of low-rank structure should be detailed discussed in the rebuttal. ", "rating": 6, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n1. Adversarial attack for good is an interesting topic. In this paper, it utilizes the adversarial training to improve test accuracy of OOD dataset. I think this topic should be encouraged. \n2. The paper gives an empirical and theoretical demonstration for the proposed method. I think it is good\n3. Experiments compared with many SOTA methods verify the better performance. \n\nWeakness:\n1. My main concern lies in the assumption that because the OOD dataset (or universal perturbation) usually shows low-rank structure, inspired by this, the authors propose the low-rank adversarial perturbations to perform attacks. Actually, in the real images, many scenes are not low-rank. Therefore, I don't know whether the proposed method still works in this situation. Authors should give some examples and discuss it. \n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ["Privacy and Security (e.g., consent)"]}, "extra_text": {}}, {"review_id": "rCQjG9PT9uc", "reviewer": "Reviewer_RPCG", "summary": "The paper highlights the severe performance degradation of sample-wise AT  in OOD tasks due to the existence of much larger perturbations in OOD tasks. And the paper shows a theoretical perspective to the stable generalization performance of universal AT, discovering that low-dimensional domain-wise perturbation benefits removing the low-rank structured background information. Inspired by this, the paper proposes two specific constructions of low-rank structured perturbations along the sample number and input scale dimension respectively, and theoretically proves their enhancements to the OOD robustness using a simple OOD task. To verify the approaches, experiments on five benchmark datasets are conducted to show the OOD performance of proposed MAT and LDAT compared to existing algorithms.", "questions": "nil", "limitations": "nil", "rating": 5, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n\n1.  The paper provides an empirical insight into the limitations of sample-wise AT on OOD tasks, and highlights the importance of domain-wise AT. Then, two effective low-rank based domain-wise AT methods are proposed. Superior performance is observed on the OOD generalization ability comparing to baselines.\n\n2.  The paper reveals some relationship between adversarial training and OOD generalization problem theoretically, allowing for deeper dive into the AT theory.\n\n\n\nWeaknesses:\n\n\n1. The theoretical results that reveal the low-rank perturbation can prevent the model from learning spurious features is obscure. It is better to show the results in a more friendly way. \n\n2. Some experimental results are not convincing. For instance, in Table 2, if not considering the colored MNIST datasets, the performance of the proposed model is on par with ERM. Without considering colored MNIST, the averaged gain is very limited. ", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "nFgQUNyhHt8", "reviewer": "Reviewer_VkPQ", "summary": "In this submission, the authors proposed to focus on the problem of domain generalization based on adversarial training. Unlike existing adversarial training methods which inject sample-wise adversarial noise, a novel universal adversarial perturbation generating mechanism (with two variant) was proposed in this paper. Experimental results show the effectiveness of the proposed MAT and LDAT on domainbed benchmark. ", "questions": "Please see above. \n", "limitations": "While the limitation of existing adversarial training has been discussed, the limitation of the proposed method is missed. ", "rating": 7, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "post rebuttal\n\nMy concerns have been addressed. \n\n\nStrength:\n\n-\tThe proposed method is quite novel and interesting. \n\n-\tThe motivation and analysis of the proposed method is strong. \n\n-\tExperimental results on benchmark justify the effectiveness of the proposed method.\n\nWeakness:\n\n-\tMy main concern is on the evaluation part. It seems that except Colored MNIST (which is quite toy), the improvement on other datasets are very limited. Moreover, the only baseline using data augmentation for comparison are conventional AT and Mixup, other adversarial augmentation for DG (e.g., GUT [Volpi, NIPS’18]) and conventional data augmentation methods (which have been shown to be quite effective [French, ICLR’18], [Tian, TPAMI’22]) should be discussed.\n\n-\t In Fig. 4, it seems that the number of pertubations adopted between MAT and LDAT is not consistent. I am wondering if there are some guidance to choose the optimal number given source domains and model?\n\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "PYTjBFiMJi", "reviewer": "Reviewer_BPEE", "summary": "This paper introduces two methods for universal adversarial attack (as opposed to sample-wise) in order to improve the OOD-generalization of image classification CNN models. It also shows theoretically how spurious feature use is reduced through UAT as well as showing empirically how these UAT methods perform well on a variety of benchmarks.", "questions": "I am a bit confused as to how the parameters in the MAT/LDAT matrices are selected. Supplement algorithms 1 and 2 seem quite crucial to the function of the proposed method, and at least one of them should be in the main text. Could you give me a plaintext overview for how one of the algorithms works?", "limitations": "The authors do not mention potential negative impacts. A small paragraph could be included to describe how classifiers can be misused.\n\nThe weaknesses described above are not particularly referenced either.", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n\nThe proposed method is novel, interesting, and relatively simple.\n\nThe paper is generally well written and easy to follow. I think the theoretical analysis is of interest to the broader community on how to apply UAT style training.\n\nThe experimental setup is comprehensive.\n\n\nWeaknesses:\n\nUnfortunately, the empirical results do not show a sufficient improvement over other baseline methods. The inclusion of colored mnist seriously alters the average, which is to be expected as the proposed method directly attacks only the color channels. When k=20, that's (in some sense) 20 different colors to be applied to each image. I do not find the colored mnist dataset to be generally insightful, therefore when I look at the other datasets, there is not a consistent improvement over other baselines. Normal AT, ERM, MAT and LDAT each get fairly good results (and often not outside the error margin). It is not clear to me when I should use MAT, LDAT or AT.\n\nMinor:\nFigure 2 has some typos (LRAT), and is a bit hard to parse. Having the blue arrow represent the entire NxN image is not intuitive. Perhaps turning the flat squares into 3d shapes would help.\n\n\nPost rebuttal:\nThanks authors. While I would preferred LDAT or MAT to have a clear choice when to use one or the other, the rebuttal is sufficient reasoning. Overall, I agree with reviewer VkPQ, I have upgraded my score from 3 to 6.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "DTMxa0xaER8", "reviewer": "Reviewer_Nziv", "summary": "This paper proposes two modified methods for adversarial training to improve robustness. The noise being added is constrained to low dimensional subspace, and it further changes to more delicate constraints.", "questions": "In line 245, if theorem 2 of [2] is so important, it should be explicitly written here. If the theoretical result is better than [2], then it should be specified what changes make the result better, i.e. subspace constraints, or the improvement in technical details to make inequalities tighter.\n\nthe paragraph from 202, if this is going to be a concrete example with \"In the grass/desert domain, the cow/camel class predominates\" specified, it might be better based on an image example showed in the paper, or not to be specified at all. I feel these examples come out of nowhere and leave for nowhere.", "limitations": "yes.", "rating": 4, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strength: the paper proposed a reasonable modification as there might be some subspace or clusters of pixels that are more irrelevant, so that they could be treated differently in adversarial training. \n\nIt also includes a theory analysis with simple example.\n\nFigure 3 provides an intuitive explanation.\n\nweakness: The adversarial training overall seem to be very unstable, whether its sample-wise in figure1, or on OfficeHome in table 2, and in figure 4. The rank of the perturbation doesn't show a reasonable correlation with test accuracy.\n\nThe theory analysis is based on a simple binary classification, which might be intuitive. But the theorem proved here, I cannot find the role of this subspace of rank K here, which should be important.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}]

Author rebuttal:

Decision: Accept