Improving Out-of-distribution Robustness by Adversarial Training with Structured Priors
Anonymous Author(s)
AffiliationAddress email
Abstract
Deep models often fail to generalize well in test domains when the data distributiondiffers from that in the training domain.
Among numerous approaches to addressthis Out-of-Distribution (OOD) generalization problem, there has been a growing surge of interest in exploiting the input-robustness obtained by Adversarial
Training (AT) to improve OOD performances.
Recent works have revealed that therobust model obtained by conducting sample-wise AT also retains transferabilityto biased test domains.
In this paper, we empirically show that sample-wise AThas limited improvement on OOD performance.
Specifically, we find that AT canonly maintain performance at smaller scales of perturbation while Universal AT (UAT) are more robust to larger-scale perturbations.
This provides us with clues that the adversarial perturbations with universal (low dimensional) structures can enhance the robustness to large data distribution shifts which are common in OODscenarios.
Inspired by this, we propose two AT variants with low-rank structures totrain OOD-robust models.
Extensive experiments on DomainBed benchmark showthat our proposed approaches outperform Empirical Risk Minimization (ERM) andsample-wise AT.
Introduction
Existing deep learning methods have achieved good performance on visual classification tasks underthe same distribution of training sets and test sets.
However, when the data distribution of the test setis different from that of the training set, the classification performance of the deep neural networks(DNNs) may decrease sharply [1].
This is mainly because the network may capture spurious featuressuch as the background and style information to assist the fast fitting during the training process [2].
However, in real-world scenarios, test data may differ from training data in the background and styleinformation, thus DNNs that use unstable spurious features to make predictions will fail.
Solving theabove problem is known as the out-of-distribution (OOD) generalization.
Another scenario where DNNs may fail is that deep models are often vulnerable to adversarialexamples [3].
Adversarial training (AT) is originally proposed as an effective way to defend againstadversarial attacks [4].
Moreover, there is work showing that adversarial training helps to solve the OOD generalization problem because OOD data can be seen as stronger perturbations to some extent[5].
The reason why AT can defend adversarial attacks meanwhile benefits OOD generalizationis that it can make the network robust to the interference of spurious features, such as randomlyinjected noise (in adversarial examples) or the spurious correlation between labels and backgroundinformation (in OOD generalization).
In other words, AT enables the network to make predictionsusing intrinsic features rather than spurious features.
A potential problem, however, is that existing AT methods for solving OOD generalization ignorethe specific design of perturbations.
They usually simply conduct sample-wise AT [6], which only brings limited performance improvement to OOD generalization.
The essential reason for the failureof this type of approach is that the perturbations it uses cannot distinguish between invariant andspurious features.
As a result, it improves the robustness at the expense of the decreasing standardaccuracy [7].
Moreover, we empirically find that when adapting universal AT [8] to OOD problems,i.e., conducting AT with domain-wise perturbations, it will show stronger input-robustness whenfacing larger-scale perturbations comparing to sample-wise AT (see Section 3.2).
Since the sampleinjected with large-scale perturbations can be regarded as OOD samples [5], we draw inspiration fromthis phenomenon that AT with universal (low-dimensional) structures can be the key to solving OODgeneralization.
To address the above problems, we propose to use structured low-rank perturbationsrelated to domain information in AT, which can help the model to filter out background and styleinformation, thus benefiting OOD generalization.
We make the following contributions in our work:
• We identify the limitations of sample-wise AT on OOD generalization through a series ofexperiments.
To alleviate this problem, we further propose two simple but effective AT
variants with structured priors to improve OOD performances.
• We theoretically prove that our proposed structured AT approach can accelerate the conver-
gence of reliance on spurious features to 0 when using finite-time-stopped gradient descent,thus enhancing the robustness of the model against spurious correlations.•
By conducting experiments on the DomainBed benchmark [9], we demonstrate that our
algorithms outperform ERM and sample-wise AT on various OOD datasets.
Related Work
Solving OOD Generalization with AT.
According to [3], the performance of deep models issusceptible to small-scale perturbations injected in the input images, even if these perturbationsare imperceptible to humans.
Adversarial training (AT) [4] is an effective approach to improvethe robustness to input perturbations.
However, many of the recent works have begun to focus onthe connection between AT and OOD due to the fact that OOD data can be regard as one kind oflarge-scale perturbations.
These works seek to exploit the robustness provided by AT to improvethe OOD generalization.
For instance, [6] applies sample-wise AT to OOD generalization.
Theytheoretically find that if a model is robust to input perturbation on training samples, it also generalizeswell on OOD data.
[5] theoretically establishes a link between the objective of AT and the OODrobustness.
They reveal that the AT procedure can be regarded as a heuristic solution to the worst-caseproblem around the training domain distribution.
Nevertheless, the discussion of [6] and [5] isrestricted to the framework of using Wasserstein distance to measure the distribution shift, which isless practical for the real-world OOD setting where domain shifts are diverse.
Additionally, they onlystudy the case of sample-wise AT and did not further investigate the effect of different forms of AT(not sample-wise) on the robustness performance of OOD.
Other works such as [10] focus on thestructure design of the perturbations.
In [10] they use multi-scale perturbations within one picture.
But they do not exploit the universal information within one training domain.
In our work, we focuson real-world OOD scenarios where there are additional clues lying in the distribution shifts, i.e, thelow-rank structures in the spurious features (such as background and style information) across onedomain.
We further design a low-rank structure in the perturbations to specifically eliminate suchlow-rank spurious correlations.
OOD Evaluation Benchmark.
The DomainBed benchmark [9] provides a fair way of evaluatingdifferent state-of-the-art OOD methods, which has been widely accepted by the community.
Byconducting rigorous experiments in a consistent setting, [9] reveals that many algorithms that claimto outperform previous methods cannot even outperform ERM.
Unlike previous works using AT toaddress OOD generalization, such as [6] and [5], in this work we adopt the Domainbed benchmarkfor a fair comparison of our approach with existing state-of-the-art methods.
Weakness of Sample-wise AT for OOD Generalization3.1 Preliminaries
Out-of-distribution (OOD) Generalization .
Let x ∈ X denote the random data in the input space
X and let y ∈ Y denote the target random data in label space Y . ϕ :
X → Z denotes the feature
[Table]
extractor. We have the predictor f = w ◦ ϕ
( x ) where z = ϕ ( x ) is the feature variable in the featurespace Z . w : Z → Y denotes the classifier.
Now we give the formal definition of the OOD generalization problem.
We have a set of mtraining domains E = { E 1 , E 2 , ..., E m } , where each domain E e is characterized by a input dataset E e := { ( x ei , y ei ) } n e i =
1 containing n e i.i.d input samples drawn from the distribution of P e , and atest domain E m +1 with data following the distribution of P te , where P te ̸
= P i , i = 1 , 2 , ..., m .
L : X → R + denotes the loss function.
The ultimate goal of OOD generalization is to find an optimalpredictor f that minimizes the risk on the unseen test domain:
[Equation]
Adversarial Training (AT) 1 .
According to [4], AT can be expressed as the following optimizationproblem:
[Equation]
where δ ∈ S is the random injected perturbation with l p norm bounded by ϵ > 0 .
The innermaximization problem can be optimized by fast gradient sign method (FGSM [11]), a simple one-stepscheme:
[Equation]
where sgn ( · ) is the sign function, or by projected gradient descent (PGD [4]), a more powerfulmulti-step variant:
[Equation]
where (cid:81) S is the projection operator onto the set S , γ is the step size and t denotes the iteration.
3.2 Weakness of AT for OOD Generalization
We now highlight some weaknesses of sample-wise AT for OOD generalization based on series ofempirical evidence.
We first conduct a toy experiment on the DomainBed benchmark [9] to evaluatethe OOD performance of AT.
We run ERM and AT on four OOD datasets: PACS [12], OfficeHome[13], VLCS [14], and NICO [15] with a fixed set of hyperparamters (Detailed experimental settingcan be found in Appendix C.1).
The results are shown in Table 1.
We can see that the improvementof OOD performance by AT is limited with an average improvement of only 0.1%.
We further investigate the reason behind the limitations of performance improvements on OODdatasets of AT.
Although previous works have revealed that the robust features obtained by AT canimprove OOD generalization ([6] [5] [16]), we find that sample-wise AT only tolerates small-scaleperturbations.
Thus, we design an experiment on NICO dataset with multiple scales of perturbations.
The scale is calculated with l 2 norm of the perturbation matrix.
The experiment details are shown in
Appendix C.1.
We show that AT will suffer severe performance degradation when using large perturbations (shown in Figure 1).
This provides clues to understanding the failure of AT in OOD scenarios.
The distributionshifts in OOD data usually have much larger scales than the invisible perturbations commonly used in
AT [5].
Hence, AT methods designed for small perturbations cannot handle these large-scale domainshifts that often appear in OOD data.
However, our experiment shows that we can alleviate this problem by adapting universal AT (UAT, [8]) to the OOD setting, i.e., using a perturbation for eachdomain.
[Figure] Figure 1 shows that UAT will keep its generalization performance when the perturbation scale islarge.
There are two empirical explanations forthis: first, the background and style informationusually have a low-rank structure, such as the grassland and snowfield that have recurring parts.
Second, similar spurious features often appear withinone specific domain, such as PACS [12] and VLCS[14] datasets.
As stated in [8], the universal perturbation lies in a low dimensional space.
Henceusing universal (domain-wise) perturbations willhelp to resist such low-rank shifts and improve therobustness of the model.
Inspired by this, we proposed two new AT algorithms with more sophisticated low-rank structureson different dimensions to improve OOD generalization in the next section.
The Proposed Structured AT Method
In order to construct low-rank structured perturbations, we start by analyzing the structure of samplewise perturbations.
Assume that each input data x has a shape of N × N × C .
N is the size of the inputimage and C is the number of the channels.
For simplicity, we assume C = 3 .
We reparameterizethe sample-wise perturbations as a series of 2-D matrices { D 11 , D 12 , ..., D 1 m } , { D 21 , D 22 , ..., D 2 m } ,{ D 31 , D 32 , ..., D 3 m } where D ce ∈ R n e × N 2 denotes the perturbations in the e -th domain for the inputchannel c , n e is the number of the samples in domain E e , and m is the number of domains.
The i -throw of D ce represents the c -th channel of the i -th sample in domain E e (see the first column in Figure2 for illustration).
By such reparameterization, it is natural to find that there are two orientations toreduce the rank of the perturbations:
Along the dimension of the number of samples (along the red arrow in the upper left
corner of Figure 2).
This corresponds to reducing the number of the perturbations used within one domain.
Along the dimension of the input scale (along the blue arrow in the upper left corner ofFigure 2).
This corresponds to reducing the rank of the perturbation used for a specific input sample.
In the following sections, we will propose two AT variants with structured priors that reduce the rankin these two directions.
4.1 MAT: Adversarial Training with Combinations of Multiple Perturbations
In this section, we propose domain-wise Multiple-perturbation Adversarial Training (MAT).
It aimsto conduct rank minimization along the dimension of the number of samples.
Instead of usingsample-wise perturbations, MAT constructs a combination of multiple perturbations and shares thismixed perturbation within a domain.
Specifically, we choose to train the linear combination of kperturbations for each domain E e to conduct AT.
Here k is a hyperparameter and k is far less thanthe number of samples in domain E e .
The optimization problem can be reformulated as:
[Equation]
[Equation]
[Equation]
, α e ∗ i ≥ 0 for i
= 1 , 2 , ..., k, (6) Domain (cid:8) (cid:2869) … Sample-wise AT (cid:1866) (cid:2869) (cid:1830) (cid:2869) (cid:1840)(cid:3400)(cid:1840) (rank= (cid:1840) ) Domain (cid:8) (cid:2870) (cid:1866) (cid:2870) (cid:1830) (cid:2870)
Domain (cid:8) (cid:2871) (cid:1866) (cid:2871) (cid:1830) (cid:2871) … MAT (cid:1840)(cid:3400)(cid:1840) (rank= (cid:1840) ) (cid:1863) (cid:1863) (cid:1863) … LDAT (cid:1840)(cid:3400)(cid:1840) (rank= (cid:1864) )
(cid:883) (cid:883) (cid:883) …
where
[Equation]
Here e denotes the subscript of a training domain and α ei is the weight that can be learned for eachperturbation δ ei .
Detailed training procedure of MAT is in Appendix D.
MAT works as a low-rank version of sample-wise AT.
In sample-wise AT, we maintain n e perturbations for each domain E e , where n e is the number of training samples in domain E e .
As for MAT,it reduces the number of perturbations available to samples from n e to k and to obtain low-rankstructures (see the second column in Figure 2 for illustration).
Therefore, it fulfills rank reductionalong the sample-number dimension.
4.2 LDAT: Adversarial Training with Low-rank Decomposed Perturbations
Based on MAT, we further propose Adversarial Training with Low-rank Decomposed perturbations(LDAT).
Analogous to MAT, LDAT still shares one perturbation in a specific domain.
Moreover,
LDAT imposes a low-rank constraint on the perturbation itself, which corresponds to the dimension ofthe input scale.
Technically, we obtain the domain-wise low-rank perturbation matrix δ ∈ R N × N × Cby multiplying two matrices: δ = AB .
Here A ∈ R N × l × C and B ∈ R l × N × C , l is a hyperparameterand l ≪ N .
Since rank ( AB ) ≤ rank ( A ) and rank ( AB ) ≤ rank ( B ) hold for arbitrary matrices A ,
B , we have rank ( δ ) ≤ l .
Therefore LDAT reduces the rank of the perturbation from a large value Nto a relatively small value
l (see the third column in Figure 2 for illustration).
The formal definitionof the LDAT objective is:
[Equation]
where
[Equation]
Detailed training procedure of LDAT is in Appendix D. Comparison to MAT, LDAT reduces thenumber of the perturbations available to the samples in a domain from k to 1. In addition, it reducesthe rank of the perturbation for a single channel of a sample from N to l (see the third column in Figure 2).
4.3 Theoretical Analysis
In this part, we theoretically explain why the domain-wise perturbation proposed in MAT and LDATcan help to improve the robustness of the model against spurious correlations following [2] and [17].
In general, we prove that MAT and LDAT can prevent the model from relying more on the spuriousfeatures to make predictions as the spurious correlations in the training data increase.
Consequently,the model trained with MAT or LDAT will generalize better on OOD data.
Notations.
Let x ∈ X denote the random data in the input space X and let y ∈ Y denote the targetrandom data in label space Y .
For simplicity, let Y ∈ { 1 , − 1 } in this section.
Let D denote anunderlying class of distributions over X × Y .
Let x inv and x sp denote the invariant features andthe spurious features respectively.
Also for simplicity, assume that there exists an identity mappingΦ : X inv × X sp → X such that each D ∈ D is induced by a distribution over X inv × X sp (so x canbe denoted as x = ( x inv , x sp ) ).
Let x sp take values in { + β, − β } for some β > 0 . [Table]
(cid:28631) (cid:28664) (cid:28678) (cid:28664) (cid:28677) (cid:28679) (cid:28630)(cid:28660)(cid:28672)(cid:28664)(cid:28671)(cid:28678) (cid:28630)(cid:28674)(cid:28682)(cid:28678) β (cid:129)β 1 (cid:127) (cid:127) 1(cid:129)(cid:127) (cid:127) 1(cid:129)(cid:127)
A Simple OOD Task.
Consider a simple OOD task wherewe have two training domains representing the grass anddesert background respectively.
Both domains have twoclasses: the cow class and the camel class.
In the grass/desertdomain, the cow/camel class predominates.
During testtime, the correlation between the labels and the backgroundflips.
We can abstract this cow-camel dataset into the following model: a training dataset S with four groups of datapoints drawn from the four quadrants of the feature space{− 1 , +1 } × {− β, + β } respectively (shown in Fig. 3).
Weset the invariant features x inv =
y and the spurious featuresx sp to be yβ with probability p ∈
[0 . 5 , 1) and − yβ withprobability 1 − p .
Note that p measures the intensity of spurious correlations in a certain environment.
When p = 0 .
5 ,there are no correlations between the labels and the spuriousfeatures.
Consider a linear classifier h ( x ) =
w inv x inv + w sp x sp .
Following [17], let us consider MAT/LDAT trained with gradient descent algorithm stopped in finitetime t .
In order to characterize the dependence of the model on spurious features during the trainingprocess, we investigate the convergence rate of
[Equation] ( t ) x inv | to 0 on the above dataset, which denotesthe ratio between the output of the spurious component to that of the invariant component.
We willprove that after adding the domain-wise perturbations in finite-time-stopped gradient descent, thelower bound of the convergence rate of this ratio does not increase monotonically with p .
Hence,the model will not learn a large prediction weight based on spurious features even if the spuriouscorrelation is strong ( p is large).
In the following theorem, we denote the domain-wise perturbation in MAT/LDAT as δ .
Theorem 4.1applies to both MAT and LDAT since they both use domain-wise perturbations.
See Appendix A fora formal statement and full proof of Theorem 4.1.
Theorem 4.1.
(informal)
Let H be the set of linear classifiers h ( x ) =
w inv ( t ) x inv + w sp ( t ) x sp .
Consider the above 2-D OOD dataset S .
Assume that the empirical distribution of x inv givenx sp · y > 0 is identical to the empirical distribution of x inv given x sp · y < 0 .
δ is the optimalperturbation obtained by optimizing the object in Eq. (7) or Eq.
Let w inv ( t ) x inv + w sp ( t ) x spbe initialized to the origin, and trained with MAT/LDAT to minimize the exponential loss on S .
Then,for any ( x, y ) ∈ S , we have:
[Equation]
[Equation] where ϵ :
= δy 2 β is a real number close to 0, c := 2(2 M − 1) β, c 1 :=
2(2 M − 1) β ( β − δy ) 2 e winvδy , c 2 :=2(2 M − 1) β
( β 2 − δ 2 ) e winvδy ( β − δy β + δy ) β + δy 2 β .
M = max x ∈ S ˆ w · x denotes the maximum value of the margin of themax-margin classifier ˆ w on S .
Ω( · ) is the lower bound of a given function within a constant factor. Therefore, the lower bound of the convergence rate does not increase monotonically with p under thecondition that c 2 + ( c 1 1 − c 1 ) {+
+} + ϵ ( c 1 − 2 ϵ ) < 0 .
To sum up, since we can prevent this lower bound from growing monotonically with p , we acceleratethe convergence rate of [Equation] | to 0 when there is stronger spurious correlation (larger p ).
Recallthat the ratio [Equation] | reflects the degree of reliance on spurious features.
Therefore, a fasterconvergence of this ratio to 0 (smaller lower bound) means that the model will end up relying lesson the spurious correlations within finite training time.
In other words, the OOD robustness can beenhanced by using domain-wise perturbations.
Remark.
Here, we demonstrate that MAT and LDAT show stronger OOD robustness comparing to ERM.
We compare the result in Theorem 4.1 to that in the Theorem 2 in [2].
The full statement ofthe Theorem 2 in [2] is in Appendix B.According to the Theorem 2 in [2], even if the max-marginclassifier does not rely on the x sp for any level of spurious correlation p ∈ [0 . 5 , 1) , ERM trained bygradient descent stopped in finite time still fails to avoid using spurious features.
Moreover, whenconducting ERM with finite-time-stopped gradient descent, the lower bound of the convergence rateof w sp β | w inv x inv | to 0 is
[Equation]
where M and c follow the definition in Theorem 4.1.
This lower bound grows monotonically with p ,thus ERM will have slower convergence for larger spurious correlations.
However, with domain-wiseperturbations, we can modify the lower bound so that it does not increase monotonically with thespurious correlation p .
Experiments
5.1 Experimental Setup
We conduct experiments on the DomainBed benchmark [9], a testbed for OOD generalization thatimplements consistent experimental protocols across various approaches to ensure fair comparisons.
We evaluate on PACS [12], OfficeHome [13], VLCS [14], NICO [15], and Colored MNIST [1].
There are several changes in our experimentation setting comparing to DomainBed:
Backbone Network.
We use ResNet-18 [18] as our backbone network for datasets excludingColored MNIST instead of ResNet-50 used in [9] for efficiency.2.
Hyperparameter Search Space.
We use a smaller hyperparameter search space than [9]. We conduct a random search of 8 trials for PACS
, OfficeHome and VLCS while 6 trials
for
NICO and Colored MNIST in the hyperparameter search space, instead of 20 trials adoptedin [9] for feasibility.
See Appendix C.2 for more details.
Model Selection Strategy.
Since hyperparameter choice has a significant impact on the OODperformance, it is critical to use appropriate model selection method.
For PACS, OfficeHome, and
VLCS datasets, we use training-domain validation proposed in [9] since it is more in line with the
OOD scenario.
For NICO, we adopt OOD validation following [19].
For Colored MNIST, we usetest-domain validation [9] since it can enlarge the gaps in OOD performance among the algorithmswhile the gap induced by training-domain validation on Colored MNIST is marginal.
Hyperparameters for MAT and LDAT.
To retain low-rank structures in perturbations, we setthe upper bound of the search space of the perturbation number k in MAT to be 20.
Similarly,the upper bound of the rank of the perturbation used in LDAT l is 20.
The complete setup of thehyperparameters for MAT and LDAT is provided in Appendix C.2.
[Table]
Table 2 summarizes the results on the five OOD datasets.
The results of other approaches for PACS,
OfficeHome, NICO, and Colored MNIST datasets are adopted from [19].
The results on VLCS ofother algorithms are missing (denoted as "-") because [19] does not experiment on this dataset.
Comparison with ERM and Sample-wise AT.
From Table 2, we observe that both MAT and
LDAT outperform ERM (on both our runs and the results in [19]) and AT on average.
In particular,
MAT achieves consistently better results than ERM on all five datasets.
Additionally, the averageperformance of AT is worse than ERM, which is consistent with our observations in Section 3.2.
Comparison with Existing State-of-the-Art Approaches.
Although the results from [19] use adifferent training protocol from ours: they use a larger search space and 20 random search for thehyperparameter combinations, the comparison between ERM ([19]) and ERM (our runs) indicatesthat their corresponding performances are close.
A similar comparison has been made in [20].
Wefind that MAT outperforms all previous algorithms and LDAT ranked fourth among all methods,merely after VREx [21] and IRM [1].
Supplementary comparison experiments is in Appendix C.3.
Comparison between MAT and LDAT.
From Table 2 we can see that MAT outperforms LDATon average.
Since LDAT reduces the number of the perturbations used in a domain from k to 1(shown in Figure 2), LDAT can be regarded as a low-rank version of MAT.
This indicates that theoversimplified perturbations may be less effective than the ones maintaining some flexibility.
On theother hand, it is usually hard to verify which factors of an algorithm play a key role in its performancedue to the complexity of the factors it contains.
Since both MAT and LDAT outperform most existing
State-of-the-Art methods and they both exploit low-rank structures, these two methods mutuallycorroborate the effectiveness of low-rank structure for OOD generalization.
5.3 Empirical Understanding
Visualization.
To empirically show that MAT and LDAT can reduce the reliance on spurious features,we visualize the pixel attention heatmap of ERM, AT, MAT, and LDAT on NICO dataset using GradCam [32].
It reflects the contribution of different components of the feature map to the predictionresults.
We pick the model with the best performance for each method.
The results in Figure 4indicate that the model trained by MAT and LDAT focuses more on the object itself, while ERM and AT adopt the background information that spuriously correlates to the class to make predictions. [Figure] [Figure]
Parameter Analysis.
The number of the perturbations used in a domain k in MAT and the rank of theperturbation l in LDAT are two key hyperparameters.
We conduct further experiments to analyze theimpact on the performances of these two parameters.
We adopt a fixed set of parameters except for kand l (see Appendix C.1), and evaluate on PACS dataset.
The results in Figure 5 show that MAT and
LDAT are able to keep their performances over ERM as long as k and l are far less than the numberof the samples N .
Additionally, we can observe from the trend that when k and l are too small ( = 5 ),the performances degenerate. This implies the oversimplified structures of the perturbations canbe less effective for generalization. Additional analysis on the impact of the learning rate for theperturbations is in Appendix C.2.  Conclusion
In this work, we empirically reveal the limitations of AT on OOD tasks.
Due to the lack of constraintson the perturbation, AT fails to generalize well when facing large-scale perturbations which is closeto the real-world OOD scenarios.
Based on the observation that UAT shows a stronger robustness tolarge perturbations, we further proposed MAT and LDAT to exploit the low-rank structure to alleviatethis issue.
We validate the effectiveness of the proposed method on OOD tasks both theoretically andempirically.
Checklist
The checklist follows the references.
Please read the checklist guidelines carefully for information onhow to answer these questions.
For each question, change the default [TODO] to [Yes] , [No] , or[N/A] .
You are strongly encouraged to include a justification to your answer , either by referencingthe appropriate section of your paper or providing a brief inline description.
For example:
• Did you include the license to the code and datasets?
[Yes] See Section .•
Did you include the license to the code and datasets?
[No] The code and the data areproprietary.•
Did you include the license to the code and datasets?
[N/A]
Please do not modify the questions and only use the provided macros for your answers.
Note that the
Checklist section does not count towards the page limit.
In your paper, please delete this instructionsblock and only keep the Checklist section heading above along with the questions/answers below.
For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper’scontributions and scope?
[Yes] See Section 1.
(b) Did you describe the limitations of your work?
[Yes] See Section 5.2. We mentionthat the oversimplified perturbation (LDAT) will be less effective than the ones withmore flexibility (MAT).(c)
Did you discuss any potential negative societal impacts of your work?
[No](d) Have you read the ethics review guidelines and ensured that your paper conforms tothem?
[Yes]2.
If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results?
[Yes] Full assumptions are in Appendix A.(b) Did you include complete proofs of all theoretical results?
[Yes] See Appendix A.3.
If you ran experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)?
[No] It will release
upon acceptance. (b) Did you specify all the training details (e.g., data splits, hyperparameters, how theywere chosen)?
[Yes] See Section C.1 and C.2.
(c) Did you report error bars (e.g., with respect to the random seed after running experiments multiple times)?
[Yes] See Table 2 for an example.
(d) Did you include the total amount of compute and the type of resources used (e.g., typeof GPUs, internal cluster, or cloud provider)?
[No]
If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...
(a) If your work uses existing assets, did you cite the creators?
[Yes] We use DomainBed benchmark and GradCam and cite their creators.
(b) Did you mention the license of the assets?
[No] All assets we use are open source.
(c) Did you include any new assets either in the supplemental material or as a URL?
[No]
(d) Did you discuss whether and how consent was obtained from people whose data you’reusing/curating?
[No] All datasets we use are open source.
(e) Did you discuss whether the data you are using/curating contains personally identifiable
information or offensive content?
[No]
If you used crowdsourcing or conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, ifapplicable?
[No] (b) Did you describe any potential participant risks, with links to Institutional ReviewBoard (IRB) approvals, if applicable?
[No]
(c) Did you include the estimated hourly wage paid to participants and the total amountspent on participant compensation?
[No]