Towards Combating Frequency Simplicity-biased Learning for Domain Generalization
Authors: Xilin He
Jingyu Hu
Qinliang Lin
Cheng Luo
Weicheng Xie
Siyang Song
Muhammad Haris Khan
Linlin Shen
Venue: NeurIPS 2024 poster (2024)
Keywords: Frequency Shortcut;Domain Generalization
Primary area: safety_in_machine_learning
TL;DR: This paper proposes two data augmentation modules, which is also the first work, to tackle the topic of domain generalization from the perspective of preventing frequency shortcuts.

Abstract: Domain generalization methods aim to learn transferable knowledge from source domains that can generalize well to unseen target domains. 
Recent studies show that neural networks frequently suffer from a simplicity-biased learning behavior which leads to over-reliance on specific frequency sets, namely as frequency shortcuts, instead of semantic information, resulting in poor generalization performance. 
Despite previous data augmentation techniques successfully enhancing generalization performances, they intend to apply more frequency shortcuts, thereby causing hallucinations of generalization improvement.
In this paper, we aim to prevent such learning behavior of applying frequency shortcuts from a data-driven perspective. Given the theoretical justification of models' biased learning behavior on different spatial frequency components, which is based on the dataset frequency properties, we argue that the learning behavior on various frequency components could be manipulated by changing the dataset statistical structure in the Fourier domain. 
Intuitively, as frequency shortcuts are hidden in the dominant and highly dependent frequencies of dataset structure, dynamically perturbating the over-reliance frequency components could prevent the application of frequency shortcuts.
To this end, we propose two effective data augmentation modules designed to collaboratively and adaptively adjust the frequency characteristic of the dataset, aiming to dynamically influence the learning behavior of the model and ultimately serving as a strategy to mitigate shortcut learning. Our code will be made publicly available.

Towards Combating Frequency Simplicity-biased Learning for Domain Generalization
Xilin He', Jingyu Hu', Qinliang Lin', Cheng Luo', Weicheng X.23 Siyang Song", Muhammad Haris Khan 5, Linlin Shen 122,3
Computer Vision Institute, School of Computer Science & Software Engineering, Shenzhen University" Shenzhen Institute Artificial Intelligence and Roboties for Society² Guangdong Provincial Key Laboratory of Intelligent Information Processing University of Exeter', Mohamed bin Zayed University of Artificial Intelligence wcxie@szu. edu. en
Abstract
Domain generalization methods aim to learn transferable knowledge from source domains that can generalize well to unseen target domains. Recent studies show that neural networks frequently suffer from simplicity-biasee learning behav- ior which leads to over-reliance on specific frequency sets, namely as frequency shortcuts, instead of semantic information, resulting in poor generalization per- formance. Despite previous data augmentation techniques successfully enhane- ing generalization performances, they intend to apply more frequency shortcuts, thereby causing hallucinations of generalization improvement. In this paper, we aim to prevent such learning behavior of applying frequency shorteuts from data- driven perspective. Given the theoretical justification models' biased learning behavior different spatial frequency components, which is based on the dataset frequency properties, we argue that the learning behavior on various frequency components could be manipulated by changing the dataset statistical structure in the Fourier domain. Intuitively, as frequency shortcuts are hidden in the domi- nant and highly dependent frequencies of dataset structure, dynamically pertur- bating the over-reliance frequency components could prevent the application of frequency shortcuts. To this end, we propose two effective data augmentation modules designed to collaboratively and adaptively adjust the frequency charac- teristic of the dataset, aiming to dynamically influence the learning behavior of the model and ultimately serving strategy to mitigate shortcut learning, Code is available AdvFrequency.
Introduction
Previous studies have shown that the performance of deep neural networks usually degrades when processing previously unseen data whose distribution differs radically from the training data distri- bution, which largely constrains the deployment of well-trained deep models [11, 10]. Domain generalization (DG) aims learn models with transferable knowledge, utilizing multiple source domains, that can generalize well to unseen test data to address this problem [19, 8]. In this paper, we focus on more challenging scenario where only one source domain available for training, namely as single source domain generalization (SDG).
+Corresponding author
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

Image
Network
Specifie Frequency Bands
Simplicity-hisseel
"Plane"
Learning
Learning
Frequency Shortent
Learning Difficulty
Loarnnng Difficolly
Our Method
Frequency Rands
Preventing
Frequency
Frequency Shorteut
Network
Human Vision
Adversarial
Concept
"Plane"
Learning
Figure I: Illustration model's frequency simplicity-biaseee learning and motivation our method. Deep models tend learn the simplest solution (e.g. the specific class-wise most distinctive fre- quency bands) classification tasks instead of the semantic cues. Our method proposes to adap- tively modify the learning difficulty of different frequency components prevent frequency shortcut learning.
The domain generalization problem can be conventionally solved by learning domain invariant fea- tures [23, 7]. conducting adversarial training [20, 15], employing meta-learning [4] or data augmen- tation [37, 16]. Among the proposed techniques, data augmentation is prone to be the simplest and the most effective one. Specifically, [12] proposes to mix various augmented version of images to enhance robustness. [16] proposes to generate diverse stylized samples to improve the generaliza- tion ability neural networks. Recently, an interest understanding the learning dynamic of neural networks to improve generalization has arisen [40, 26, 35, 31]. Neural networks function from low frequencies first in the regression task, which is widely known as F-principle [40] spectral bias [26]. [28] reveal that neural networks tend to learn simple but effective patterns, such as the shortcut solutions that disregard semantics related to the problem at hand but are simpler to minimize the optimization loss, which leads to over-fitting and generalization performance degra- dation. With the generalization performance improvement brought by previous data augmentation techniques [12, 16], it's widely believed that previous successful data augmentation techniques can effectively prevent shortcut learning. However, recent study on frequency shortcuts [35], which is the certain set of frequency bands used specifically to classify certain classes in classification task, empirically disproves this assumption. They have demonstrated that previous data augmentation techniques [12, 16]. despite gaining generalization performance improvement, learn more frequency shortcuts instead and cause the hallucination of generalization improvement.
found to fit
To this end, aim to develop data augmentation techniques capable of preventing the learning of the frequency shortcuts, which are neglected by previous works and thereby achieving enhanced generalization performances. Since [24] has introduced a mathematical formulation demonstrating that the learning dynamics of neural networks are biased towards dominant frequencies of the sta- tistical structure of the dataset, which maybe necessarily low frequencies, the study [24] offers model's learning behavior across different frequency
a theoretical basis for the manipulation components. In this paper, building upon the theoretical underpinning provided by [24], we attempt to modify the frequency spectrum of the dataset statistical structure with aggressive frequency data augmentation, as this would affect the dominant frequencies of the dataset and thereby change the model's 's learning strength on different frequencies. By dynamically modifying the frequency prop- erty of the dataset, we would be able adaptively manipulate model's learning behavior on various frequency components and thereby prevent the application of frequency shortcuts by suppressing the learning on the highly dependent frequencies. To this end, we propose two effective and practical ad- versarial frequency augmentation modules, i.e., Adversarial Amplitude Uncertainty Augmentation (AAUA) and Adversarial Amplitude Dropout (AAD). These modules are designed effectively and aggressively alter the frequency characteristie, aiming to dynamically manipulate the model's

---

learning behavior on over-reliance frequency components and ultimately help prevent the learning of frequency shorteuts. Our core contributions are summarized follows:
Given the theoretical justification of neural network's s learning behavior frequency short- cuts, we propose two effective data augmentation modules, adversarial amplitude uncer- tainty augmentation module (AAUA) and adversarial amplitude dropout module (AAD) to dynamically and implicitly regularize and suppress the model's over-reliance to specific frequency bands, serving as the pioneering work to combat frequency shortcuts for domain generalization.
Our proposed method is evaluated under a variety of DG tasks and datasets, ranging from image classification instance retrieval. The results show the superiority of the proposed augmentation algorithm over state of the arts (SOTAs).
2
Related Work
Domain generalization (DG) aims to learn transferable knowledge that can generalize well to unseen target domains. In general, the DG problem has two settings: multi- source domain generalization and single domain generalization (SDG), distinguished by the number of available source domains during training. The SDG is considered as more realistic scenario with greater difficulty due to the lack of diversity data. Inspired by domain adaptation methods, early studies attempt to carry out domain alignment to learn domain invariant features by reducing the bias of feature distributions among multiple source domains [20, 15, 21] with adversarial training [15, 29] or minimizing distance metries between distributions [20, 21J. In addition, self- self-supprvised learning [1, 17] and meta-learning [4, 30] have also been studied.
Domain Generalization:
Recently, data augmentation [50, 37, 51], especially style augmentation [37, 51, 22, 16], has been widely studied and shows appealing performances. L.2D [37] diversifies image styles by playing a min-max game with mutual information. DSU [22] proposes to perform style augmentation in the feature level and augment feature statistics with uncertainty estimation to simulate the domain shifts. Rising methods resort data augmentation in the frequency domain [41, 14, 38, 34, 43J. Repre- sented by FACT [38], the vast majority the frequency augmentation methods resort to randomly swapping or mixing the amplitude spectrum maps within the same batch. SADA [43] proposes to inject adversarial perturbations into the amplitude spectrum maps, intending to suppress frequency sensitivity gain generalization performance improvement. However, previous successful data aug- mentation techniques cannot successfully prevent the learning and application of frequency shortcuts despite gaining generalization performance improvement. Wang al. [35] demonstrate that data augmentation techniques such as AugMix [13] and style augmentation [37] surprisingly learn more frequency shorteuts and cause hallucinations of generalization ability improvement.
Shorteut Learning:
Shortcuts in classification tasks are known as the spurious correlations between the input data and ground truth labels, rather than the semantic information [5]. Since models that apply shorteuts tend to give predictions based on the presence of the specific shorteuts instead of the transferable semantic content of data, shorteut learning could damage the generalization ability of learned models. Recent studies look into the frequency domain to understand the shortcut learning behavior. [38] prove that neural networks tend to fit function from low-frequencies priorly and then higher frequencies in the regression tasks, which is known as the F-principle. [24] mathematically formulated the finding that the neural network's 's learning behavior is biased towards the dominant frequencies the dataset and could be manipulated. [35] demonstrate that models' learning order of different frequency components in the classification tasks is data-driven and models tend to learn the simplest and the most class-wise distinctive frequency components to give predictions, which is known as the frequency shorteut learning. They further reveal that previous successful data augmen- tation techniques cannot prevent the learning and application of frequency shortcuts.
3 Analysis
In this section, we provide clarification on the definition of the discussed 'frequency' this paper and theoretical analysis of network's 's learning behavior frequency components following [24].
3

---

3.1 Clarification of the Frequency in this Work
Clarifying the definition frequeney. worth noting that wide misconceptions between spatial frequency and responsive frequency exist in the research community. We would have to emphasize that the frequency discussed in this paper not the same as the frequency discussed in the well- known F-principle [40] and spectral bias [26]. The frequency discussed in this paper is the nature of the image data itself, known as spatial frequency, which is the characteristic of the image data pro- cessed by Fourier transformation. In contrast, the term "frequency" in F-principle [40] and spectral bias [26] refers to the concept of responsive frequency or function frequency. This term describes the characteristics of the mapping function learned by networks, which transforms input images into probability vectors. signifies the smoothness of the learned mapping by the trained networks.
Learning behavior on frequencies. Since the two frequency definitions are different, the claim that networks prefer to learn low frequencies first [40] does not necessarily hold true. The claim holds true for responsive frequency. As for spatial frequency, frequency components that the model prioritizes to learn are not necessarily low-frequency components but are determined by the dataset's properties [24]. However, in real-world datasets, as low-frequency components contributors to the Fourier spectrum of the singular vectors to a dataset's statistical structure, neural networks might prioritize learning low-frequency components of images in most cases.
often the main
Tasks of concern. Previous works on the responsive frequency focus on the regression task. In contrast, this paper aligns with recent studies on spatial frequency [35], which mainly focus on the classification task.
3.2 Theoretical Justification
In this section, refer to the theoretical observation from [24] as the justification for the provability of manipulating the model's learning behavior on various frequency components and preventing frequency shorteut learning.
Preliminaries. For the sake of clarity, we first provide descriptions of basic variables.
(i) Two-dimensional images are vectorized to 1 x n..
(ii) To represent the statistical structure of dataset with classes, following Pinson al. [24], p x n² input-output correlation matrix ΣVS is adopted. Given one-hot encoded ground truth labels y. each row of ~R contains the average of images in the corresponding class. The input-input correlation matrix E and prediction-input correlation matrix プ is similarly defined.
(ii)) In SVD basis, ΣEe can be decomposed as Σ = USVT, where S and V contain the singular values and right singular vectors, respectively. The first p rows of VT, denoted as da E O..... p 1, the principal components of the dataset structure. In the SVD basis of y the puudiction-output matrix y can be decomposed in a way similar to SVD with U and VT as y = UAV So. and da denote the a-th diagonal elements in S and A, respectively.
Data-driven frequency learning behavior. For the learning dynamics of networks, the learning behavior is biased toward the frequencies that are dominant in the Fourier spectrum of the singular vectors derived from the statistical structure of a dataset. This is mathematically formulated as [24]:
d d(((r), c((rr, x (s((.) - -ax), di
where Q and denote the vectorized 2D discrete Fourier transformation and the convolution kernl in a network. j and e denote the indices of the frequencies contributing to d" and an irrelevant constant representation for simplicity, respectively. In Eq. 1, Sa can be considered as the input, while (Qr), represents the Fourier coefficients
the kernel r, indicating the kernel's reliance on the j-th frequency. As analyzed by Pinson et al. [24], when the convolution kernels are initialized with small values, aa and (Qr), would be small. In this case, the learning dynamic is mainly driven by the frequency property of the dataset structure (Q )z)) )- Thereby, the model would be highly dependent on the dominant frequencies that contribute most to the singular vectors & of the dataset.
Risk of applying frequency shortcuts. Intuitively, in the natural image datasets, strong correlations between patterns could exist. These kinds of correlations could exhibit themselves in the statistical structure of the datasets, and thereby be picked up by the networks during the learning process [5].

---

Adversorial Amplitude Uncertainty
Adverserial Modul:
Augmestation Modull
(L400)
ULL
AAUA
AAD
Men griie Man Mopple
V
Halk
Network
Representation
440 DXL
Haberd
Clonko
Classiferr
Figure 2: Overview of the proposed adversarial frequency augmentation modules.
Given the formulated biased learning behavior in Eq. 1, the focus of learning is biased toward fre- quency components that dominant in the singular vectors of the Fourier spectrum of the dataset. As it is acknowledged that specific frequency components beneficial for the classification task in specific domains [5, 35] are outstanding in the dataset structure, a model directly trained on these images might take the risks of being over-reliance on these frequency components.
Recently, [35] has highlighted tendency in networks to leverage shortcuts of frequencies that specific and highly discriminative frequency components across classes, rather than semantic cues. This frequency shorteut learning phenomenon is evidenced in Eq. 1, which shows that the learning process prioritizes frequency components specific to the singular vectors of E with higher singular values [24]. Consequently, the model captures the most discriminative frequency components in the spectrum of the dataset structure, potentially leading to the adoption of frequency shortcuts.
Leveraging the learning behavior as shield against frequency shortcuts. As the learning be- havior Eq. mathematically defined, it would difficult to go against with fixed dataset structure. However, if the model is deemed to capture frequency shorteuts hidden in the dominant frequencies on a static dataset, natural questions emerge: Can we adaptively modify the dataset to reduce the possibility frequency shorteuts by leveraging biased frequency learning rule? Can we construct dynamic dataset to modify its frequency property in each learning iteration?
Motivation in preventing frequency shortcuts. To this end, this paper attempts to combat the frequency shorteut learning by dynamically modifying the frequency characteristic of the dataset structure. As shown in Eq. 1, the gradient is dynamically influenced by the highly coupled interplay of dataset frequency characteristics and model weight. Given the dynamic and decoupled learning process in Eq. 1, we attempt make use of adversarial learning to bridge between model weight and the dataset frequency characteristics.
To effectively remould the spectrum map of the dataset structure, we propose to conduct aggres- sive augmentation on the frequency spectrum, instead of simply injecting noises. Further, different from prior frequency augmentation methods [43], this paper utilizes adversarial gradients w.r.t. the amplitude spectrum maps locate the over-reliance frequency components, instead of following simple assumptions to mask specific frequency bands. However, it could be technically difficult to directly manipulate the frequency characteristic of the whole dataset statistical structure due to com- putational resource constraints when backpropagating the entire dataset. Therefore, propose two specific in-batch adversarial frequency augmentation modules to effectively modify the dataset frequency characteristic.
Methodology
The main goal in single source domain generalization is to train a model f on a source domain, which should generalize well in various previously unseen target domains. We denote the source domain as Dg = {(x })}, x E RW @WxMxC a source image, W, H and C are the width, height and the number of channels of source images and y is the corresponding label.
For convenience, we use F(-) to denote Fast Fourier Transformation (FFT). A(:) and P(-) denote the calculation of amplitude and phase, respectively. In the following sections, propose the module
5

---

of Adversarial Amplitude Uncertainty Augmentation (AAUA) and its complementary module of Adversarial Amplitude Dropout (AAD).
4.1 Adversarial Amplitude Uncertainty Augmentation (AAUA)
As low-frequency components are the main contributors in natural image datasets, to effectively modify the frequency spectrum of dataset structure, propose to inject adversarial noises into the low-frequency components. Instead of directly injecting noises [43] into the frequency spectrum map, we propose combine adversarial learning with instance normalization craft aggressive augmentations to effectively disrupt the overall dataset structure and thereby prevent the learning of frequency shorteuts hidden in the low-frequeet frequeeeneey components.
Different from previous frequency augmentation methods [43, 34] that treat each amplitude spec- trum map as deterministic values, we hypothesize that statistics of each low-frequency part of am- plitude spectrum maps are samples drawn from a Gaussian distribution, based on which we adver- sarially draw new statistics to reconstruct the augmented low-frequency spectrum maps properly model the uncertainty of domain shifts rich in diversity and hardness.
Adversarial low-frequency component transformation. Given batch of input images {ar ( where B denotes the batch size, we first obtain the amplitude spectrum maps. We employ Mio and Mugh = 1
Jk=1
Mew as the masks to separate low- freeuueye and eigh-frequency components as:
E [Imin Mmaar
[Wminn Wmman]
Mun (i,
(2)
otherwise
where Nanin = [(1 /2), Rmmax =((1 ((1++)) / Wmin 1(1-((1)))) X)W/21 L(1-X)
and Wmmx
L(1 + 2)W denote the lower and upper bounds to obtain height and width of low-frequency mask. denotes the round-down operator. À controls the size of the mask. As it would be difficult to precisely determine which band is low-frequency or not, À is drawn from a uniform distribution U (O, 0.5). Then, we calculate the channel-wise statistics of the low-frequency part following:
Jann
m(A(z)) WHE
Σ
Σ
A(x)(..
i=hmin Womin
(3)
hman
(A(2)) WHH
Σ
[A()) m(A(=)))
i=hmen J=
where y and denote mean and variance, respectively. We further use the standard deviations of these channel-wise statistics for the uncertainty estimation E, RC and E. E RR:
Σ (A(a)) = 1/1
B ((A(x())) Bo(m(A()))))
Σ
(4)
E3(A()))
Σ (((A()))) Es(o()(z))))
B
where (*k) and E-) denote the k-th image in a batch and the expectation operator within this batch. Then, sugmentation-rrlatee statistics updated following:
( = m(A(x)) di Σ,
(5)
o'(A(x)) = o(Ax)) O.
where d'' and df control the intensity and direction of the adversarial perturbations to the channel- wise mean and variance at the f-th iteration of AAUA, which are to be optimized. O denotes the Hadamard product. Then the augmented amplitude is derived as:
Arcul (2) ((A(z))
AAAU
)o (A((A))) p¹ (A(x))
o(A(x))
(6)
A AMUA = Min c A + Magh @ A(x)
where Alow(a) = A(x) @ Mow denotes the low frequency parts of A(z). We can formulate the t 2 1-th optimization step of adversarial channel- wise statistics control factors (dlt dj41) as:
d di sign(V ( (o y5
(7)
do = de signl V AALA y3 f))

---

where x AAAUA = =F (AAAUA, -A P(z)) is generated with augmented amplitude A and original phase P(x) by Inverse Fast Fourier Transformation (IFFT) F and sign(), V and Lusk denote the sign function, gradient operator and the task-related loss for adversarial gradient calculation, e.g. cross-entropy loss for image classification. We use the above optimization for steps to generate the AAUA-augmenter image AAAUAA
4.2 Adversarial Amplitude Dropout (AAD)
Despite AAUA shifting the network's attention away from the low-frequency components, it may lead to models' over-fitting into the imperceptible noises hidden in high-frequeney bands, leading a decline in models' generalization abilities. Therefore, prevent the learning of frequency shortcuts in the
components, we further propose an Adversarial Amplitude Dropout (AAD) module to adaptively modify the frequency spectrum maps with backward gradients to es- timate the model's frequency characteristics, which allow to dropout the highly dependent fre- quency components to remove the learned frequency shortcuts.
Different from prior frequency masking methods [14, 41] that simply mask specific bands, we at- tempt to adaptively mask the over-reliance frequency bands. To locate the critical frequency bands, inspired by [43, 2], we utilize the gradients w.r.t. amplitude as the surrogate of model's 's frequency sensitivity map across frequency bands:
Ja V Cusk (5.
Since the larger magnitude of these gradients represents a stronger sensitivity to the corresponding frequency band, larger probability of this frequency band is preferred to be dropout. Further, to craft more augmented samples with various learning difficulties, random threshold for dropping frequency bands is introduced. Thereby, we generate a dropout mask with the amplitude gradients Ia and random threshold p drawn from uniform distribution (O, 1) following:
1 if (ig((i))) p(i.))
MAAD (i. =
(9)
otherwise
where MAAD( j) denotes the mask value at the coordinate of [ and min max mapping s(x) (xc max(x)) (max(r) min(x)). Then the dropout amplitude is derived by Hadamard product:
A (x) = A(x) @ AA
(10)
where (x) denotes the dropout amplitude of image x. We further obtain the augmented sample AAD with dropout amplitude and the original phase through IFFT following:
ADD F (A (x), P(x))
(11)
where F-1 denotes the IFFT. For clarity of narrative, the pseudo-code of generating augmentation samples and samples generated by AAUA and AAD are shown in the supplementary material.
4.3 Model Training
To further regularize the prediction consistency between benign and augmented images, Jensen- Shannon divergence consistency loss LJs [13] also adopted. Therefore, we can formulate the total training loss as:
(12) where Lusk denotes the task-related loss, e.g. cross-entropy loss for image classification and in- stance retrieval. Both of the proposed modules generate augmented images for training in each iteration.
c = Lank + Lis
5 Experiments
We first conduct cross-domain image classification and instance retrieval experiments to study the proposed method's performance. We compare our method with traditional data augmentation based methods [13, 33, 39], image-level style augmentation based methods [25, 37], feature- style augmentation methods [45, 22, 44], two most relative frequency augmentation-bast
methods [38, 43] and other representative methods [23, 1]. Then we evaluate the frequency shorteuts following Wang's 's metric [35]. We further provide insights into models' data-driven frequency characteristics with the proposed AAUA and AAD. We also carry out detailed ablation studies.
7

---

Table Experimental results on Digits,
Table 2: Experimental results of the task of person re- ID on Mar and DukeMTMC. We follow the
PACS and CIFAR-100 -O-C..
Method
Digits PACS CIFAR-IO-C
benchmark by [44].
ERM
55.67
49.29
57.56
46.09
59.91
54.08
Method
mAP
Market- Duke
R1
R10
mAP
Duke-Marrke
R1
Maaket R5
R10
M-ADA[25 Jover 20
59.49
59.40
64.65
OSNet
27.9 48.2 62.3 68.0
25.0 52.8 70.5 77.5
AugMix[12 11
72.45
62.47
75.28
RandConv(39
72.88
62.76
71.23
RandomEEuse
20.5
36.2 52.3 59.3
22.4
49.1
66.1
73,0
1.2D(37]cv9w2
64.74 74.45
72.88
+ DropBlock
23.1 41.5 56.5
62.5
21.7
48.2
65,4
71.3
FACT[38]c
59.10
+ MixXitle
28.0 49.5 63.6
68.8
27.5
57.4
74.1
80.
DSU
53.70
+EFDMix
29,9 50.8 65.0 70.3
29.3
76.5
SADA[43]AMM)
76.56
65.71
77.33
+StyleNeophile
29.7 50.6 65.4 74.2
32.2
80.2 89.1
FFM[,6]
73.45
70.4
77.77
+DSU
31.0 53.0 66.6 71.7
29.3
59.9
77.2 82.8
ALT[6]
72.49
64.72
+AdvStyle
33.2 55.7 69.1 73.5
32.0
63.2
79.7 85,0
ABA[3 lievea
74.76
66.02
+SADA
33.1 53.2
67.4
72.5
30.0
62.7
78.4 84.2
80.10 64.10
+Ours
33.8 56.3
70.1
75.8
34.2
64.9
81.8 90.1
Ours
81.39 67.91
78.33
5.1
Datasets and Implementation Details
Datasets for image classification and instance retrieval: We evaluate the performance of the proposed method on three cross-domain image classification benchmarks of PACS [19], Digits and CIFAR-10-C [10]. We conduct the cross-domain instance retrieval task on person re-identification (re-ID) datasets of Market1501 [47] and DukeMTMC [27] with OSNet [48].
Implementation Details In all the experiments, we set the iterative optimization steps T of AAUA as 5. For fair comparison, the number of augmented samples used in each iteration is set as 3, following [43]. We adopt the same network architectures in previous works [43, 44). Please see the supplementary material for more implementation details.
5.2 Comparisons with State-of-The-AW
We conduct experiments of cross-domain image classification and person re-ID in five benchmarks. models trained with our generated data obtain significant generalization performance improvement (Tab. 1). Specifically, our method outperforms SADA [43] and FACT [38] with large margins of 2.20% and 8.81% in PACS. Following [45, 44], we conduct cross-domaae instance retrieval of person re-II) task on Market 151 [47] and DukeMTMC [27]. Experiment results are shown Tab. 2. Our method outperforms previous SOTA StyleNeophile [16] with large margin, boosting the mAP on Mukket->Duke from 29.7 33.8, further validating the effectiveness of the proposed method.
5.3 Frequency Shortcut Evalutation
Wang et al. [35] propose to use TPR and FPR on the dominant frequency maps (DFM) filtered test sets of ImageNet- 10 evaluate how many frequency shortcuts the classifier is induced learn and apply. Semantic information of DFM-filteed samples is severely damaged. In this paper, we use the average of class-wise TPR and FPR on the DFM-filteedd test sets of ImageNet-10 10 to quantify how many frequency shortcuts are applied. Higher TPR and FPR values indicate that more frequency shortcuts are applied. Samples from the DFM-filtered test set are shown in the supplementary mate- rial.
The frequency shorteuts quantitative results of the standard network of ResNet-18, AugMix, style augmentation and ours are shown in Tab. 3. As shown in Tab. 3, the combination of AAUA and AAD effectively suppresses the learning of frequency shortcuts, while previously successful data augmentation techniques apply more frequency shortcuts, causing the hallucinations of generaliza- tion ability improvement. It's worth noting that when AAUA or AAD solely applied, the model achieves more frequency shortcuts. As AAUA solely perturbates the low-frequency components, leaving high-frequency components unchanged. Hence, using AAUA alone, without AAD, may lead to exploring frequency shortcuts in high-frequency bands.
8

---

Sketch
Baseline
Ours
ARTRRRER
Peaera Shiigg
(a) Frequency sensitivity maps
(h) Feature manifolds of augmented samples
Figure 3: (a) Visualizations of model frequeney sensitivity maps on source (Photo) and target do- mains (Art painting, Cartoon, Sketch). (b) Features manifolds of augmented samples and various domains.
5.4
Empirical Analysis
5.4.1 Frequency sensitivity of trained models
To validate that modifying the frequency property of the dataset structure could indeed affect the model's preferences on various frequency components, we further show the frequency sensitivity of baseline models and models trained with our method. Following Zhang et al. [2]. we obtain frequency sensitivity maps with backward gradients w.r.t amplitude spectrum maps surrogates of model's 's frequency sensitivity. It's worth noting that the frequency sensitivity maps are derived after a softmax function. Therefore, would only show relative sensitivity across frequency bands, but not indicate the application of frequency shortcuts.
5.4.2 Augmentation strength of different augmentation methods
To validate the effectiveness of modifying the dataset frequency property with AAUA, we further compare the hardness between previous in-batch mixing frequency augmentation methods (e.g. FACT [38]) and our proposed AAUA. As shown in Fig. 3(b), we use one sample in the source domain to generate sufficient augmented samples. Compared with in-batch mixing methods, feature embeddings of samples generated with the proposed AAUA span across different domains and are further from the source domain, indicating that the proposed AAUA generates augmented samples with greater hardness and diversity.
Table 3: Frequency shorteuts quantification on the DFM-filtered test set of ImageNet- 10.
Table 4: Ablation studies conducted on
PACS dataset. PD denotes performance
Method
Avg TPR
FPR
degradation compared with the full method.
ReaNet-18
0.286
AND AMUA Lss PT
SC
Avg/PD)
ResNet- AugAux
0.242
0.046
47.38
72.60
78.28
56,97
63.31 (-1.0)
RosSet-18-Style Augmentation
0.250
6-.53 18)
ResNet-I8-PACT
0.274
0.048
53.73
74.59
78.30
$8.78
66.35 (-1.6)
RcsNec-I--AAUA
0.265
I&
0.280
0.083
(-4.0)
ResNet-18+Ours
0.212
57.83
74.85
78.44
54.75
66.47 (-1.44)
58.41
5.5
Ablation Study
We conduct ablation studies in Tab. 4 on the proposed adversarial frequency augmentation modules study the performance of each component the proposed method. As shown in Tab. 4, the gen- eralization performance on PACS dataset degrades by 1.44% or 4.00% without the proposed AAD or AAUA module, validating the effectiveness of the proposed augmentation modules. Analysis of the iterative optimization steps T in AAUA and the proportion of augmentation modules' impact on generalization performance shown the supplementary material.
Conclusion
In this paper, we attempt to counteract models' simplicity-bian learning behavior of applying fre- quency shorteuts from the perspective of data-driven and learning difficulty, with previous successful

---

data augmentation techniques failing to do so. Since neural networks take up the simplest and the most wlsse distinctive fireuueccy patterns as frequency shorteuts, we propose to adaptively mod- ify the learning difficulty of different frequency components by combining adversarial training with instance normalization (AAUA) and dropout operation (AAD) in the frequency domain. By effec- tively preventing the iimllcitt-bbasee learning of frequency shortcuts and manipulating the trained models' frequency characteristics, the experimental results on five public benchmarking datasets ranging from classification to instance retrieval demonstrate the superiority of our method.
Limitations and future work. Despite the proposed method indirectly preventing frequency short- cuts from a data-driven perspective, the direct location of frequency shortcuts remains a challenging problem with no effective solutions in the field. In the future, we would like to investigate more es- sential properties of frequency shortcuts to help locate them effectively and thereby introduce more effective techniques combating frequency shortcuts. Meanwhile, the robustness of the metric for frequency shortcut evaluation is yet to be explored in the future.
7
Acknowledgement
The work was supported by the National Natural Science Foundation of China under grants no. 62276170, 82261 138629, 62306061, the Science and Technology Project of Guangdong Province under grants no. 2023A151500 1549, 2023A1515010688, the Science and Technology Innovation Commission of Shenzhen under grant no. (CY1202205311011120000 Guangdong Provincial Key Laboratory under grant no. 2023B 1212060076.
10

---

References
[1] Fabio M Carlucci, Antonio D' D'Innocente, Silvia Bucci. Barbara Caputo, and Tatiana Tommasi. Domain generalization by solving jigsaw puzzles. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 2229-2238, 2019.
[2] Alvin Chan, Y. Ong, and Clement Tan. How does frequency bias affect the robustness of neural image classifiers against common corruption and adversarial perturbations? In International Joint Conference Artificial Intelligence, 2022.
[3] Sheng Cheng. Tejas Gokhale, and Yezhou Yang. Adversarial bayesian augmentation for single-source domain generalization. In Proceedings of the IEEF/CVF International Conference on Computer Vision, pages 1 11400-1 1440, 2023.
[4] Yingjun Du, Jun Xu, Huan Xiong, Qiang Qiu, Xiantong Zhen, Cees GM Snoek, and Ling Shao. Learning to learn with variational information bottleneck for domain generalization. In Computer Vision-ECCV 2020: 16th European Conference, Glasgow, UK, August 23-28, 2020, Proceedings, Part X 16, pages
200-216. Springer, 2020.
[5] Robert Geirhos, Jörn-Henrik Jacobsen, Claudio Michaelis, Richard Zemel, Wieland Brendel, Matthias Bethge, and Felix A Wichmann. Shortcut learning in deep neural networks. Nature Machine Intelligence,
2(11):665 (60)666663666666 2020.
[6] Tejas Gokhale, Rushil Anirudh, Jayaraman J Thiagarajan, Bhavya Kailkhura, Chitta Baral, and Yezhou Yang. Improving diversity with adversarially learned transformations for domain generalization. In Pro- ceedings of the IEEE/CVE Winter Conference on Applications of Computer Vision, pages 434-443, 2023. Domain flow for adaptation and generalization. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 2477- 2486, 2019.
[7] Rui Gong, Wen Li, Yuhua Chen, and Luc Van Gool.
[8] Ishaan Gulrajani and David Lopez-Paz. In search of lost domain generalization. arXiv preprint arXiv:2007.012
2020.
191 Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference computer vision and pattern recognition, pages 770-778, 2016. [10] Dan Hendrycks and Thomas G. Dietterich. Benchmarking neural network robustness to common corrup- tions and perturbations. In 7th International Conference on Learning Representations, ICLR 2019, New Orleans, LA, USA, May 6-9, 2019. OpenReview.net 2019.
[11] Dan Hendrycks and Kevin Gimpel. A baseline for detecting misclassified and out-of-distrihution exam- ples in neural networks. arXiv preprint arXiv:1610.021 036, 2016.
[12] Dan Hendrycks. Norman Mu, Ekin D Cubuk, Barret Zoph, Justin Gilmer, and Balaji Lakshminarayanan. Augmix: A simple data processing method to improve robustness and uncertainty. arXiv preprint 192.00787, 2019.
[13] Dan Hendrycks, Norman Mu, Ekin Dogus Cubuk, Barret Zoph, Justin Gilmer, and Balaji Lakshmi- narayanan. Augmix: A simple data processing method to improve robustness and uncertainty. In 8th International Conference Learning Representations, ICLR 2020. Addis Ababa, Ethiopia, April 26-30, 2020. OpenReview.net. 2020.
[14] Jiaxing Huang, Dayan Guan, Aoran Xiao, and Shijian Lu. Fsdr: Frequency space domain randomization for domain generalization. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 6891-6902, 2021.
[15] Yunpei Jia, Jie Zhang. Shiguang Shan, and Xilin Chen. Single-side domain generalization for face anti- spoofing. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8484-8493, 2020.
[16] Juwon Kang, Sohyun Lee, Namyup Kim, and Suha Kwak. Style neophile: Constantly seeking novel styles for domain generalization. In Proceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recognition. pages 7130-7140, 2022.
[17] Dachee Kim, Youngjun Yoo, Seunghyun Park, Jinkyu Kim, and Jackoo Lee, Selfreg: Self-supervised contrastive regularization for domain generalization. In Proceedings of the IEEE/CVF International Con-
ference on Computer Vision. pages 9619-9628, 2021.
[18] Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. 2009.
[19] Da Li, Yongxin Yang, Yi-Zhe Song, and Timothy M Hospedales. Deeper, broader and artier domain generalization. In Proceedings of the IEEE international conference on computer vision, pages 5542-
5550, 2017.
[20] Haoliang Li. Sinno Jialin Pan, Shiqi Wang. and Alex C Kot. Domain generalization with adversarial feature learning. In Proceedings of the IEEE conference computer vision and pattern recognition, pages 5400-5409, 2018.
[21] Haoliang Li. YuFei Wang. Renjie Wan, Shiqi Wang. Tie-Qiang Li, and Alex Kot. Domain generalization for medical imaging classification with linear-deenddncy regularization. Advances in Neural Information Processing Systems. 33:3118-3122 2020.
[22] Xiaotong Li, Yongxing Dai, Yixiao Ge, Jun Liu, Ying Shan, and LINGYU DUAN, Uncertainty modeling for out-o--distribution generalization. In International Conference on Learning Representations, 2022.
[23] Saeid Motiaan, Marco Piccirilli, Donald A Adjeroh, and Gianfranco Doretto. Unified deep supervised domain adaptation and generalization. In Proceedings of the IEEE international conference on computer 2017.
vision, pages
11

---

[24]
Hannah Pinson, Joeri Lenaerts, and Vincent Ginis. Linear cans discover the statistical structure of the dataset using only the most dominant frequencies. arXiv preprint arXiv:2333. 020344 2023.
[25] Fengchun Qiao, Long Zhao, and Xi Peng. Learning to learn single domain generalization. In Proceedings paaeess...pnsss.........
of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 12556-12565, 2020. Nasim Rahaman, Aristide Baratin, Devansh Arpit, Felix Draxler, Min Lin, Fred Hamprecht, Yoshua Bengio, and Aaron Courville. On the spectral bias of neural networks. In International Conference on Machine Learning. pages 5301-5310. PMLR, 2019.
[26]
[27] Ergys Ristani, Francesco Solera, Roger Zou, Rita Cucchiara, and Carlo Tomasi. Performance measures and a data set for multi-tarree, multi-camera tracking. In Computer Vision-ECCV 2016 Workshops: Am- sterdam, The Netherlands, October 8-10 and 15-16, 2016, Proceedings, Part II, pages 17-35. Springer, 2016.
[28] Harshay Shah, Kaustav Tamuly, Aditi Raghunathan, Prateek Jain, and Praneeth Netrapalli. The pitfalls of simplicity hias in neural networks. Advances Neural Information Processing Systems, 33:9573-9585, 2020.
[29] Rui Shao, Xiangyuan Lan, Jiawei Li, and Pong C Yuen. Multi-adversarial discriminative deep domain gencralization for face presentation attack detection, In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10023-10031, 2019.
[30] Yang Shu, Zhangjie Cao, Chenyu Wang, Jianmin Wang, and Mingsheng Long. Open domain generaliza- tion with domain-augmentted meta-learning. In Proceedings of the IEEE/CVF Conference Computer Vision and Pottern Recognition. pages
2021.
[31] Ajay Subramanian, Elena Sizikova, Najib Majaj, and Denis Pelli. Sratiil- channels, shape bias, and adversarial robustness. Advances in Neural Information Processing Systems, 36, 2024.
[32] Laurens Van der Maaten and Geoffrey Hinton. Visualizing data using i-sne, Journal of muchine learning research, 9(11). 2008.
[33] Riceardo Volpi, Hongscok Namkoong, Ozan Sener, John C Duchi, Vittorio Murino, and Silvio Savarese. Generalizing to unseen domains via adversarial data augmentation. Advances in neural information pro-
cessing systems, 31, 2018.
[34] Jingye Wang. Ruoyi Du, Dongliang Chang. Kongming Liang, and Zhanyu Ma. Domain generalization via frequency-domain- based feature disentanglement and interaction. In Proceedings of the 30th ACM
International Conference on Multimedia, pages 4821-4823 2022.
[35] Shunxin Wang, Raymond Veldhuis, Christoph Brune, and Nicola Strisciuglio. What do neural networks learn in image classification? frequency shortcut perspective. arXiv preprint arXiv:2307 007009899, 2023.
[36] Zijian Wang. Yadan Luo, Zi Huang, and Mahsa Baktashmotlagh. Fim: Injecting out-of oft-oi-domain knowl- edge via factorized frequency modification. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 4135-4144, 2023.
[37] Zijian Wang. Yadan Luo, Ruihong Qiu, Zi Huang, and Mahsa Baktashmotlagh. Learning to diversify for single domain generalization. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 834 843, 2021.
[38] Qinwei Xu, Ruipeng Zhang, Ya Zhang, Yanfeng Wang, and Qi Tian. A fourier-hased framework for domain generalization. In Proceedings of the IEEF/CVF Conference Computer Vision and Pattern Recognition, pages 14383-1433 14392. 2021.
[39] Zhenlin Xu, Deyi Liu, Junlin Yang, Colin Raffel, and Mare Niethammer. Robust and generalizable visual representation learning via random convolutions. In International Conference on Learning Representa- rions, 2021.
[40] Zhi-Qin John Xu, Yaoyu Zhang, Tao Luo, Yanyang Xiao, and Zheng Ma. Frequency principle: Fourier analysis sheds light on deep neural networks. arXiv preprint arXiv:1901.0633 2019.
[41] Yanchao Yang and Stefano Soatto. Fda: Fourier domain adaptation for semantic segmentation. Pro-
ceedings of the IEFF/CVF Conference on Computer Vision and Pattern Recognition, pages 4085-4095, 2020.
[42] Sergey Zagoruyko and Nikos Komodakis. Wide residual networks. In British Machine Vision Conference
2016. British Machine Vision Association, 2016.
[43] Jiajin Zhang, Hanqing Chao, Amit Dhurandhar, Pin-Yu Chen, Ali Tajer, Yangyang Xu, and Pingkun Yan. When neural networks fail generalize? model sensitivity perspective. arXiv preprint
arXiv:2212.
2022.
[44] Yabin Zhang. Bin Deng, Ruihuang Li, Kui Jia, and Lei Zhang. Adversarial style augmentation for domain generalization. arXiv preprint arXiv:2301. /22.233...2643, 2023.
[45] Yabin Zhang. Minghan Li, Ruihuang Li, Kui Jia, and Lei Zhang. Exact feature distribution matching for arbitrary style transfer and domain gencralization. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8035-800 8045, 2022.
[46] Guangtao Zheng, Mengdi Huai, and Aidong Zhang. Advst: Revisiting data augmentations for single domain generalization. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 21832-21844 2024.
[47] Liang Zheng, Liyue Shen, Lu Tian, Shengjin Wang, Jingdong Wang, and Qi Tian. Scalable person re- identification: benchmark. In Proceedings of the IEEE international conference on computer vision, pages 1166-122, 2015.
[48] Kaiyang Zhou, Yongxin Yang, Andrea Cavallaro, and Tao Xiang. Omni-scale feature learning for person re-identification. In Proceedings of the IEEE/CVF international conference on computer vision, pages
12

---

3702-3712, 2019.
[49] Kaiyang Zhou, Yongxin Yang, Timothy Hospedales, and Tao Xiang. Deep comain-adversarit image generation for domain generalisation. In Proceedings of the AAAI Conference Artificial Intelligence,
volume 34, pages 13025-13032, 2020.
[50] Kaiyang Zhou. Yongxin Yang. Timothy Hospedales, and Tao Xiang. Learning generate novel domains domain generalization. Computer Vision-ECCC 2020: 16th Eurepean Conference, Glasgow, UK. August 23-28, 2020, Proceedings, Part XVI 16, pages 561-578. Springer. 2020.
[51] Kaiyang Zhou, Yongxin Yang. Yu Qiao, and Tao Xiang. Domain generalization with mixstyle. In Inter- national Conference on Learning Representations 2021.
13

---

A
Appendix
A.1
More Experiment Results
In this section, we provide more experiment results in cross cross-domain classification tasks and gradients amplitude analyze model's learning behavior different frequency components.
A.1.1 Detailed Classification Performances
We provide detailed single domain generalization performances in Digits, PACS and CIFAR-10-C Tabs. 5, and 7, respectively.
Table Experimental results on Digits dataset. Models are trained on the MNIST and results are reported the listed domains. The best results are labeled in bold.
Method
USPS
MNIST-M
SVHN
SYNTH
Avg.
ERM
76.90
52.74
27.85
39.65
49.29
CCSA
83.72
49.29
25.89
37.31
49.05
JiGen
77.16
57.80
33.81
43.79
53.14
AugMix
80.24
75.86
63.85
69.84
72.45
GUD
77.26
60.41
35.51
45.32
55.67
M-ADA
78.53
67.94
42.55
48.95
59.49
RandConv
84.37
87.77
57.56
62.85
72.88
L2D
83.95
87.32
62.85
63.72
74.45
SADA
89.29
75.61
68.45
72.90
76.56
AAgA+
89.34
75.74
68.34
72.10
76.38
Ours
89.32
82.74
73.58
79.92
81.39
Table Experimental results on PACS dataset, where the listed domain is adopted for training and the reported results evaluated the remaining three domains. PT, AP, CT, SC denote the four domains photo, art painting, cartoon and sketch, respectively.
Method
PT
AP
CT
SC
Avg.
ERM
33.52
57.86
67.84
25.12
46.09
CCSA
42.77
61.89
67.46
26.43
51.08
JiGen
43.49
63.66
70.08
32.47
52.43
AugMix
48.27
72.92
73.81
54.88
62.47
GUD
45.62
69.47
73.46
41.67
57.56
M-ADA
48.22
70.46
75.67
43.26
59.40
RandConv
50.86
75.82
75.46
48.90
62.76
L2D
51.17
76.90
77.80
53.68
64.74
FACT
42.7
69.7
75.2
48.9
59.1
DSU
39.10
63.80
73.60
38.20
53.70
EFDMix
48.00
75.30
77.40
44.20
61.20
SADA
51.18
77.68
76.35
57.61
65.71
SADA SADA+AugMix
51.22
77.82
76.94
57.76
66.18
AdvStyle
45.50
67.80
74.50
47.20
58.70
Ours
58.41
75.71
78.59
58.92
67.91
A.1.2 Gradients w.r.. Amplitude
We show gradients of models in the first ten epochs of training in Tab. As shown Tab. models trained with our generated augmented samples concentrate learning medium and high- frequency components, while baseline training protocol induces models to learn on low-frequency components.
14

---

Table Experimental results on CIFAR-IO-C dataset, where the models are trained on CIFAR-10 and evaluated on images with the listed corruption categories.
Method
Weather
Blur
Noise
Digits
Avg.
ERM
67.21
56.73
30.26
62.30
54.08
CCSA
67.66
57.81
28.37
61.96
54.04
JiGen
67.20
58.06
30.37
62.05
54.43
AugMix
78.53
82.04
64.45
76.17
75.28
GUD
69.94
60.57
48.66
60.37
59.91
M-ADA
75.54
63.67
54.21
65.10
64.65
RandConv
76.87
55.36
75.19
77.51
71.23
L2D
75.98
70.21
73.29
72.02
72.88
SADA
79.44
80.68
70.77
78.42
77.33
AADA+
79.14
82.38
71.42
78.38
77.83
Ours
80.42
81.73
73.68
77.50
78.33
Table 8: Mean of the absolute value backward gradients on different frequency components of amplitude spectrum maps in the first 10 epochs. LF, MF and HF denote low, medium and high- frequency components. A larger value represents that more gradients are used to learn on the specific frequency components. A pretrained ResNet- 18 is adopted.
Domain
Data
LF
MF
HF
MF
HF LF
LF
Benign
7.3 06 1.47e-06 7.01e-07
0.01
Act painting
Ours
1.76e-06 4.79e-06 2.34e-06
2.72 1.33
Benign
5.99e-06
8.65e-07 4.02e-07 0.14 0.07
[c]Cartoon
Ours 1.13e-06 2.40e-06 1.20e-06 2.12 1.06
A.2
More Visualizations
In this section, we provide more visualizations on feature manifold, t-SNE visualizations, generated augmented samples and dominant frequency maps (DFM) filtered samples.
A.3 Feature Manifold Visualization
To intuitively show the augmentation space of different methods, we provide feature manifold vi- sualization of images from the source domain with different augmentation variants. As shown in Fig. 4, both the proposed AAD and AAUA explore border augmentation space compared with the conventional in-batch mixing method [41, 38].
A.3.1 t-SNE Visualizations
To study the distributions of features for the source and target domains, we provide more t-SNE visualizations model trained with proposed methods in Fig. 5. One can see that large overlap- ping areas exist between the source and the unseen target domains, indicating the superiority of the proposed method in exploring wider augmentation space.
A.3.2 Generated Augmented Samples
To give an intuitive perception how our approach affects the images, we provide visualizations of benign images drawn from the PACS dataset and their corresponding augmented samples generated by AAUA and AAD Fig. 6.
A.3.3 DFM-filtered Samples
In this section, we provide benign samples drawn from ImageNet- and their corresponding ones with DFM-filtere out in Fig. 7.
15

---

Source Domain
AAD
In-batch Mixing
AAUA
Figure 4: Feature manifold of images from source domain, AAD, in-batch mixing [41, 38] and AAUA. t-SNE [32] is used in this visualization.
Photo > Art Painting
Photo > Cartoon
Source Domain Target Domain
0
Figure 5: t-SNE visualizations of features from the source domain (Photo) and the unseen target domains (Art Painting and Cartoon) extracted by model trained with our method.
A.4 Implementation Details
In the task of cross-domain image classification, for Digits, we train convolution network with the same architecture [49] with a SGD optimizer for 60 epochs. The initial learning rate 0.001, which decays by the ratio of 0.1 for every 20 epochs. All the grey images converted into RGB images. For PACS [19], we train an ImageNet pretrained ResNet-18[9] on the source domain with an Adam optimizer for 60 epochs. The initial learning rate is 0,0001, which follows cosine annealing decay strategy. For CIFAR-1O-C we train a Wide Residual Network[42] with width factor and 16 layers following previous 10r The network is optimized with SGD optimizer for 120 epochs, with an initial learning rate of 0.1 which linearly decays by 0.1 every 40 epochs. Models trained CIFAR-101 10[18] dataset and evaluated CIFAR-10-C dataset with corruption severity level of 5. All the experiments are run on a Nvidia A100 80GB.
In the task of cross-domain instance retrieval, we adopt the OSNe[[48] pretrained ImageNet, following experiment settings in [45, 44]. The network is optimized with an Adam optimizer for 60 epochs, with an initial learning rate of 0.0003 which linearly decays by 0.1 every 20 epochs.
16

---

Benign
AAUA
AAD
Benign
AAUA
AAD
Figure 6: Images from PACS and their corresponding augmented samples generated by AAUA and AAD.
Benign
DEM-liltred
Benign
DEM-Gilteud
Benign
DEM-Gilterd
Figure 7: Visualizations of DFM-filtered samples.
A.5 Frequency Sensitivity Maps
In this section, discuss how to obtain model's 's frequency sensitivity maps. We utilize gradients as a surrogate of model's 's sensitivity to different frequency bands, following:
N
G=
Σ V A(att) Crask (2) 9
(13)
1=1
where A(-), N, (1) and (t) denote the obtained amplitude spectrum map, number of images in the dataset, the i-th image from the datset and its label, respectively. G E R nxxx3,, We then calculate the mean of G across the channels, following
G G(,so)
G(,A,c)
(14)
We further scale the absolute value of & element-wise to the range between 0 and 1 to obtain the frequency sensitivity maps:
=rs=s((i
(15)
where s the min max mapping, which is defined in Eq. (10) of the main body, and M Fs denotes the frequency sensitivity map.
17

---

Generalization Performancet %))
68.0
Goneraltzation Performancee
67.0
Platt Paiating Average
65.0
8.86
0.48
8.10
0.32
0.14
0.16
0.18
(a) Hyperparameter study on X.
(b) Hyperparameter study on T.
Figure 8: (a) Generalization performances against different hyperparameter settings of X in AAUA. (b) Hyperparameter analysis of the number of iterative optimization steps, i.c. T in AAUA.
A.6 Frequency Contribution
In this section, we discuss how to obtain the quantity frequency contribution (FC). Specifically, FC is formulated the normalization of G' in Eq. (14) as:
|C|
G ΣIG
(16)
where G, denotes the frequency contribution of the whole amplitude spectrum map. To further obtain the low, medium and high frequency contributions, masks for extracting corresponding fre- quency components are formulated as follows
1 if (i+ S Drow
Moow (i,5)
(17)
otherwise
Diow < (12 + j s Dinid
Mmsa(i.
(18)
otherwise
Mnigh (1,j) = Miiia(i,j) Miow (i,5)
(19)
where Miow Mmid and Mhigh denote the masks specific to the low, medium and high frequency components, respectively. Diow and Dmai denote the distances for distinguishing different fre- quency components. Since the area of low frequency components is relatively small and to obtain equal size of medium and high frequency areas, Diow and Dmid are set as 10 and 90 for PACS. Then the frequency contribution is obtained by:
Mlow Gn Gn
FClow
Minid Gn Gn
(20)
FCmid
FChigh =1 FCmid - FClow
where FClow, FCmid and FChigh denote frequency contributions of low, medium and high fre- quency components, respectively.
A.7
Hyperparameter Analysis
A.7.1 Low Frequency Mask Control Factor À in AAUA
In this section, we conduct hyperparameter analysis on X, which controls the size of the low fre- quency mask AAUA. Experiment results are shown in Fig. 8(a). As increases, generalization performances first increase. When the perturbated area continues to expand, the injected adversar- ial noises could harm the stability of the generalized medium frequency components, leading to performance degradation.
18

---

A.7.2 Iterative Times T
In this section, we analyse the influence of the number of iterative optimization steps, i.e. T'n AAUA on model's generalization performances. As shown in Fig. 8(b), as T increases, the generalization performance increases and then stays stable. Meanwhile, one can see that our AAUA is not sensitive to this hyperparameter setting.
19

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: The abstract and introduction states that the paper serves as the first paper to combat frequency shortcut learning for domain generalization.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
It is fine to include aspirational goals motivation as long as it is clear that these goals are not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: Limitations of the works is discussed at the second paragraph of the conclu- sion section.
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (c.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The au- thors should reflect on how these assumptions might be violated practice and what the implications would be.
The authors should reflect the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the ap- proach. For example, facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-tex system might not be used reliably provide closed captions for online lectures because fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach ad- dress problems of privacy and faimeess.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community, Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
20

---

Answer: [No]
Justification: The main theoretical observation referred from Pinson et al. [24], the theoretical analysis part serves motivational study.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theo- rems.
The proofs can either appear the main paper or supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be comple-
mented by formal proof' provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclu- sions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: Detailed implementation details to reproduce the main experimental results are provided in the appendix. The code would also be made publicly available after peer review.
Guidelines:
The answer NA means that the paper does not include experiments. If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary either make it possible for others replicate the model with the same dataset, or provide access to the model. In general. releasing code and data often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all sub- missions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily new algorithm, the paper should make it clear how to reproduce that algorithm.
(b) the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results way to re- produce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case au- thors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (c.g., to registered users), but it should be possible for other researchers to have some path to reproducing verifying the results.
21

---

5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, described in supplemental material?
Answer: [No]
Justification:
Code would made publicly available after peer review.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (uttps://nips.. cc/
httpl//gguiiees//oduuubbmissee wtt.esuuisssssoooooooooooooo onPolicy) for more details.
While we encourage the release of code and data, understand that this might not be possible, so No is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g,, for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https: / //nips. httpppullgggudddle lttgggdd//eoeuuumiiieee.e htt////euuuuissiieeeeee wwwmbbmmsssoooooooyy for more details.
The authors should provide instructions data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: Detailed implementation details are provided the appendix.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment Statistical Significance
Question:
Does the paper report error bars suitably and correctly defined or other appropri- ate information about the statistical significance of the experiments?
Answer: [Yes]
Justification:
Experiment results are reported correctly in the paper.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi-
dence intervals, statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
22

---

The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should prefer- ably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not show in tables figures symmetric error bars that would yield results that are out of range (c.g. negative emror rates).
If error bars reported in tables plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables the text.
8. Experiments Compute Resources Question:
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Answer: [Yes]
Justification:
The type of GPU used in the paper is stated the implementation details.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g.. preliminary or failed experiments that didn't t make it into the paper).
9. Code Of Ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https: : htte:///neeu /ttp///eeers.p.eeeeeeeeee http//pulic/thhcccuiuieeeee
Answer: [Yes]
Justification:
The research conducted in the paper is with the NeurlPS Code of Ethics.
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special cireumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., if there is a special consid- eration due to laws or regulations in their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [NA]
Justification: There is no socictal impact of the work performed.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact spe- cific groups), privacy considerations, and security considerations.
23

---

The conference expects that many papers will be foundational research and not tied particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there negative societal impacts, the authors could also discuss possible mitiga- tion strategies (e.g., gated release of models, providing defenses addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility MI..
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g.. pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification:
The paper does not pose safety risks.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by re- quiring that users adhere usage guidelines or restrictions access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do require this, but we encourage authors to take this into account and make best faith effort.
12. Lieenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: Citations made properly.
Guidelines:
The answer NA means that the paper does existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset. For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
assets are released, the license, copyright information, and terms of use in the pack- age should be provided. For popular datasets, paperswi thcode com/ dataests has curated licenses for some datasets. Their licensing guide can help determine the cense of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.

---

If this information is not available online, the authors are encouraged to reach out to the asset's creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documenta- tion provided alongside the assets?
Answer: [No]
Justification: Code would be made publicly available after peer review.
Guidelines:
The answer NA means that the paper does not release new assets,
Researchers should communicate the details of the dataset/code/moodl as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the pa- per include the full text instructions given to participants and screenshots, if applicable, well as details about compensation (if any)?
Answer: [NA]
Justification:
The paper does not involve crowdsourcing nor research with human subjects
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contri- bution of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According the NeurIPS Code of Ethics, workers involved in data collection, tion, or other labor should be paid at least the minimum wage the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does involve crowdsourcing research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equiva- lent) may be required for any human subjects research. you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
25


Reviews:
[{"review_id": "2BqjC6R7Uh", "reviewer": "Reviewer_uwHe", "summary": "This paper aims to develop data augmentation techniques to prevent the learning of frequency shortcuts and achieve enhanced generalization performances. They modify the frequency spectrum of the dataset statistical structure with aggressive frequency data augmentation, aiming to adaptively manipulate the model’s learning behavior on various frequency components. This paper proposes two effective and practical adversarial frequency augmentation modules, AAUA and AAD, designed to alter the frequency characteristic and prevent the learning of frequency shortcuts.", "questions": "Please see the Weaknesses.", "limitations": "Please see the Weaknesses.", "rating": 6, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "1. This paper explains and analyzes the frequency bias in neural network learning.\n2. The authors propose two effective and practical adversarial frequency enhancement modules, which are designed to dynamically alter the frequency characteristics of the dataset.\n3. The paper combines data augmentation and frequency analysis to address the learning behavior of frequency bias.", "weaknesses": "1.According to Figure 2, you used the AAUA and AAD networks for data augmentation. We would like to know the parameters and time complexity of the AAUA and AAD networks. Has the increase in parameters led to improved effectiveness? Do we still need to train your networks separately? How does the training time compare to that of the main network? Can we directly use the pre-trained parameters you provide for most networks?  \n2. There are currently many research methods such as splicing, augmentation, mixup, cutmix, Mosaic, etc. How is their performance in terms of frequency? What is the difference between them and yours?  \n3. The paper [1] elucidates that certain components of frequency are not conducive to network generalization, and therefore adaptive filtering out some frequencies may have additional effects if the remaining frequencies are enhanced. What are the differences between your paper and [1]? Can you compare it with [1]?  \n[1] Deep frequency filtering for domain generalization[C]//Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2023: 11797-11807.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "## Parameters and Time complexity.\n\n**(1) [No Additional Parameters]** We sincerely thank you for your insightful comments. Notably, as AAUA and AAD are data augmentation methods instead of neural networks, they don’t have any additional parameters. Thus, there won’t be additional parameters to train. \n\n**(2) [Additional Training Time and Efficency Study]**\nThere would be additional training time costs for generating the augmentation samples. We provide an efficiency study in terms of running time per epoch and performance improvement over baseline (ERM), including our method and related adversarial augmentation works[3,6,46] on PACS. Experiment results are shown in Tab. Re1. The time is calculated on a single NVIDIA A100-80G, with the same batch size of 32. The \"Photo\" domain of PACS is used as the training domain.\n\nAs can be seen, compared with the related adversarial augmentation SOTAs[3,6,46], we achieve the best performances with minimal additional training time over ERM, demonstrating the superior efficiency of our method. We reckon this is attributed to the parameter-free characteristic (i.e. ABA has an additional Bayesian network to help generate augmentation) and smaller iterative optimization steps of our method (Ours:5 steps, AdvST[46]: 50 steps, ALT[6]:10 steps, ABA[3]:14 steps).\n\n**(3) [Simple Application]**\nTo apply AAUA and AAD, one would only need to set the hyperparameters for them and input the images with corresponding labels.\n\n## Frequency Shortcut Evaluation with Traditional Augmentation methods.\n\n**(1) [Additional Experiments on Frequency Shortcut Evaluation]**\nWe follow your suggestion to conduct frequency shortcut evaluation on more augmentation methods, as shown in Tab. Re2. The experiment results demonstrate the effectiveness of our method in preventing frequency shortcut learning, while the traditional augmentation techniques fail to do so and leads to hallucinations of generalization ability improvement by applying more frequency shortcuts.\n\n**(2) [Method Differences]**\nThe core difference between our method and the recommended ones is that we conduct augmentation in the Fourier domain, while the mentioned methods focus on the spatial domain. Further, adversarial learning is also applied to enhance the augmentation samples' learning difficulty and to adaptively disrupt the learned frequency shortcuts.\n\n## Difference with Deep Frequency Filtering (CVPR'23, not open-source)\n\n**(1) [Data-centered vs Network-centered]**\nOurs is a data-centered method, and DFF is a network-centered method. We focus on preventing frequency shortcut learning with aggressive frequency augmentation, while DFF aims to learn adaptive Fourier filters for intermediate features. \n\n**(2) [Experiment Comparisons]**\nFollowing your suggestion, we provide a comparison with DFF (implemented by us, as the code of DFF is not open-source) in terms of frequency shortcut evaluation and performances on single-domain generalization in Tab. Re3. \nAs DFF learns the filter adaptively, with no additional regularization, it eventually learns more frequency shortcuts. Further, experiment results indicate that DFF would fail in the single-source domain generalization scenario.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "aqaiUn7FEB", "reviewer": "Reviewer_jMvT", "summary": "The paper proposes a method for single source domain generalization. The method is based on the insight of countering frequency shortcuts learnt during training. The paper proposes to augment images in the Fourier domain to achieve this. Such augmentation is done adversarially, by changing the mean and variance of low frequency components to maximize the loss on such samples, and by adversarially dropping some frequencies based on their gradients. The method is benchmarked on a variety of classification and retrieval tasks, and superior performance is demonstrated.", "questions": "* The choice of using AAUA only for low frequency components and AAD mainly for higher frequency seems a bit arbitrary. One could apply AAUA on both components, or only on high frequency components as well. Do the authors have an intuition/justification on whether this would work better?\n* What is the JS loss over? Is it consistency between predictions on augmented and clean samples?\n* The experimental setup is not clear enough - what are the training domains in Table 1? Was the experiment run multiple times? How were the hyper-parameters for each method chosen (i.e. was there an external validation set?) What is the in-domain accuracy (i.e. does AAUA hurt in-domain performance a lot)?\n* Fig 3 is hard to follow. What is the conclusion from Fig 3(a)? \n* In Fig.5, what do baseline tSNE plots look like?", "limitations": "The authors have adequately addressed their limitations.", "rating": 5, "confidence": 4, "soundness": 2, "presentation": 1, "contribution": 2, "strengths": "* Extensive ablation studies are performed on each component of the method.\n* Adequate background on the method is provided.", "weaknesses": "* The writing seems a bit belabored and hard to follow. The paper's presentation can be helped by a thorough rewrite, focussing on the grammar and readability.\n* Some of the implementation and experimental details are not clearly mentioned. See list of questions below.\n* Some of the design choices seem arbitrary, and I was not able to follow the motivation for them. See list of questions below.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "## Writing.\n\nThank you for your detailed comments. We will thoroughly revise the manuscript.\n\n## Implementation details and clarity of experiment results.\n\n**[JS loss]** Yes, the JS loss is applied between predictions on augmented and clean samples\n\n**[Training domains]**\n\t\n**(1)** Kindly note that the training domains are illustrated in the captions of Tables. 5,6,7 in the appendix of the submission (Line:494,506). We follow this established protocol from previous works[43,36,6,3,46].\n\n**(2)** For Digits, MNIST is used as the training domain. For PACS, each of the four domains in PACS is used as the source domain in turn, with the remaining three domains serving as target domains. This process is repeated for each domain as the source, and the model performance on the target domains is averaged to obtain the results in the table. For CIFAR-10, the training domain is CIFAR-10 itself, and CIFAR-10-C is the target domain. \n\n**[Multiple Runs]** Yes, we've run the experiments three times and provided the average results.\n\n**[Hyper-parameter choosing]** Following previous works[43,36,6,3,46], there is an external validation set for choosing models to evaluate the performance on the unseen target domains. We choose the hyper-parameters with a hyper-parameter study on the model generalization performances.\n\n**[In-domain performances]** In the SDG scenario, we generally don't focus much on in-domain performances[37,38,22,43]. We follow your suggestion to provide in-domain performance comparisons on PACS. As shown in Tab. Re4, compared with prior related SOTAs, we generally outperform them. Despite our method only achieving similar in-domain performance against ABA[3] and ALT [6] on the “Photo” source domain, we outperform the related SOTAs across all the metrics with clear advantages.\n\n**[Explanation of Fig. 3(a)]** We are sorry for the confusion caused. Fig 3(a) should be referred to in Sec. 5.4.1. In Sec. 5.4.1 we would like to justify that, given the different frequency preferences of the baseline and ours as shown between Fig. 3(a) and (b), the proposed techniques could indeed modify the model’s frequency preferences. We would further revise the manuscript and provide an illustration of Fig. 3(a) in Sec. 5.4.1.\n\n**[Baseline tSNE plots]**\n We follow your suggestion to provide a baseline tSNE visualization in Fig. Re1, with ERM baseline and AugMix baseline. As can be seen, feature distributions in the baselines are more distinct from each other, indicating a weaker signal of generalization performance compared with our method.\n\n## Module design choices in AAUA.\n\n**(1) [Intuition]** Intuitively, low-frequency components are the dominant ones in real-world datasets, which could have a higher possibility of containing frequency shortcuts. Therefore, we apply the current design of forcing AAUA to inject aggressive adversarial perturbation into low-frequency components only.\n\n**(2) [Justification]** Following your suggestion, we further provide a comparison with different settings of the AAUA in terms of generalization performances. As shown in Tab. Re5, we compare the low-frequency-centered settings (Ours) with two settings from your advice. One is applying it to both low and high frequency components(All), and only on the high frequency components (Reverse). As can be seen, our setting achieves the strongest performances, with advantages over the “All” setting, and significantly outperforms the “Reverse” setting. The experiment results justify the low-frequency-centered design of AAUA. \n\nWe sincerely thank you for this valuable comment, and we will further clarify it in the revised manuscript with the intuitive motivation and ablation study justification of the AAUA design.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "CPHFLSblRz", "reviewer": "Reviewer_57HS", "summary": "The paper addresses the challenge of domain generalization by focusing on the issue of frequency simplicity-biased learning in neural networks. This phenomenon leads to an over-reliance on specific frequency sets, known as frequency shortcuts, instead of semantic information, thereby impairing generalization performance. The innovative data augmentation techniques adaptively manipulate the learning difficulty of different frequency components, thereby enhancing domain generalization performance by combating frequency shortcut learning.", "questions": "Why did you choose the specific baselines for comparison, and how do you think your methods would perform against other recent or diverse approaches?\nWhat is the computational overhead associated with implementing AAUA and AAD?", "limitations": "The authors acknowledge the challenge of directly locating frequency shortcuts. Providing preliminary insights or future directions on how to address this limitation would be a constructive addition. This could involve discussing potential methods for identifying and mitigating these shortcuts more directly.", "rating": 5, "confidence": 3, "soundness": 2, "presentation": 2, "contribution": 3, "strengths": "The paper presents a novel approach to tackling the issue of frequency simplicity-biased learning in domain generalization. The introduction of adversarial frequency augmentation modules, specifically Adversarial Amplitude Uncertainty Augmentation (AAUA) and Adversarial Amplitude Dropout (AAD), is innovative. These methods provide a new perspective by manipulating the dataset's frequency characteristics to prevent over-reliance on specific frequency components, which is a significant departure from traditional data augmentation techniques. This creative combination of adversarial learning with frequency domain manipulation represents a fresh and original contribution to the field.The theoretical foundation provided in the paper is robust, offering a clear justification for the proposed methods. The experimental validation is thorough, with the proposed techniques being rigorously evaluated across various benchmarks, including image classification and instance retrieval tasks. The results demonstrate a substantial improvement over state-of-the-art methods, highlighting the effectiveness of the proposed approach.", "weaknesses": "The effectiveness of AAUA and AAD might be sensitive to the choice of hyperparameters, such as the intensity of adversarial perturbations and the threshold for frequency dropout. The paper does not provide an in-depth analysis of the sensitivity of the results to these hyperparameters. The proposed adversarial frequency augmentation methods, AAUA and AAD, involve iterative optimization steps and adversarial gradient computations, which could be computationally expensive.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "## More Hyper-Parameter Analysis.\n\n**(1) [Additional Results]** Thank you for your valuable comments. Following your suggestion, we further provide hyper-parameter studies of the intensity of AAUA perturbations and the probability threshold $p$ for AAD in Tab. Re6 and Tab. Re7, respectively.\nAs we don't have a clamping operation to directly restrict the noise intensity like traditional adversarial attacks [Re1] do, we thus provide an ablation on the update step size of the AAUA, as this would directly control the intensity of AAUA given static iterative steps of 5.\n\n**(2) [Additional Analysis]** As shown in Tab. Re6 and Tab. Re7, the hyper-parameter setting in the main paper generally achieves the best performance. In Tab. Re6, it can be seen that both overly strong and weak perturbation intensity leads to sub-optimal performances. Because an overly strong intensity would produce augmentation domains with excessive domain-shift to learn while an overly weak intensity fails to create enough domain shift. In Tab. Re7, it be seen that a relatively higher threshold leads to better performances, because it could help filter out the frequencies with higher dependency.\n\n## Computational Overhead Analysis.\n\n**(1) [Additional Experiments]** We agree that the proposed technique, involving adversarial gradient computations, would bring additional computation overhead. We conducted an efficiency study to evaluate running time per epoch and performance improvement over the baseline method (ERM). This study includes our proposed method and several related adversarial augmentation techniques [3,6,46] on the PACS dataset. The experimental results are presented in Tab. Re1. All timings were measured on a single NVIDIA A100-80G GPU with a batch size of 32. The \"Photo\" domain of PACS was utilized as the training domain.\n\n**(2) [Experiment Analysis]** As illustrated in Tab. Re1, our method achieves the highest performance compared to the state-of-the-art adversarial augmentation techniques [3,6,46], with only a minimal increase in training time over ERM. This demonstrates the superior efficiency of our approach, which we attribute to its parameter-free nature (i.e. ABA has a Bayesian network to help generate augmentation) and the reduced number of iterative optimization steps (Ours:5 steps, AdvST: 50 steps, ALT:10 steps, ABA:14 steps).\n\n## Reason for Choosing the Specific Baselines and Comparison with Other Methods.\n\n**(1)** Most of the chosen baselines are adversarial augmentation methods and Fourier augmentation methods, which are highly related to ours.\n\n**(2)** Following your suggestion, we further include more diverse methods for comparison in Tab. Re8, showing comparative performances of the proposed method. Due to space limits, we show below the citations to the corresponding works in Tab. Re8.\n\n\n\n## References:\n\nRe1: Madry, Makelov, et al. \"Towards Deep Learning Models Resistant to Adversarial Attacks.\" ICLR (Poster) 2018\n\nMetaCNN: Chen, Jin, et al. \"Meta-causal learning for single domain generalization.\" *Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition*. 2023.\n\nPro-RandConv: Choi, Seokeon, et al. \"Progressive random convolutions for single domain generalization.\" *Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition*. 2023.\n\nR-XCNorm: Chuah, WeiQin, et al. \"Single Domain Generalization via Normalised Cross-correlation Based Convolutions.\" *Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision*. 2024.\n\nPDDOCL: Li, Deng, et al. \"Prompt-Driven Dynamic Object-Centric Learning for Single Domain Generalization.\" *Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition*. 2024.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "e5IYTaSweJ", "reviewer": "Reviewer_szDc", "summary": "This paper proposed an augmentation technique in frequency domain, motivated by the phenomenon of frequency bias/frequency shortcut learning. The technique AAUA adversarially perturbs frequency components of images  which the models over-rely for classification. As AAUA might encourage shortcut learning in high-frequency, the authors further use AAD to drop out randomly frequency components that models highly depend on for classification. AAUA and AAD together can avoid frequency shortcut learning. It is shown that AAUA and AAD together can improve domain generalization capability of models.", "questions": "1. How are d_t^mu and the other initialized? Did the experiments consider different random seeds?\n\n2. Speaking of hardness of the augmented samples, how do AAUA and AAD compare with other adversarial techniques? \n\n3. Line 121-123: If correctly understood, it should be: responsive frequency describes the characteristics of mapping functions learned by networks and the mapping function transforms images into probability vectors. How does the smoothness represented by the probability vectors and how does it relate to frequency? The concept of 'smoothness' appears abruptly. \n\n4. What are the meaning of the two target domains in Fig. 2? Are they to show the distributions of augmented images are close to them? How are the density distributions of each domain computed? What do the peak values of each distribution mean?", "limitations": "The authors addressed the limitations of their methods, regarding the infeasibility to directly locate frequency shortcuts.", "rating": 5, "confidence": 3, "soundness": 2, "presentation": 2, "contribution": 3, "strengths": "1. Well-written introduction. The introduction explains well the domain generalization problem and how it related to frequency bias / frequency shortcut learning. \n2. Interesting and insightful ablation study on the impact of AAUA and AAD on frequency shortcut learning. The authors observed that AAUA or AAD solely might encourage more shortcut learning, supporting the necessity to apply both AAUA and AAD together.", "weaknesses": "1. The first and second contribution points can be merged for a compact presentation. Moreover, whether this is the first work to combat frequency shortcuts for domain generalization is uncertain. Domain generalization is a broad topic, and it includes single-source domain generalization problem, e.g. training models on a clean dataset and evaluate them in a corrupted dataset. And the [1] focuses on reducing frequency shortcut learning to improve model robustness towards image corruptions, one of domain generalization problems. \n\n2. As this work proposed a frequency augmentation technique, there are more related work despite FACT and SADA mentioned in the paper, such as VIPAug [2], AFA [3], HybridAugment++ [4], etc. \n\n3. Vague statement: line 327-328: it is uncertain how the comparison is operated given the condition 'without the proposed AAD or AAUA module'. One has to calculate on their own to figure out the comparison is between AAD+L_JS and AAD+AAUA+L_JS, and AAUA+L_JS and AAD+AAUA+L_JS. \n\n4. Sec. 3.2 Theoretical justification is from others' work, motivating the design of the techniques but not having direct mathematical relationship with the proposed technique.  The input-input correlation matrix is introduced but not used.  Strictly speaking, Sec. 3 is more like a literature review explaining motivations instead of analysis carried out by the authors. \n\n5. Definition of the masks separating low and high frequency components: h_min of M_low is H/4, meaning that for frequency components close to the center will be considered as high-frequency, conflict to what is considered as low-frequency. This needs more clarification on why h_min is not 0. Same for w_min.  From the definition of M_low, AAUA seems to shift models' attention to partial mid-high frequencies. \n\n6. Experiments need to be comprehensive: \n\n*As AAUA and AAD augment images in the frequency domain, comparison with more other frequency augmentation techniques should be included, such as HybridAug++ [4], VIPAug [2], AFA [3], DFM-X [1], AugSVF [5]. \n\n*The experiments are limited to small datasets, with a few classes. It lacks comprehensive experiments on large datasets, e.g. ImageNet-1k with 1000 classes, to show the feasibility of the proposed technique as well as a fair comparison with other methods which provide results on ImageNet-1k. \n\n*Some results in Table 1 are missing and there lacks explanations. \n\n*In Sec. 3.1 the authors claimed that they focus on classification tasks. However, they abruptly show results on instance retrieval. It is understood that domain generalization is important to instance retrieval. But there needs more clarifications on how the proposed technique can address frequency shortcut learning in retrieval tasks when there are not many work analyzing shortcut learning in image retrieval. \n\n*Sec. 5.3 evaluates frequency shortcuts of trained models. Intuitively, the analyzed models are those trained in Sec. 5.2. However, the evaluation is carried out on ImageNet-10, which is not mentioned previously and not used for training. It is unclear at this stage whether the authors use the models trained in Sec. 5.2 for shortcut evaluation on the dataset ImageNet-10, or train new models on ImageNet-10 for shortcut evaluations. This needs clarifications.\n\n*Sec. 5.4.1 demonstrates that the proposed augmentation technique increases the frequency sensitivity of models compared to baseline. This seems to be a bad sign to model generalization capability according to [43]. However, there is no related discussions and Fig. 3a is not referred anywhere in the main paper. \n\n*Sec. 5.4.2 analyzes the hardness of augmented samples. However, the comparison is between AAUA and FACT, which seems unfair as FACT does not include adversarial setup while AAUA does. \n\nMinor: \nEach formula should end with period/comma. Line 271 'relative' should be 'related' if understood correctly. Table 3 not in the same page where mentioned. Table 4 exceeds width limit. \n\n[1] Wang, et al., 'DFM-X: Augmentation by Leveraging Prior Knowledge of Shortcut Learning', ICCVW2023.\n\n[2] Lee, et al., 'Domain Generalization with Vital Phase Augmentation', AAAI2024.\n\n[3] Vaish, et al., 'Fourier-basis Functions to Bridge Augmentation Gap: Rethinking Frequency Augmentation in Image Classification', CVPR2024.\n\n[4] Yucel, et al., 'HybridAugment++: Unified Frequency Spectra Perturbations for Model Robustness', ICCV2023.\n\n[5] Soklaski, et al., 'Fourier-Based Augmentations for Improved Robustness and Uncertainty Calibration', Neurips2021.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "## Merging Contributions\n\nThank you for your valuable comments. We will merge the first two points as suggested.\n\n## Claim as The First\n\n**(1) [General Robustness vs. Domain Generalization(DG)]** We acknowledge the correlation between general robustness studies (i.e. DFM-X) and DG, but respectfully highlight that DG involves stronger distribution shifts and semantic variances, compared to simply training on clean data and testing on its corruption subset.\n\n**(2) [Specific Focus on Frequency Shortcuts]** We evaluate frequency shortcut mitigation and improved DG performance, offering a novel contribution compared to DFM-X, which uses learned frequency shortcuts for augmentation but lacks a comprehensive evaluation of frequency shortcut defending.\n\n**(3) [Further Experiments]** As shown in the latter, the failure of the recommended papers (mostly work on general robustness) on single source DG, underscores the distinction between these two challenges. If general robustness were equivalent to DG, these papers would not consistently fail on the DG dataset. This suggests that general robustness and DG are indeed distinct tasks.\n\n**(4) [Further Modification]** We are aware that there might be potential uncertainty, and we would replace the “first work” with “one of the pioneering works”.\n\n## Comparison with More Papers\n\n**(1) [Additional Experiments]**  We compare with recommended papers on PACS(DG), ImageNet-1k(general robustness), and ImageNet-10(frequency shortcut). Results in Tab. Re9,Re10,Re11 show our method's competitiveness, outperforming others on PACS and ImageNet-10 while remaining competitive on ImageNet-1k.\n\n**(2) [Reason of Dataset Scope]** We didn't compare these methods and on ImageNet because our focus is on DG, with relevant methods (FACT and SADA) and established datasets. Our dataset scope aligns with established single-source DG papers[43,36,6,3].\n\n**(3) [Large-scale DG dataset]** We've added results on a larger-scale DG dataset: DomainNet, showing superior performance (Tab. Re10).\n\n## Statement on Line 327-328\n\nWe are sorry for the confusion caused. We will add a performance degradation term in the ablation table for clarity so that one can easily understand the comparison with detailed numbers, as shown in Tab. Re12.\n\n## Theory Section\n\n**(1)** The theory section is to bridge the gap between the theoretical proof in [24] and the frequency shortcut paper[35]. While [24] doesn't mention frequency shortcuts and [35] is purely empirical, we connect these works (Line:160-173), motivating our method. We've correctly cited [24] for the theory.\n\n**(2)** We will remove the spare input-input correlation matrix, while it is represented by \"the dataset statistical structure\" in the paper.(i.e. Line:151 and Line:191).\n\n**(3)** The matrix is calculated on the entire dataset. Yet, we can't backward on the entire dataset in one back propagation (due to GPU memory), thus we don’t directly use it and propose our in-batch technique.\n\n## Mask in AAUA\n\n**(1)** We clarify that the lowest frequency is shifted to the center after FFT. Thus, frequency in coordinate (2/H, 2/W) is the lowest frequency and higher frequencies are farther from this center point.\n\n**(2)** AAUA alone would shift the model's attention to partial mid-high frequencies(Line:239-240). This is based on the intuition that low frequencies dominate real-world datasets and are more likely to hide frequency shortcuts. \n\n## Comprehensive Experiments\n\n**[Missing Results]** Results in Tab. 1 are drawn from the corresponding papers, the missing ones indicate they didn’t run experiments on the corresponding datasets.\n\n**[Shortcut Learning in Re-ID]** The re-ID model is trained with the same loss as classification (each person has a corresponding ID), resulting in the same optimization targets. We'll provide a brief explanation of re-ID, but omit detailed analysis due to its overlap with classification.\n\n**[Frequency Shortcut Evaluation]** We follow the protocol in [35] for shortcut evaluation, where the models are trained on ImageNet-10. We will further clarify it.\n\n **[Sensitivity Map in Fig. 3]**  **(1) [Different Definition]** We kindly emphasize that sensitivity maps in [43] are different from ours. They are accumulation errors on the source domain with Fourier-basis noise, while ours are derived from gradients (as shown in Appendix A.5).  **(2) [Uncertain Claim]** [43] only provides sensitivity maps on the source domain. The claim \"lower sensitivity, better generalization\" remains uncertain for target domains (conclusion section of [43]). **(3) [Additional Figure]** Additional sensitivity maps (Fig. Re3) following [43], show that our method suppresses source domain sensitivity [43], indicating good generalization. Fig.3(a) would be cited in Sec.5.4.1\n\n **[Hardness Comparison]** Adversarial is a core component of our method. Thus, the comparison is fair. Fig. Re2 shows that AAUA covers a larger range than adversarial SOTAs (ABA[3] and ALT[6]), demonstrating higher hardness.\n\n **[Initialization and Random Seeds]** $d_t^{\\mu}$ and others are randomly initialized with a normal distribution N(0,1). Experiments are run over three different random seeds.\n\n## \"Smoothness\" and Responsive Frequency\n\nHigh responsive frequency means that small input changes would significantly affect the model output (i.e. adversarial attack). As this concept is closely related to the Lipschitz constant, which describes the model's smoothness, we thus use the term.\n\n## Manifold Map\n\n**(1)** We assume that you are referring to Fig. 3(b). The target domains demonstrate our effectiveness in crafting augmentation domains, rather than suggesting the augmented images are close to the target ones, as they are unseen in DG. **(2)** Density distributions are computed with kernel density estimation based on the t-SNE results. One could do so with the Python package seaborn.kdeplot. Peak values mean more augmented images lie in that point after dimension reduction.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
We thank the reviewers (uwHe, jMvT, 57HS, szDc) for all the informative and constructive feedback and we appreciate the comments to improve our work. 

**Reviewer uwHe**: "Two effective and practical adversarial frequency enhancement modules. Combines data augmentation and frequency analysis to address the learning behavior of frequency bias."

**Reviewer jMvT**: "Extensive ablation studies. The method is benchmarked on a variety of classification and retrieval tasks, and superior performance is demonstrated."

**Reviewer 57HS**: "Innovative data augmentation techniques, provide a new perspective. The theoretical foundation is robust and experimental validation is thorough."

**Reviewer szDc**: "Well-written introduction. Interesting and insightful ablation study."

We've provided detailed explanations to address each concern from the reviewers with additional experiment results attached in the individual PDF file. We summarize the main points presented in our response and we kindly hope that we have addressed all the concerns.

- We address the questions and provide clarifications and details that have improved our work.
- We include comparisons with more papers and experiments on large-scale datasets to show the effectiveness of our method.
- We include more variants of our proposed method that bring further insight into our choice.
- We will update the final version to incorporate all the additional results with the reviewers' comments.

Meta-review: The paper received mixed ratings at the first place. After discussion, Review szDc, who gave the most negative rating, changed his/her mind to increase the score to 5. leading to all positive, borderline accept ratings. After carefully reading the paper and all the rebuttal, the decision is a borderline acceptance. While the average score is not high enough, the paper truly has some merits, including novel problem setting, sufficient theoretical analysis, and compeling experimental results. The following is a summary of the review.

Reviewers acknowledge the innovative approach of using adversarial frequency augmentation techniques, AAUA and AAD, to mitigate frequency shortcuts in neural networks and improve domain generalization (reviewer uwHe, 57HS). However, the paper is criticized for lacking detailed explanations of the computational complexity, hyperparameter sensitivity, and training costs associated with the proposed methods (reviewer uwHe). Additionally, the experimental design and presentation are seen as problematic; the clarity of implementation details is insufficient, with arbitrary design choices and unclear motivations for some decisions (reviewer jMvT). The evaluations are limited in scope, involving only small datasets and missing comparisons with relevant frequency augmentation techniques, which undermines the generalizability and applicability of the findings (reviewer szDc). Moreover, inconsistencies in the theoretical justification and an unclear distinction from prior work further weaken the contribution (reviewer szDc). The writing and structure also need improvement, as the presentation is difficult to follow and lacks coherence in explaining the experimental results (reviewer jMvT). 

Overall, while the idea of addressing frequency bias in domain generalization is recognized as valuable.
Decision: Accept (poster)