Robust Learning with Progressive Data Expansion Against Spurious Correlation
Authors: Yihe Deng
Yu Yang
Baharan Mirzasoleiman
Quanquan Gu
Venue: NeurIPS 2023 poster (2023)
Keywords: spurious correlation
robustness
robust learning
TL;DR: This paper theoretically examines the impact of spurious features on deep learning models, proposing a new training algorithm called PDE that effectively improves worst-group accuracy.

Abstract: While deep learning models have shown remarkable performance in various tasks, they are susceptible to learning non-generalizable _spurious features_ rather than the core features that are genuinely correlated to the true label. In this paper, beyond existing analyses of linear models, we theoretically examine the learning process of a two-layer nonlinear convolutional neural network in the presence of spurious features. Our analysis suggests that imbalanced data groups and easily learnable spurious features can lead to the dominance of spurious features during the learning process. In light of this, we propose a new training algorithm called **PDE** that efficiently enhances the model's robustness for a better worst-group performance. PDE begins with a group-balanced subset of training data and progressively expands it to facilitate the learning of the core features. Experiments on synthetic and real-world benchmark datasets confirm the superior performance of our method on models such as ResNets and Transformers. On average, our method achieves a $2.8$ \% improvement in worst-group accuracy compared with the state-of-the-art method, while enjoying up to $10\times$ faster training efficiency.

Robust Learning with Progressive Data Expansion Against Spurious Correlation
Yihe Deng" Yu Yang" Baharan Mirzasoleiman
Quanquan Gu
Department of Computer Science University of California, Los Angeles
Los Angeles, CA 90095
Sylhedeng,yuyange waw.yyyaagnnbaaaaaannaaa baharan, qgu)aacs. wcwa..llle...o
Abstract
While deep learning models have shown remarkable performance in various tasks, they are susceptible to learning non non-generalizable spurious features rather than the core features that are genuinely correlated to the true label. In this paper, beyond existing analyses of linear models, we theoretically examine the learning process of two-layer nonlinear convolutional neural network in the presence of spurious features. Our analysis suggests that imbalanced data groups and easily learnable spurious features can lead the dominance of spurious features during the learning process. In light of this, we propose a new training algorithm called PDE that efficiently enhances the model's robustness for better worst-group performance. PDE begins with a group-balanced subset of training data and progressively expands it facilitate the learning of the core features. Experiments on synthetic and real-world benchmark datasets confirm the superior performance of our method on models such as ResNets and Transformers. On average, our method achieves 2.8% improvement in worst-group accuracy compared with the state-of-the art method, while enjoying up to 10x faster training efficiency. Codes are available at https: /tt///gittt github....... com/ hamm/uclamm/PP
Introduction
Despite the remarkable performance of deep learning models, recent studies (Sagawa et al. 2019 2020 Izmailov et al. 2022 Haghtalab et al. 2022: Yang et al. 2022 2023b ch have identified their
vulnerability to spurious correlations in data distributions. A spurious correlation refers to an easily learned feature that, while unrelated to the task hand, appears with high frequency within specific class. For instance, waterbirds frequently appear with water backgrounds, and landbirds with land backgrounds. When training with empirical risk minimization (ERM), deep learning models tend to exploit such correlations and fail to learn the more subtle features genuinely correlated with the true labels, resulting in poor generalization performance minority data (e.g., waterbirds with land backgrounds as shown in Figure D]. This observation raises a crucial question: Does the model genuinely learn classify birds, or does it merely learn distinguish land from water? The issue is particularly concerning because deep learning models are being deployed in critical applications such healthcare, finance, and autonomous vehicles, where require reliable predictor.
Researchers formalized the problem by considering examples with various combinations core features (c.g., landbird/waterbird) and spurious features (c.g., land/water backgrounds) as different groups. The model is more likely to make mistakes on certain groups if it learns the spurious feature. The objective therefore becomes balancing and improving performance across all groups. Under this formulation, we can divide the task into two sub-problems: groups, which are not always known in dataset, and (2) effectively using the group information
(1) accurately identifying the
"Equal contribution
37th Conference on Neural Information Processing Systems (NcurIPS 2023).

---

(A)
Training data
Our melhad
(B)
am
Warniag balanced groups
Expansion slage: progrossive expansion
Waterbird dataset statistics
Examples
GradCAM results
Original
Core feature
ERM
Our methed
landiard
edoir
wateciid
ERM: 72.7%
Ours: 90.0%
Figure I: overview of the problem, our proposed solution, and the resultant outcomes. (A) We demonstrate the data distribution and provide an example of the statistics of Waterbirds. (B) The overall procedure of PDE. (C) we use GradCAM (Selvaraju et al., 2017) to show the attention of the model trained with PDE as compared to ERM. While ERM focuses the background, PDE successfully trains the model to capture the birds.
to finally improve the model's robustness. While numerous recent works (Nam et al., 2020 Liu al. (202T] Creager al. 2021 Ahmed et al., 2021] TTahanaki et al. 2021| Zhang et al. 2022) focus on the first sub-problem, the second sub-problem remains under studied. The pioneering work (Sagawa et al., (219)) still serves as the best guidance for utilizing accurate group information. In this paper, we focus on the second sub-probe sub-problem and aim provide more effective and efficient is worth noting that the theoretical understanding of spurious correlations lags behind the empirical advancements of mitigating spurious features. Existing theoretical studies (Sagawa et al. 2020 Chen et al. 2020 Yang et al. 2022 Ye et al. 2022) limited to the setting of simple linear models and data distribution that are less reflective of real application scenarios.
algorithm to utilize the group information.
We begin by theoretically examining the learning process of spurious features when training two- layer nonlinear convolutional neural network (CNN) on a corresponding data model that captures the influence of spurious correlations. We illustrate that the learning of spurious features swiftly overshadows the learning of core features from the onset of training when groups are imbalanced and spurious features more easily learned than core features. Based upon theoretical understanding,
we propose Progressive Data Expansion (PDE), a neat and novel training algorithm that efficiently group information enhance model's robustness against spurious correlations. Existing approaches, such as GroupDRO (Sagawa et al.) 2019) and upsampling techniques (Liu et al., 2021), aim to balance the data groups in each batch throughout the training process. In contrast, employ a small balanced warm-up subset only at the beginning of the training. Following a brief period of balanced training, progressively expand the warm-up subset by adding small random subsets of the remaining training data until using all of them, as shown in the top right of Figure Here, we utilize the momentum from the warm-up subset to prevent the model from learning spurious features when adding new data. Empirical evaluations on both synthetic and real-world benchmark data validate our theoretical findings and confirm the effectiveness of PDE. Additional ablation studies also demonstrate the significance and impact of each component within our training scheme. In summary, our contributions are highlighted as follows:
We provide a thcoretical understanding of the impact of spurious correlations beyond the linear setting by considering two-layer nonlinear CNN.
We introduce PDE, a theory-insping approach that effectively addresses the challenge posed by spurious correlations.
PDE achieves the best performance on benchmark vision and language datasets for models including ResNet and Transformer. On average, it outperforms the state-of-the-arr method by 2.8% in terms of worst-group accuracy.
-DE enjoys superior training efficiency, being 10x faster than the state-of-the -f-the--rttttt methods.

---

2 Why is Spurious Correlation Harmful to ERM?
In this section, we simplify the intricate real-world problem of spurious correlations into a theoretical framework. We provide analysis on two-layer nonlinear CNNs, extending beyond the linear setting prevalent in existing literature on this subject. Under this framework, we formally present our theory concerning the training process of empirical risk minimization (ERM) in the presence of spurious features. These theoretical insights motivate the design of our algorithm.
2.1 Empirical Risk Minimization
We begin with the formal definition of the ERM-based training objective for binary classification problem. Consider a training dataset S ((x,)), |i where x; E IRd is the input and E (+1) } is
the output label. We train model f(x; W) with weight W to minimize the empirical loss function:
N ((W) = & (vr W)))
N i=
(2.1)
where is the logistic loss defined as {(z) = log(1 + exp( The empirical risk minimizer
refers to W that minimizes the empirical loss: W == argminly L(W). Typically, gradient-based optimization algorithms are employed for ERM. For example, at each iteration t, gradient descent (GD) has the following update rule:
W W V/LC(W()
(2.2)
Here, 17 > 0 is the learning rate. In the next subsection, we will show that even for a relatively simple data model which consists of core features and spurious features, vanilla ERM will fail to learn the core features that are correlated to the true label.
2,2 Data Distribution with Spurious Correlation Fails ERM
Previous work such as (Sagawa et al. 2020) considers a data model where the input consists of core feature, spurious feature and noise patches at fixed positions, i.c., x = [Xeore Xspu Xaoise]. In real-world applications. however, features in an image do not always appear at the same pixels. Hence, we consider more realistic data model where the patches do not appear at fixed positions.
Definition 2.1 (Data model). A data point (x, y, a) E (IR4) (+1) (+1) is generated from the distribution D as follows.
Randomly generate the true label {11]
Generate spurious label a E { where
with probability a > 0.5.
Generate x as collection of patches: x = (x(2), x(2) ..... ....,,(()) (P)) E (Rd)P, where 1.- Ve with ||vell 1. Core feature. One and only one patch is given by Be B.
-Spurious feature. One and only one patch is given by Bg a. with ||v = 1 and (ve. = 0. Random noise. The rest of the P 2 patches are Gaussian noises & that are independently drawn from (0, ( (d) La) with op as an absolute constant. And 0 < 3e < E.
-
Similar data models have also been considered in recent works on feature learning (Allen-Zhu & Li 22000) You et al., 202T Chen al. 2022 Telassi & Li] 2022). where the input data is partitioned into feature and noise patches. We extend their data models by further positing that certain feature patches might be associated with the spurious label instead of the true label. In the rest of the paper, we assume P for simplicity. With the given data model, we consider the training dataset S {(x, yt: ax) a and let be partitioned into large group Si and small group S2 such that S; contains all the training data that can be correctly classified by the spurious feature, i.e., 0; = yi. and S2 contains all the training data that can only be correctly classified by the core feature, i.e., -y--- We denote [8al
y-1
a=+1
0.5
Decision boundary
0.5
Figure 2: Visualization the data.
1h11 and therefore
Visualization of our data. In Figure 2 we present the visualization in 2D space of the higher- dimensional data generated from our data model using t-SNE (Van der Maaten & Hinton 2008). where data within each class naturally segregate into large and small groups The spurious feature is sufficient for accurate classification of the larger group data, but will lead misclassification of the small group data.
3

---

2.3 Beyond Linear Models
We consider two-layer nonlinear CNN defined follows:
f(x: w) ΣjE(J Σp= ((w),x(1)))
(2.3)
where Wy E Fd is the weight vector of the j-th filter, J is the number of filters (neurons) of the network, and (22) =2 the activation function. W WW[w1......
[Wi...., .....,,w]] E Rx denotes the weight matrix of the CNN. Similar two-layer CNN architectures are analyzed in in (Chen et al. 2022] Jelassi & Li 2022) but for different problems, where the cubic activation serves simple function that provides linearii.. Similar to Jelassi & Li (202)): Cao et al. (022), we assume mild overparameterization of the CNN with J polylog d). We initialize W ~ (O where
of = polylog(d) Due the CNN structure, our analysis can handle data models where each data can have arbitrary order of patches while linear models fail to do so.
2.4 Understanding the Training Process with Spurious Correlation
In this subsection, we formally introduce our theoretical result on the training process of the two-layer CNN using gradient descent in the presence of spurious features. We first define the performance met- rics. A frequenly considered metric the test accuracy: Acc( Acc(W)= (x.g.a))> [s W)) = y)
With spurious correlations, researchers are more interested in the worst-group accuracy:
min Aco ()= [=(+1),0E==1} ((x.v.))~D [sgn( (xx; W)) = y)
P
which accesses the worst accuracy of model among all groups defined by combinations of y and a.
We then summarize the learning process of ERM in the following theorem. Our analysis focuses on
the learning of spurious and core features, represented by the growth of (w("). va) and (w (t) Ve)
respectively:
Theorem 2.2. Consider the training dataset 5 {(x,, 31) ) that follows the distribution in
Definition 2.1 Consider the two-layer nonlinear CNN model as in (2.3) initialized with w())
(O of) After training with GD in 2.2) for To ((1 oo)) iterations, for all j [J] and
t e [0, To). have
e(m) - 1). (w) vz)² < (++1) v2) (w]) (t)
5 v.) - (())saa (w-")) v.)² (t)
(2.4)
ē(n)saa . (w vv)2 <
ve) (w/" } ve) - (())) (w/)) ve)²
(2.5)
After training for To iterations, with high probability, the learned weight has the following properties: (1) learns the spurious feature v,: ma (T) v.)) 2 ((1/3.); (2) it almost does not learn the
core feature Ve: max (w) (T) ve) = Ö(oo).
Discussion. The detailed proof is deferred Appendix E. and we provide intuitive explanations of the theorem as follows. A larger value of (w/() v) for v E (v., Ve} implies better learning of the feature vector by neuron w, at iteration t. As illustrated in (2.4) and (2.5). the updates for both spurious and core features are non-zero, as they depend on the squared terms of themselves with
non-zero coefficients, while the growth rate of (w(") (t) vs) is significantly faster than that of (w (t) va)- Consequently, the neural network rapidly learns the spurious feature but barely learns the core feature,
as it remains almost unchanged from initialization.
We derive the neural network's 's prediction after training for To iterations. For a randomly gener- ated data example (x, y, a) ~ D, the neural network's 's prediction is given by sgn(f(x: W)) sgn( ESecJ (w,, vo)3 ( (wj, vz) dominates the summation, the prediction will be sxn(f(x; W)) = a. Consequently, we obtain the test accuracy as Acc(W)= a, since y with probability a, and the model accurately classifies the large group. However, when considering the small group and examining examples where y & a, the models consistently make errors, resulting in Acc (W) 0. To cireumvent this poor performance on worst- group accuracy, an algorithm that can avoid learning the spurious feature is in demand.
+ {) Since the term B maxjejj] (w,, vz)
3 -nsprrddd Two-Stage Training Algorithm
In this section, we introduce Progressive Data Expansion (PDE), novel two-stage training algorithm inspired by our analysis to enhance robustness against spurious correlations. We begin with illustrating

---

the implications of our theory, where we provide insights into the data distributions that lead to the rapid learning of spurious features and clarify scenarios under which the model remains unaffected.
3.1 Theoretical Implications
Notably in Theorem 2.2 the growth of the sequences (w: Vs) in (2.4) and (w(?) Ve) in 2.5
follows the formula + = 24 + 7)A2, where T2 represents the inner product sequence with regard iteration and the coefficient containing ā, 3g or 3g. This formula is closely related to the analysis of tensor power methods (Allen-Zhu & Li 2020). In simple terms, when two sequences have slightly different growth rates, one of them will experience much faster growth in later times. As we will show below, the key factors that determine the drastic difference of spurious and core features in later times are the group size ā and feature strengths Be, B,,.
When the model learns spurious feature (5% < (2 - 1)). We examine the lower bound
for the growth of (w( (t) v,) in (2.4) and the upper bound for the growth of (w, Vz) in 2.5 in Theorem 2.2 If 38 (2a 1), we can employ the tensor power method and deduce that the spurious feature will be learned first and rapidly. The condition on data distribution imposes two necessary conditions: ã > 1/2 (groups are imbalanced) and 3. < 3, (the spurious feature is stronger). This observation is consistent with real-world datasets, such as the Waterbirds dataset, where ā = 0.95 and the background is much easier to learn than the intricate features of the birds. When the model learns core feature (3. 3.). However, if we deviate from the aforementioned conditions and consider 3. > Ba, we can examine the lower bound for the growth of (w? ve)
in (.5 and the upper bound for the growth of (w("). v.) in 2.4). Once again, apply the tensor power method and determine that the model will learn the core feature rapidly. In real-world datasets, this scenario corresponds to cases where the core feature not only significant but also easier to learn than the spurious feature. Even for imbalanced groups with â > 1/2, the model accurately learns the core feature. Consequently, enhancing the coefficients of the growth of the core feature allows the model to tolerate imbalanced groups. We present verification through synthetic experiments in the next section.
As we will show in the following subsection, we initially break the conditions of learning the spurious feature by letting â 1/2 in group- balance data subset. Subsequently, utilize the momentum to amplify the core feature's 's coefficient, allowing for tolerance of â 1/2 when adding new data.
3.2 PDE: A Two-Stage Training Algorithm
We present new algorithm named Progressive Data Expansion (PDE) in Algorithm Saaihimmaaa Tand explain the details below, which consist of (I) warm-up and (2) expansion stages.
Algorithm Progressive Data Expansion (PDE)
Require:
Number of iterations To for warm-up training, number of times for dataset expansion; number of iterations J for expansion training; number data m for each expansion; learning rate y7, momentum coefficient y; initialization scale do; training set S = {(x;, yi, a model fw.
1: Initialize ())))
Warm-up stage
2: Divide the S into groups by values of and a: Sy.a ((x, yı, a. h..))...........
3: Generate warm-up set 5 from S by randomly subsampling from each group of S such that |S
miny',al aa |Sy for y E {+1} and a E {+1}.
4: for t 0, 1,..., To do
5: Compute loss SS:: Lgo (W()) 1541
1 Eieso f W()).
6: Update W(++) by and 3.2)
7: end for
Expansion stage
8: for 1,..., do
9: Draw m examples (S(m.) from S/5k-1 and let Sh - Spm)
10: for t = 1,...,J J do
11:
Compute loss on S*: Ligh (T) 1st Eiesk (frffxx W(T)) where T To (k- 1) ++t
12: Update ((+++) by and 3.2
13: end for
14:
end for
15:
return W = -argmax Accig. (W())).
5

---

As accelerated gradient methods are most commonly used in applications, we jointly consider the property of momentum and our theoretical insights when designing the algorithm. For gradient descent with momentum (GD+M), at each iteration & and with momentum coefficient y > 0, it updates as follows
g ((+1) =g (1) (1- ( (()) 17- -gggttt))
(11-)) (V->))
(3.1)
(3.2)
Warm-up Stage. In this stage, we create a fully balanced dataset S in which each group is randomly subsampled to match the size of the smallest group, and consider it as warm-up dataset. We train the model on the warm-up dataset for a fixed number of epochs. During this phase, the model is anticipated to accurately learn the core feature without being influenced by the spurious feature. Note that, under our data model, a completely balanced dataset will have ā 1/2. We present the following lemma theoretical basis for the warm-up stage.
Lemma 3.1. Given the balanced training dataset 5 =
No with ā = 1/2 as in Definition 2.1 and CNN in (.33) The gradient on v, will be 0 from the beginning of training.
{(x,, Ve. a:)
In particular, with ã = 1/2 we have |S = |: an equal amount of data is positively correlated with the spurious feature the data negatively correlated with the spurious feature. In each update, both groups contribute nearly the same amount of spurious feature gradient with different signs, resulting in cancellation. Ultimately, this prevents the model from learning the spurious feature. Detailed proof' can be found in Appendix
Expansion Stage. In this stage, we proceed to train the model by incrementally incorporating new data into the training dataset. The rationale for this stage is grounded in the theoretical result by the previous work (Jelassi & Li 2022) on GD with momentum, which demonstrates that once gradient descent with momentum imitially increases its correlation with feature v, it retains substantial historical gradient in the momentum containing v. Put it briefly, the initial learning phase has a considerable influence on subsequent training for widely-used accelerated training algorithms. While ERM learns the spurious feature v, and momentum does not help, as we will show in synthetic experiments, PDE avoids learning v, and learns V. in the warm-up stage. This momentum from warm-up, in turn, amplifies the core feature that is present in the gradients of newly added data, facilitating the continued learning of ve in the expansion stage. For specific illustration, the learning of the core feature by GD+M will be
(w) (w(+1)) ve) = (w/") (19g + (1--7) (1 ((W()) ve),
where ((9) is the additional momentum as compared to GD with y = 0. While the current gradient along Ve might be small (i.e., Be), we benefit from the historical gradient in gi) to amplify the growth of (w (1+1) ve) and make it larger than that of the spurious feature (i.e., 32). This learning process will then correspond to the when the model learns the core feature discussed in Subsection 3.1 Practically, we consider randomly selecting m new examples for expansion every J epochs by attempting draw similar number of examples from each group. During the last few epochs of the expansion stage, we expect the newly incorporated data exclusively from the larger group, as the smaller groups have been entirely integrated into the warm-up dataset.
is worth noting that while many works address the issue of identifying groups from datasets containing spurious correlations, we assume the group information is known and our algorithm focuses on the crucial subsequent question of optimizing group information utilization. Aiming to prevent the learning of spurious features, PDE distinguishes itself by employing a rapid and lightweight warm-up stage and ensuring continuous improvement during the expansion stage with the momentum acquired from the warm-up dataset. Our training framework is both concise and effective, resulting in computational efficiency and ease of implementation.
4 Experiments
In this section, we present the experiment results from both synthetic and real datasets. Notably, report the worst-group accuracy, which assesses the minimum accuracy across all groups and is commonly used to evaluate the model's robustness against spurious correlations.
4.1 Synthetic Data

---

In this section, we present synthetic ex- periment results in verification our the- oretical findings. In Appendix A we illustrate the detailed data distribution, hyper-parameters of the experiments and more extensive experiment results. The data used in this seciton is generated following Definition 2.T We consider the worst-group and overall test accu- racy. As illustrated Table ERM, whether trained with GD or GD+M, is unable to accurately predict the small group in our specified data distribution where ā = 0.98 and Be < (3.. In con- trast, our method significantly improves worst-group accuracy while maintaining overall test accuracy comparable to ERM. Furthermore, depicted in Figure 3(a) ERM rapidly learns the spurious feature as it minimizes the training loss, while barely learning the core feature. Meanwhile, in Figure 3(b] we show the learning of ERM when the data distribution breaks the conditions of our theory and has B. > 3, instead. Even with the same a as in Figure (3a) ERM correctly learns the core feature despite the imbalanced group size. These two figures support the theoretical results we discussed to motivate our method. Consequently, on the same training dataset as Figure [3) Figure B(c))ssows that our approach allows the model to initially learn the core feature using the warm-up dataset and continue learning when incorporating new data.
Table 1: Synthetic data experiments. We report the worst- group accuracy and the gap (i.e., overall worst). We further consider several variations of PDE to demonstrate the importance of each component of our method. Reset: we reset the momentum zero after the warm-up stage. Warmup+Alll we let PDE to incorporate all of the new training data at once after the warm-up stage.
Worst-group (%)
Gap (%)
ERM (GD)
0.00
97.71
ERM (GI+M))
0.00
97.71
Warmup+All (Reset)
67.69
31.18
Warmup+All
74.24
24.76
PDE (Reset)
92.51
2.29
PDE
93.01
0.03
1.5
1.0
ragio Mel
Mal Jeyl
1.0-
15:
Espansion stage
10
training loss
0.01
0.0
1600
300
400
150
280
360
400
0.0
750 1000
(a) ERM (case 1)
(b) ERM (case 2)
(c) PDE
Figure 3: Training process of ERM vs. PDE. We consider the same dataset generated from the distribution as in Definition 2. for ERM (case 1) and PDE. On the same training data, ERM learns the spurious feature while PDE successfully learns the core feature. We further consider ERM (case 2) when training max inner product between the model's 's neuron and core/spurious signal vector and the decrease of training loss with regard to the number of iterations t.
the data distribution where B. > 3, and â 0.98. We show the growth of the
4.2 Real Data
We conduct experiments on real benchmark datasets to: (1) compare our approach with state-of- the-art methods, highlighting its superior performance and efficiency, and (2) offer insights into the design of our method through ablation studies.
Datasets. We evaluate on three wildly used datasets across vision and language tasks for spurious correlation: (1) Waterbirds (Sagawa al., 2019) contains bird images labeled waterbird or landbird, placed against a water or land background, where the smallest subgroup is waterbirds on land background. (2) CelebA (Liu et al. 2015) used to study gender the spurious feature for hair color classification, and the smallest group in this task is blond-haired males. (3) Commmnnsstss-- WILDS (Koh al.) 2021b) classifies toxic and non-toxie online comments while dealing with demographic information. It creates 16 overlapping groups for each of the 8 demographic identities.
Baselines. We compare proposed algorithm against several cate-of-the-art methods. Apart from standard ERM, we include GroupDRO (Sagawa et al. (019) and DFR (Kirichenko et al. 2023) that assume access to the group labels. We only include DFR "T for fair comparison all methods only use the training dataset to train and finetume the model, while DFR Val also finetunes on validation data. We also design a baseline called subsample that simply trains the model on the warm-up dataset only. Additionally, we evaluate three recent methods that address spurious correlations without the need for group labels: LfF (Nam al. 2020). EIII. (Creager al. 2021), and JTT (Liu et al. 2021). We
7

---

Table The worst-group and average accuracy (%) of PDE compared with state-of- thtt--f methods. The bold numbers indicate the best results among the methods that require group information, while the underscored numbers represent methods that only Irain once. All methods use validation data for early stopping and model selection, while VV indicates that the method also re-trains the last layer using the validation data
Group info
Train
Val info
Waterbirds
CelebA
CivilComments
Method
once
Worst
Average
Worst
Average
Worst
Average
ERM
X
70.0
V7.110.1
45.0115
94.810.2
58.2
92.210.1
LfF
X
X
78.0N/4
91.2N/A
77.2N/A
85.1
58.8
92.5x/a
EIIL
X
X
1..2
96.510.2
81.7108
85.7+0.1
67 67.0=2.4
90.520.2
JTT
X
86.7N/4
93.3N/4
81.1N/A
88.0N/4
69.3w/4
91.1x/A
Subsample
V
V
V
86.942.3
89.2 19.221.2
96.1
91.3+0.22
64.7
83.7 =3.4
DFR
V
90.2+0.8
97. +7.33
80.7+2.4
90.6
58 58.0g1.
92.0.0.1
DFR
92.9
94.210.4
88.311.1
91.310.3
70.1
87. 87.210.3
GroupDRO
86.710.6
93.2
86.311.1
92.910.3
69.4
89.6110.5
PDE
90.3
92.410.8
91.0
92.0g0.6
71.5
86.3
Table 3: Training efficiency of PDE and GroupDRO on Waterbirds. We compare with GroupDRO at their learning rate and weight decay, as well as at ours. We report the worst-group accuracy, average accuracy and the number of epochs till early stopping as the model reached the best performance on validation data. Note: for fair comparison, consider one training epoch as training over the N data as the size of the training dataset.
Method
Learning rate
Weight decay
Worst
Average
Etrly-stopping epoch*
GroupDRO GroupDRO
le-5
le-0
76.7
93.2
92=44
lc-2
le-2
77.3
77.
97.1+0.5
15415
PDE
le-2
le-2
40.3
92.4
8.9
report results for ERM. Subsample, GroupDRO and PDE based our own runs using the WILDS library Koh et al. (2021a); for others, we directly reuse their reported numbers.
We present the experiment details including dataset statistics and hyperparameters as well as compre- hensive additional experiments in Appendix B
4.2.1 Consistent Superior Worst-group Performance
We assess PDE the mentioned datasets with the state-of-the-an methods. Importantly, emphasize the comparison with GroupDRO, as it represents the best-performing method that utilizes group information. As shown in Table 2 PDE considerably enhances the porst-performing group's performance across all datasets, while maintaining the average accuracy comparable to GroupDRO. Remarkably, although GroupDRO occasionally fails to surpass other methods, PDE's performance consistently exceeds them in worst-group accuracy.
4.2.2 Efficient Training
In this subsection, we show that our method is more efficient as it does not train a model twice (as in JTT) and more importantly avoids the necessity for a small learning rate (as in GroupDRO). Specifically, methods employing group-balanced batches like GroupDRO require a very small learning rate coupled with large weight decay in practice. We provide intuitive explanation as follows. When sampling to achieve balanced groups in each batch, smaller groups appear more frequently than larger ones. If training progresses rapidly, the loss on smaller groups will be minimized quickly, while the majority of the large group data remains unseen and contributes to most of the gradients in later batches. Therefore, these methods necessitate slow training ensure the model encounters diverse data from larger groups before completely learning the smaller groups We validate this observation in T3]le where GroupDRO trained faster than the default results in significantly poorer performance similar to ERM. Conversely, PDE can be trained to converge rapidly on the warm-up set and reaches better worst-group accuracy 10x faster than GroupDRO at default. Note that methods which only finetune the last layer (Kirichenko et al. 2023 Wei et al. 2023) are also efficient. However, they still require training model first using ERM on the entire training data till convergence. In contrast, PDE does not require to further finetune the model.
8

---

4.2.3 Understanding the Two Stages
We examine each component and demonstrate their effect in PDE. In Table 4 we present the worst-group and average accuracy of the model trained following the warm-up and expansion stages. Indeed, the majority of learning occurs in the warm-up stage, during which a satisfactory worst-group accuracy is established. In the pansion stage, the model persists learning new data along the established trajectory, leading continued performance improvement. In Figure(5]
Table Performance of PDE after each stage.
We report the worst-group and average accuracy.
Warm-up
Addition
Dataset
Worst
Avg
Worst
Avg
Waterbirds CelebA
86.0
91.9
90.3
92.4
87.8
92.1
91.0
92.0
CivilComm
67.7
78.8
71.5
86.3
corroborate and emphasize that the model has acquired the appropriate features and maintains learning based on its historical gradient stored in the momentum by presenting the following evidence. As shown, if the optimizer reset after the warm-up stage and loses all its historical gradients, rapidly acquires spurious features, resulting in swift decline in performance accuracy as shown in the blue line.
1.00
0.93
0.95
0.94
0.00
0.855
Aune
0.850
0.825
0.800
0.85
(a) Worst-group accuracy.
(b) Average accuracy.
Figure 4: The effect of reset the momentum to zero after the warm-up stage for PDE on Waterbirds.
4.2.4 Ablation Study on the Hyper-parameters of PDE
PDE is robust within reasonable range of hyperparemeter choices, although some configurations outperform others. As shown Table is necessary to limit the number of data points introduced during each expansion to prevent performance degradation. Similarly, in Appendix A emphasize the importance gradual data expansion. In T we show that wast-w learning rate decay essential, though PDE exhibits tolerance to the degree of this decay. Lastly, illustrated in Figure 5 adopting smaller learning rate often necessitates increased data expansions. Nonetheless, reduced learning does necessarily lead improved performance.
1.0
3.00
acourary
Auarage accuracy
0.9
0.55
0.50
Expanston
Papansion
140
160
150
200
220
240
260
280
300
140
160
180
200
220
240
260
280
300
Epoch
Epoch
Worst-group accuracy.
(b) Average accuracy.
Figure 5: The variations in both worst-group and average accuracy the test set of Waterbirds during expansion stage under different expansion learning rates. Each vertical dashed line denotes an expansion and the arrow denotes the early stopping.
Table 5: Ablation study on Waterbirds. Exp. size: number data points added in each expansion.
Table 6: Ablation study on Waterbirds. Exp. Ir: the learning rate in expansion stage.
Exp. size
Exp. Ir
Worst
Average
Exp. size
Exp.
Worst
Average
5
le-4
89.940.5
92.1 92.1+0.3
le-2
85.44a.1
92.1
10
le-4
90.3
92.4
10
le-3
89.440.7
92.640.3
50
le-4
88.140.
93.4
10
le-5
89.540.2
92.1

---

5 Related Work
Existing approaches for improving robustness against spurious correlations can be categorized into two lines of research based the tackled subproblems. A line of research focuses on the same subproblem we tackle: effectively using the group information to improve robustness. With group information, one can use the distributionally robust optimization (DRO) framework and dynamically
increase the weight of the worst- group loss in minimization (Hu et al. 2018 Oren et al. Sagawa et al.) 2019 Zhang al.] 2021). Within this line of work, Group (Sagawa al.
2019
2019
achieves state-of th- performances across multiple benchmarks. Other approaches use importance
weighting to reweight the groups (Shimodaira 2000, (yrd & Lipton] 2019 Xu et al.
2021) and
class balancing to downsample the majority or upsample the minority (He & Garcia 2009) Cui et al.
2019; Sagawa al. 2020p. Alternatively, Goel et al. (202T) leverage group information augment
the minority groups with synthetic examples generated using GAN. Another strategy (Cao et al.
2019 2020) involves imposing Lipschitz regularization around minority data points. Most recently, methods that train a model using ERM first and then only finetune the last layer on balanced data from training or validation (Kirichenko et al. 2023) or learn post-doc scaling adjustments (Wei et al. 2023 are shown to be effective.
The other line of research focuses the setting where group information is not available during training and tackles the first subproblem we identified as accurately finding the groups. Recent
notable works (Nam et al. 2020 Liu et al. [02T Creager al. 2021 Zhang et al. 2022 Yang et al.)
2023a) mostly involve training two models, one of which is used to find group information.
To finally
use the found groups, many approaches (Namkoong & Duchi 2017 Duchi al.
2019
Oren et al.)
2019 Sohoni et al. 2020) still follow the DRO framework.
The first theoretical analysis of spurious correlation is provided by Sagawa et al. 2020). For self- supervised learning, Chen et al. (2020) shows that tne-tuning with pre-trained models can reduce the harmful effects of spurious features. Ye et al. (2022) provides guarantees in the presence of label noise that core features are learned well only when less noisy than spurious features. These theoretical work only provides analyses of linear models. Meanwhile, parallel line of work has established theoretical analysis of nonlinear CNNs in the more realistic setting Allen-Zhu & Li (2020) : Zou al. (202T): Wen & Li (202T): Chen al. (2022); Delassi & Li (022). Our work builds on this line of research and generalizes it to the study of spurious features. Lastly, we notice that concurrent work (Chen et al., 2023) also uses tensor power method (Allen-Zhu & Li (020) to
analyze the learning of spurious features v.s. invariant features, but in the setting of out-of-distribution generalization.
Conclusion
In conclusion, this paper addressed the challenge of spurious correlations in training deep learning models and focused on the most effective use of group information to improve robustness. We provided theoretical analysis based on a simplified data model and two-layer nonlinear CNN. Building upon this understanding, we proposed PDE, a novel training algorithm that effectively and efficiently enhances model robustness against spurious correlations. This work contributes both the theoretical understanding and practical application of mitigating spurious correlations, paving the way for more reliable and robust deep learning models.
Limitations and future work. Although beyond the linear setting, our analysis still focuses on relatively simplified binary classification data model. To better represent real-world application scenarios, future work could involve extending to multi-class classification problems and examining the training of transformer architectures. Practically, our proposed method requires the tuning of additional hyperparameters, including the number of warm-up epochs, the number of times for dataset expansion and the number of data to be added in each expansion.
Acknowledgements
We sincerely thank Dongruo Zhou for the constructive suggestions the structure and writings of the paper. We also thank the anonymous reviewers for their helpful comments. YD and QG are supported in part by the National Science Foundation CAREER Award 1906169 and IIS-2008981, and the Sloan Research Fellowship. BM is supported by the National Science Foundation CAREER Award 2 2146492. The views and conclusions contained in this paper are those of the authors and should not be interpreted as representing any funding agencies.
10

---

References
Ahmed, F., Bengio, Y., Van Seijen, H., and Courville, A. Systematic generalisation with group
invariant predictions. In International Conference Learning Representations, 2021.
Allen- Z. and Li, Y. Towards understanding ensemble, knowledge distillation and self-distillatio in deep learning. arXiv preprint arXiv:20/2.098/6, 2020.
Byrd, J. and Lipton, Z. What is the effect of importance weighting in deep learning? In International conference on machine learning, pp. 872-881. PMLR, 2019.
Cao, K., Wei, C., Gaidon, A., Arechiga, N., and Ma, T. Learning imbalanced datasets with label- distribution-and aware margin loss. Advances in neural information processing systems, 32, 2019.
Cao, K., Chen, Y., Lu, J., Arechiga, N., Gaidon, A., and Ma. T. Heteroskedastic and imbalanced deep learning with adaptive regularization. arXiv preprint arXiv:2006. 117666006.776, 2020.
Cao, Y., Chen, Z., Belkin, M., and Gu, Q. Benign overfitting in two-layer convolutional neural networks. Advances in neural information processing systems, 35:25237-25250 2022.
Chen, Y., Wei, C., Kumar, A., and Ma. T. Self-training avoids using spurious features under domain shift. Advances in Neural Information Processing Systems, 33:21061-21071, 2020.
Chen, Y., Huang, W., Zhou, K., Bian, Y., Han, B., and Cheng, J. Towards understanding feature learning in duitrf-diitriuution generalization. arXiv preprint 2rXiv:2304.1 2023.
Chen, Z., Deng, Y., Wu, Y., Gu, Q., and Li, Y. Towards understanding the mixture-of-experts layer in deep learning. Advances in neural information processing systems, 2022.
Creager, E.,. Jacobsen, J.-H., and Zemel, R. Environment inference for invariant learning. In International Conference on Machine Learning, pp. 2189-2200. PMLR, 2021.
Cui, Y., Jia, M., Lin, T.-Y., Song, Y., and Belongie, S. Class-balanced loss based on effective number of samples. In Proceedings of the IEEE/CVF conference computer vision and patlern recognition, pp. 9268-9262 9277, 2019.
Duchi, J. C., Hashimoto, T., and Namkoong, H. Distributionally robust losses against mixture covariate shifts. Under review, 2:1, 2019.
Goel, K., Gu, A., Li, Y., and Re, C. Model patching: Closing the subgroup performance gap with data augmentation. In International Conference on Learning Representations, 2021.
Haghtalab, N., Jordan, M., and Zhao, E. On-demand sampling: Learning optimally from multiple distributions. Oh, A. H., Agarwal, A., Belgrave, D., and Cho, K. (eds.), Advances in Neural Information Processing Systems, 2022.
He, H. and Garcia, E. A. Learning from imbalanced data. IEEE Transactions on knowledge and data engineering, 1299):1263-1284 2009.
He, K., Zhang, X., Ren, S., and Sun, J. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and puttern recognition, pp. 770-778, 2016.
Hu, W., Niu, G., Sato, I., and Sugiyama, M. Does distributionally robust supervised learning give robust classifiers? In International Conference Machine Learning, pp. 2037. PMLR, 2018.
Izmailov, P., Kirichenko, P., Gruver, N., and Wilson, A. G. On feature learning in the presence of
spurious correlations. Advances in Neural Information Processing Systems, 35:38516-38532 2022.
Jelassi, S. and Li, Y. Towards understanding how momentum improves generalization in deep learning. International Conference on Machine Learning, pp. 9965-10040. PMLR, 2022.
Kirichenko, P., Izmailov, P., and Wilson, Last layer re-training is sufficient for robustness to spurious conrelations. In International Conference on Learning Representations, 2023.

---

Koh, P. W., Sagawa, S., Marklund, H., Xie, S. M., Zhang, M.. Balsubramani, A., Hu, W., Yasunaga, M., Phillips, L., Gao, I., Lee, T., David, E.. Stavness, I.. Guo. W., Earnshaw, B. A., Haque, S., Beery, S., Leskovec, J., Kundaje, A., Pierson, E., Levine, S., Finn, C., and Liang. P. WILDS: A benchmark of in-the-wild distribution shifts. In International Conference on Machine Learning (ICML), 2021a.
Koh, P. W., Sagawa, S., Marklund, H., Xie, S. M., Zhang, M., Balsubramani, A., Hu, W., Yasunaga, M., Phillips, R. L.,, Gao, I., et al. Wilds: A benchmark of in-the-wild distribution shifts. In International Conference Machine Learning, pp- 5637-5664, PMLR, 2021b.
Liu, E. Z., Haghgoo, B., Chen, A. S., Raghunathan, A., Koh, P. W. Sagawa, S., Liang, P., and Finn, C. Just train twice: Improving group robustness without training group information. International Conference Machine Learning, pp. 6781-6792. PMLR, 2021.
Liu, Z., Luo, P., Wang. X., and Tang, X. Deep learning face attributes in the wild. In Proceedings of the IEEE international conference on computer vision, pp. 3730-3738, 2015.
Nam, J., Cha, H., Ahn, S., Lee, J., and Shin, J. Learning from failure: De-biasing classifier from biased classifier. Advances in Neural Information Processing Systems, 33:20673-2066 2020.
Namkoong, H. and Duchi, J. C. Variance-basse regularization with convex objectives. Advances in neural information processing systems, 30, 2017.
Oren, Y., Sagawa, S., Hashimoto, T. B., and Liang. P. Distributionally robust language modeling. In Proceedings of the 2019 Conference Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pp- 4227 4237, Hong Kong, China, November 2019. Association for Computational Linguisties. doi: 10.18653/v1/D19-144
Paszke, A., Gross, S., Massa, F., Lerer, A., Bradbury, J., Chanan, G., Killeen, T., Lin, Z., Gimelshein, N., Antiga, L., Desmaison, A., Kopf, A., Yang. E., Vito, Z., Raison, M., Tejani, A., Chilamkurthy, S., Steiner, B., Fang, L., Bai, J., and Chintala, S. Pytorch: An imperative style, high-performance deep learning library. In Wallach, H., Larochelle, H., Beygelzimer, A., d' d'Alché-Bus Buch F., Fox, E., and Garnett, R. (eds.), Advances in Neural Information Processing Systems 32, pp. 8024-8035. 2019.
Sagawa, S., Koh, P. W., Hashimoto, T. B., and Liang, P. Distributionally robust neural networks. In International Conference on Learning Representations, 2019.
Sagawa, S., Raghunathan, A., Koh, P. W., and Liang, P. An investigation of why overparameterization exacerbates spurious correlations. In International Conference on Machine Learning, pp. 8346- 8356. PMLR, 2020.
Selvaraju, R. R., Cogswell, M.. Das, A., Vedantam, R., Parikh, D., and Batra, D. Grad-cam: Visual explanations from deep networks via gradient-based localization. In Proceedings of the IEEE international conference on computer vision, pp. 618-626, 2017.
Shimodaira, H. Improving predictive inference under covariate shift by weighting the log-likelihood function. Journal of statistical planning and inference. 90(2):227-244. 2000.
Sohoni, N., Dunnmon, J., Angus, G., Gu, A., and Ré, C. No subclass left behind: Fine- Fine-grained
robustness in coarse-grained classification problems. Advances in Neural Information Processing Systems, 33: 19939-193522 2020.
Taghanaki, S. A., Choi, K., Khasahmadi, A. H., and Goyal, Robust representation learning via perceptual similarity metrics. In International Conference on Machine Learning, pp. 10043-10053. PMLR, 2021.
Van der Maaten, L. and Hinton, G. Visualizing data using t-sne. Journal of machine learning research, 9(11), 2008
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L.., Gomez, A. N., Kaiser, E.,, and Polosukhin, Attention is all you need. Advances in neural information processing systems, pp- -908-6008, 2017.
12

---

Wah, C., Branson, S., Welinder, P., Perona, P., and Belongie, S. The caltech-ucsd firds-200-201 dataset. 2011.
Wei, J., Narasimhan, H., Amid, E., Chu, W.-s,, Liu, Y., and Kumar, A. Distributionally robust post-hoc classifiers under prior shifts. In International Conference Leurning Represenlations (ICLR), 2023.
Wen, Z. and Li, Y. Toward understanding the feature learning process of self-supervised contrastive learning. In International Conference Machine Learning, pp. 11112-11111 11122. PMLR, 2021.
Wolf, T., Debut, L., Sanh, V., Chaumond, J., Delangue, C., Moi, A., Cistac, P., Rault, T., Louf, R., Funtowicz, M., Davison, J., Shleifer, S., von Platen, P., Ma, C.. Jernite, Y., Plu, J., Xu, C., Scao, T. L., Gugger, S., Drame, M., Lhoest, Q., and Rush, A. M. Transformers: State-of-the- art natural language processing. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pp. 38 45, Online, October 2020. Association for Computational Linguistics. URLhttps: www. aclweb. org/ hetg/anthologyy222
emn emnlp-demos.6
Xu, D., Ye, Y., and Ruan, C. Understanding the role of importance weighting for deep learning. In International Conference Learning Reprexentations, 2021.
Yang, Y. Gan, E., Dziugaite, G. K., and Mirzasoleiman, B. Identifying spurious biases early in training through the lens of simplicity bias. arXiv preprint arXiv:2305. 18761, 2023a.
Yang, Y., Nushi, B., Palangi, H., and Mirzasoleiman, B. Mitigating spurious correlations in multi- modal models during fine-tuning. In International Conference on Machine Learning. 2023b.
Yang, Y., Zhang, H., Katabi, D., and Ghassemi, M. Change is hard: A closer look at subpopulation shift. arXiv preprint arXiv:2302. /2254, 2023c.
Yang, Y.Y.,, Chou, C.-N., and Chaudhuri, K. Understanding rare spurious correlations in neural networks. arXiv preprint arXiv:2202.05189 2022.
Ye, H., Zou, J., and Zhang, L. Freeze then train: Towards provable representation learning under spurious correlations and feature noise. arXiv preprint arXiv:2210. 1075, 2022.
Zhang. J., Menon, A. K., Veit, A., Bhojanapalli, S., Kumar, S., and Sra, S. Coping with label shift via distributionally robust optimisation. In International Conference on Learning Representations, 2021.
Zhang, M., Sohoni, N. S., Zhang, H. R., Finn, C., and Ré, C. Correct-n-comtrast: A contrastive approach for improving robustness to spurious correlations. arXiv preprint arX1v:2203.05517, 2022.
Zhou, B., Lapedriza, A., Khosla, A., Oliva, A., and Torralba, A. Places: A 10 million image database for scene recognition. IEEE Transactions Pattern Analysis and Machine Intelligence, 2017.
Zou, D., Cao, Y., Li, Y., and Gu, Q Understanding the generalization of adam in learning neural networks with proper regularization. arXiv preprint arXiv:2108. 11371, 2021.
13


Reviews:
[{"review_id": "Ml7XOvExuO", "reviewer": "Reviewer_vLux", "summary": "The authors propose a new method called PDE to address spurious features and improve generalization. Building on the existing literature, the authors consider an imbalanced binary classification problem where two types of features coexist: core features and spurious features. The core features can lead to good generalization performance but a classification model may pick up spurious features. To avoid that, the proposed method (1) starts with a downsampled balanced classification as a warm-up, which will not pick up the spurious features, and (2) then gradually increase the imbalance ratio until the full dataset is used for training.\n\nThe main justification for this two-stage algorithm is that gradient descent with momentum can benefit from the warm-up stage where no spurious features are picked up. Numerous experiments on synthetic data and real data are used to support the proposed method.", "questions": "See the weaknesses section.", "limitations": "NA", "rating": 4, "confidence": 5, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "Classification with spurious features is a very practical and relevant problem. The proposed method is easy to implement and new to my knowledge. The experiments result show promising performance compared with alternatives.\n\nOverall, the warm-up stage in optimization seems to be an interesting idea. It starts with an \"easy\" downsampled dataset without data imbalance---which is known to be the culprit of picking up spurious features. Then, in the second stage, we hope to use the full datasets without picking up spurious features by using momentum.\n\nThe presentation of this paper is clear and theoretical/experimental results are easy to follow.", "weaknesses": "In my opinion, this paper misses a crucial element in classification with spurious features---that is, the effect of overparametrization. It is known, for example in [1], that there is a very important difference between underparameterization and overparametrization when spurious features are present. It is, therefore, expected to have analyses on the inductive bias of the proposed method and how it may select a superior solution compared with ERM and recent alternatives. Unfortunately, the effect of overparametrization is absent in the analyses, as the number of neurons $J$ does not play a role in the analysis.\n\nBesides, while the two-stage optimization idea looks interesting, the analysis is a bit handwaving, so it is unconvincing to me why momentum method is able to keep avoiding spurious features in the second stage. An investigation of inductive bias would make this paper much stronger; perhaps even some in-depth experiments on synthetic datasets would shed lights on the behavior of momentum in terms on avoiding spurious features.\n\nLastly, I think the experimental results look fine, but I am not convinced that the proposed method is better than the recent alternatives. For example, in Table 2, while PDE achieves better accuracy on worst classes, it usually incurs a worse accuracy on average. Since there is a clear tradeoff between majority and minority classes in binary classification, better accuracy for the minority class can be often trivially improved by shifting the decision boundary. It would be more convincing if empirical results show improvements for *all* classes.\n\n\n[1] Sagawa, S., Raghunathan, A., Koh, P. W., and Liang, P. An investigation of why overparameterization exacerbates spurious correlations. In International Conference on Machine Learning, pp. 8346– 8356. PMLR, 2020.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for the feedback. We believe that there are some misunderstandings of the contribution of our paper. In light of our clarifications below, we hope that the reviewer considers increasing their evaluation of our work.\n\n---\n\n**Q1**: This paper misses the effect of overparameterization. \n\n**A1**: We believe there is a misunderstanding of our results. We'd like to clarify that the CNN considered in our analysis is indeed overparameterized. Specifically, \"overparameterization\" refers to settings where a model's parameters exceed the data's dimension. In our study, the parameters in our CNNs amount to $J \\times d$ which is indeed larger than the data dimension. As similar to previous work [2], the order of $J$ is assumed to be $J=\\text{polylog}(d)$ and the model has a mild overparameterization. We will emphasize this point more explicitly in the revision.\n\nThe fundamental work [1] studies the effect of spurious correlations on underparametrized model compared to overparameterized model, and confirmed that overparametrized models are specifically negatively affected by spurious correlations. Hence, we do not investigate the parameterization effect again but focus specifically on the overparameterized regime. **Based on their initial findings, we aim to further understand what properties of data cause the learning of spurious features and consequently how to avoid it for overparameterized models.**  The major focus of PDE is to improve the robustness of current deep learning models against spurious correlations. Hence, we do not investigate the underparameterized models that are less used in practice and proven not affected. We want to highlight that, not only in our analysis, but also all models considered in our synthetic and real experiments are overparameterized. Lastly, as many related theory papers explore different aspects of analysis under the overparameterized setting, we especially focus on the non-linear setting and consider convolutional architectures for the first time.\n\n---\n\n**Q2**:  It is unconvincing to me why momentum method is able to keep avoiding spurious features in the second stage. \n\n**A2**: To show the importance of momentum, we have dedicated line 210-228 for explanation and real data experiments in Figure 4. Briefly, the role of momentum that preserves historical trend of learning is proven in the previous work [2]. We use this theoretical finding to motivate our method. As discussed, the model learns the core feature in the warmup stage. During expansion, the momentum from warmup amplifies the core feature that is also present in the gradients of newly added data. This learning process will then correspond to the second case when model learns core feature discussed in subsection 3.1, and the model can tolerate the imbalanced expansion data and continues learning of core feature. \n\nFurthermore in Figure 4, our findings suggest that re-initializing the optimizer, and consequently its historical gradient after the warmup stage, will harm the model performance compared to preserving momentum from the warmup stage. Moreover, we provide more ablation studies on synthetic data in Table 1 in the 1-page PDF. As shown, keeping the momentum from the warmup stage provides effective improvement to the worst-group accuracy of the model.\n\n---\n\n**Q3**: Not convinced that PDE is better than the recent alternatives. It would be more convincing if empirical results show improvements for all classes.\n\n**A3**: We clarify that **worst-group accuracy is the main objective for studies in spurious correlation, and has been regarded as the main evaluation criteria in prior work**. As evident in prior works [3-6], the focus has consistently been on enhancing worst-group accuracy. A high average accuracy can often be misleading, as it might suggest a dependable predictor while the model relies on spurious correlations. Consequently, the predictor performs remarkably well on larger groups but fail when the spurious correlation is not present or indicate the contrary.\n\nWe also believe there is a misunderstanding of the evaluation metric. Worst-group accuracy is not computed with regard to the **classes** and a simple \"shift of the decision boundary\" is not practical. Instead, the **groups** represent varying combinations of spurious signals (backgrounds) and true labels (birds). Small groups are present within all classes, such as waterbirds with land background and landbirds with water background. **They are not a single under-performing class.** Hence, class accuracy is similar to average accuracy and doesn't indicate reliability in this context.\n\nWe are happy to include an additional evaluation metric here. While a high average accuracy does not indicate a reliable predictor but can in fact be alarming, we include here another evaluation metric as per prior literature [7]. This metric measures the gap between the worst-group and average accuracy, where a smaller gap indicates a more robust model that's potentially less reliant on the spurious correlation. The results are presented in Table 3 in the PDF.\n\n---\n\n[1] Sagawa et al. \"An investigation of why overparameterization exacerbates spurious correlations.\" \n\n[2] Jelassi et al. \"Towards understanding how momentum improves generalization in deep learning.\" \n\n[3] Liu et al. \"Just train twice: Improving group robustness without training group information.\" \n\n[4] Creager et all. \"Environment inference for invariant learning.\" \n\n[5] Zhang et al. \"Correct-N-Contrast: a Contrastive Approach for Improving Robustness to Spurious Correlations.\" \n\n[6] Kirichenko et al. \"Last Layer Re-Training is Sufficient for Robustness to Spurious Correlations.\" \n\n[7] Haghtalab et al. \"On-demand sampling: Learning optimally from multiple distributions.\" \n\nWe hope that you can re-evaluate our work based on the clarifications above. If any concerns have remained unaddressed, we appreciate it if you let us know so that we can provide further details."}, {"review_id": "SvqWCD8VAt", "reviewer": "Reviewer_6r1Q", "summary": "The paper works on understanding and mitigating the impacts on spurious features. Specifically, the authors provide theoretical analysis on the learning process of a non-linear two-layer CNN, under spurious features. Theoretical insights reveal the need to start with balanced data, and progressively expand the train set. The proposed PDE is right based on such insights, and is demonstrated to be efficient and effective on certain datasets.", "questions": "* (1) The results on the synthetic data is a bit confusing: in table 1, the performance of ERM on worst-group is 0%; besides, PDE has a better worst-group performance than the overall one, which is kind of conterintuitive.\n\n* (2) It would be much better if authors could add more discussions on the role and selection/suggestions of certain parameters, i.e., the number of times for dataset expansion and the number of data to be added in each expansion.", "limitations": "The authors clearly stated their limitations at the end of paper, \n\n* The theoretical analysis rely on simplified model;\n\n* Certain hyper-parameters might be crucial in model training: i.e., the number of warmup epochs, the number of times for dataset expansion and the number of data to be added in each expansion.", "rating": 8, "confidence": 3, "soundness": 4, "presentation": 3, "contribution": 4, "strengths": "* (1) The authors provide theoretical insights on the learning process of a two-layer non-linear CNN under spurious features, which aligns with empirical observations.\n\n* (2) The proposed method starts with group-balanced training set and then gradually injects more data samples to capture the core features. And its effectiveness and efficiency are demonstrated on several popular datasets when compared with exiting methods.\n\n* (3) The paper is clearly presented and well organized, and insights/motivations are easy to follow.\n\n", "weaknesses": "Although the proposed PDE is super efficient by referring to existing learning/training based approaches, it would be better to mention/discuss the efficiency of some fine-tuning or post-hoc approaches during the related work section, for example, fine-tune the last layer is sufficient to spurious features [R1], post-hoc adjustment on model prediction improves robustness to spurious features [R2].\n\nReferences:\n\nR1: Last Layer Re-Training is Sufficient for Robustness to Spurious Correlations. [ICLR'23]\n\nR2: Distributionally Robust Post-hoc Classifiers under Prior Shifts. [ICLR'23]", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We're grateful for your strong support and suggestions on our work, for which we have accordingly made modifications to our manuscript. Please find our detailed response below for the questions raised in the review. \n\n---\n\n**Q1**: It would be better to mention/discuss the efficiency of some fine-tuning or post-hoc approaches during the related work section.\n\n**A1**: We thank the reviewer for pointing out the post-hoc approaches and we will add a discussion in the related work section in our revision. Briefly, we believe that fine-tuning approaches [R1, R2] that re-trains the last layer are also efficient methods that can effectively mitigate the influence of spurious correlations. Nevertheless, these methods still require training a model first using ERM in the first stage and finetune the last layer in the second stage. For example, on the Waterbirds dataset, the model is first trained for 300 epochs on the entire training dataset using ERM and further finetuned in the second stage. As we train PDE with no more than 9 epochs as compared to ERM, we believe PDE can be more efficient since PDE also does not require to further finetune the model. Nevertheless, we believe that for [R1, R2], finetuning the last layer only does not incur significant additional computation to ERM.\n\n---\n\n**Q2**: The results on the synthetic data is a bit confusing.\n\n**A2**: Thanks for pointing out the inconsistency of PDE’s result in Table 1. We apologize for the oversight in reporting the incorrect worst-group results; the value we presented was actually for the small group accuracy. This has now been corrected, and the table has been updated accordingly in Table 1 of the attached PDF.\n\nThe 0% accuracy of ERM is accurate. Our synthetic data was specifically designed so that a model only relying on spurious features will completely fail the small-group test data. If the model leans heavily on the spurious feature, its predictions will align with the spurious features that are the opposite to the true labels in the small groups, resulting in worse accuracy than random guessing. Admittedly, the synthetic setting is a simpler and more extreme version of the real data. In real-world datasets, ERM does achieve 45.0% worst-group accuracy on CelebA and 58.2% on CivilComments.\n\n---\n\n**Q3**. Add more discussions on the role and selection/suggestions of certain parameters.\n\n**A3**. We appreciate the suggestion and added ablation studies on Waterbirds dataset as in Table 4 and Figure 1 of the attached PDF. Namely, our methods are robust within a reasonable range of hyperparameter selections. Meanwhile, there are preferred choices for the hyperparameters.\n- The number of data added during each expansion cannot be too large as it will harm the performance. As similarly demonstrated in Figure 6 of Appendix A, gradual expansion is important to our method and adding all data at once will result in a worse performance. \n- There is a necessity to decay the learning rate after the warmup stage, while PDE is relatively robust to the extent of decay. \n- The number of times for data expansion depends on the early-stopping on the validation data. As in Figure 1 in the PDF, a smaller learning rate will result in more data expansions. \n\nWe will add the ablation studies and corresponding discussion to the revised version of our work.\n"}, {"review_id": "UL1ttdh0VQ", "reviewer": "Reviewer_YdZz", "summary": "In this paper, the author propose a learning framework for improving the performance of classifiers on the worst group in the presence of spurious features. The authors focus on a two-layer convolutional neural network. They first show that imbalanced data and easy-to-learn spurious features can lead to bias in the classifier. Then, they propose a learning procedure that focuses on balanced data and gradually progresses how the model learns from core features to all features. The new method is evaluated on several datasets demonstrating that the new approach is fast and can improve accuracy on worst groups. ", "questions": "How is the method applied to non-binary data? \nThe analysis focuses on the activation z^3. Can this be replaced with more commonly used activations?\nWhy is there a significant drop in average accuracy in some examples?\nSome parts are not well explained; for example, the caption in Figure 1 is unclear. \n", "limitations": "Limitations are discussed in the last section of the paper. Addressing some of the limitations presented by the authors is indeed important to increase the usability of the proposed approach.", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "The paper is mostly well-written and easy to follow, and the idea makes sense. The proposed approach is motivated theoretically using a two-layer CNN model. The method is simple and relies on a two-stage training procedure, therefore can be easily implemented and adapted to other algorithms. \nThe proposed method improves the performance of the worst group in all evaluated examples; in some cases, it leads to significant improvements. \n", "weaknesses": "The theoretical analysis in the paper is focused on binary classification. The method requires tuning several hyperparameters. While the worst group performance is improved in most cases the overall accuracy degradation is non negligible. The method is only evaluated on a small number of datasets. ", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We appreciate the feedback and questions raised on the potential confusions. Please find our detailed response below, for which we hope that the reviewer considers increasing their evaluation of our work.\n\n---\n\n**Q1**: The theoretical analysis in the paper is focused on binary classification. The method requires tuning several hyperparameters.\n\n**A1**: We have indeed recognized the aforementioned points in our limitation section and believe they offer valuable directions for future research. We would like to further emphasize that the previous papers [1-4] also focused on the binary classification problem. To the best of our knowledge, no existing study has studied multi-class classification problems in the theoretical analysis of spurious correlations. Meanwhile, we consider the more complex case of a non-linear CNN, addressing data distributions that conventional linear models find challenging.\n\nWe also want to kindly remind the NeurIPS guidelines on the limitation discussion that “authors should be rewarded rather than punished for being up front about the limitations of their work.”\n\n[1] Sagawa et al. \"An investigation of why overparameterization exacerbates spurious correlations.\"\n\n[2] Chen et al. \"Self-training avoids using spurious features under domain shift.\"\n\n[3] Yang et al. \"Understanding rare spurious correlations in neural networks.\" \n\n[4] Ye et al. \"Freeze then train: Towards provable representation learning under spurious correlations and feature noise.\"\n\n---\n\n**Q2**: While the worst group performance is improved in most cases the overall accuracy degradation is non negligible. Why is there a significant drop in average accuracy in some examples?\n\n**A2**: Thanks for raising the question. We wish to clarify that the research on spurious correlations primarily aims to develop a **robust and reliable predictor**, and therefore **worst-group accuracy is regarded as the main evaluation metric** rather than striving for an even higher average accuracy than ERM. This can be found in previous studies [1-5], which highlighted the importance of improving worst-group accuracy and similarly exhibited such “trade-off”. The state-of-the-art methods like GroupDRO and DFR all improves worst-group accuracy while dropping the average accuracy. \n\nFurthermore, the decrease in the average accuracy is not necessarily considered as a drawback in the context of spurious correlations. The presence of spurious correlations can mislead us into perceiving a high average accuracy as an indicator of a reliable predictor. However, a model may perform exceptionally well on larger groups, yet fail smaller ones. In contrast, the gap between average accuracy and worst-group accuracy, as outlined in [6], can help identify if a model is overly influenced by spurious features. For a more straightforward comparison, we updated a table comparing the gaps for all models in the 1-page pdf in Table 3. The objective of creating a more dependable model is therefore to minimize this gap while enhancing worst-group accuracy. \n\n---\n\n**Q3**: The method is only evaluated on a small number of datasets.\n\n**A3**: We have included the most common benchmark datasets used for spurious correlations as in previous works [1-6]. Admittedly, the benchmark datasets for spurious correlation are limited, where Waterbirds and CelebA have been the most used data. We believe the development of more benchmark datasets specifically designed to evaluate models' performance on spurious correlations is an important future work. However, within the scope of our current study, we have endeavored to provide a comprehensive evaluation using the most prevalent datasets available.\n\n[1] Sagawa et al. \"Distributionally Robust Neural Networks.\" \n\n[2] Liu et al. \"Just train twice: Improving group robustness without training group information.\" \n\n[3] Creager et al. \"Environment inference for invariant learning.\"\n\n[4] Zhang et al. \"Correct-N-Contrast: a Contrastive Approach for Improving Robustness to Spurious Correlations.\" \n\n[5] Kirichenko et al. \"Last Layer Re-Training is Sufficient for Robustness to Spurious Correlations.\" \n\n[6] Haghtalab at al. \"On-demand sampling: Learning optimally from multiple distributions.\" \n\n---\n\n**Q4**: The analysis focuses on the activation z^3. Can this be replaced with more commonly used activations?\n\n**A4**: Yes, it indeed can be replaced with the activations such as ReLU, as demonstrated by other works applying a similar analysis but for different problems [1]. The cubic activation has been widely used in the other theoretical works [2] for the simplicity of analysis and results that align with experiments. The inclusion of ReLU activation, while feasible, will make our analysis more intricate, especially in terms of proof details. Our primary goal in this study is to maintain simplicity and ensure our theoretical motivation is easy to follow, thereby providing clear motivation for our proposed method. Lastly, our experiments also confirmed that our results hold in practice for ReLU activations.\n\n[1] Kou, et al. \"Benign overfitting for two-layer relu networks.\" \n\n[2] Jelassi et al. \"Towards understanding how momentum improves generalization in deep learning.\" \n\n---\n\nWe hope that you can revisit your assessment of our work in light of the clarifications provided above. Should there be any other concerns, we are happy to provide further information."}, {"review_id": "sazAovJgEx", "reviewer": "Reviewer_uURe", "summary": "The paper describes Progressive Data Expansion (PDE), a new method for training models which are robust to spurious features. The idea of the method is to split the training into two phases: warmup and expansion. In the warmup stage, the dataset is balanced, and the model learns a classifier that ignores the spurious feature. In the expansion stage, the remaining data is used to finetune. The authors also provide a theoretical argument about feature learning, and show strong empirical performance on benchmark datasets.\n\n**W** &mdash; weakness,  **S** &mdash; strength, **Q** &mdash; question. ", "questions": "**Q3. Expansion phase:** In line 225 you mention that during the expansion phase you try to sample data uniformly across groups. What do you do exactly in this phase? Isn't all of the minority data used in the warmup phase?\n\n**Q4. Intuition for expansion phase:** Related to Q3, why is performance on the training data distribution go down in Fig 4 (b), even when you reset the momentum variables? Is the training loss going up then? Why would this happen? Or is the data not just sampled from the training distribution early in the expansion phase?", "limitations": "**References**\n\n[1] [_Last Layer Re-Training is Sufficient for Robustness to Spurious Correlations_](https://openreview.net/forum?id=Zb6c8A-Fghk);\nP. Kirichenko, P. Izmailov, A. G. Wilson;\nICLR 2023\n\n[2] [_On Feature Learning in the Presence of Spurious Correlations_](https://proceedings.neurips.cc/paper_files/paper/2022/hash/fb64a552feda3d981dbe43527a80a07e-Abstract-Conference.html);\nP. Izmailov, P. Kirichenko, N. Gruver, A. G. Wilson;\nNeurIPS 2022", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "**S1.** The authors provide a theoretical analysis, that provides some intuition for the method\n**S2.** The proposed method is simple and computationally cheap\n**S3.** The authors achieve strong empirical performance across three benchmark datasets", "weaknesses": "**Q1/W1. Relationship to DFR**\n\nThe works [1, 2] advocated for an approach that is almost opposite to PDE: they train a model with standard ERM, and then retrain only the last layer on a group-balanced dataset. PDE on the other hand pretrains the model on a group-balanced dataset, and then finetunes with standard ERM. The methods achieve similar performance, if we consider DFR^Val, which is the main method in [1] (the authors argue that DFR^Val uses the validation data, but I believe PDE also uses the same amount of validation data to tune the hyper-parameters?). In fact, PDE achieves better performance on CelebA and CivilComments, while DFR achieves better performance on waterbirds.\n\nHowever, conceptually [1, 2] argue that in fact standard ERM learns the core features as well as GroupDRO on the standard benchmarks, and once the last layer is retrained we can recover the performance of GroupDRO. This intuition is quite the opposite of the intuition presented in this paper, which is that ERM does not learn the core features without intervention. I think it would be good if the authors could comment on this distinction.\n\nMy intuition is that the pretraining in the warmup stage prevents the model from learning the spurious feature, and makes it harder for the model to use the spurious features in the expansion stage. Alternatively, it could be that the model still represents the spurious features, but because of the momentum, it is unable to switch to using this feature in the expansion phase. It would be interesting to explore more deeply which of these options is happening.\n\nLastly, in line 287 you mention that the method is more efficient than DFR, because it does not train the model twice. However, DFR only retrains the last layer of a pretrained ERM model, which is computationally cheap, so I don't believe this statement is correct. \n\n **Q2/W2. Robustness to the choice of hyper-parameters**\n\nIt is important to understand how sensitive the method is to the choice of hyper-parameters. In particular, I would be interested in the impact of\n- Length of the expansion training phase\n- Learning rate in the expansion phase\n\nMy intuition is that the method continues going in a good direction for some time during the expansion phase, but then eventually it should converge to an optimal solution on the full data distribution, i.e. maximize the average accuracy and not the worst group accuracy. However, due to early stopping, the authors are able to achieve good performance, before the optimization learns to use the spurious feature. Is this intuition correct?\n\n**W3. Theoretical analysis**\n\nThe theoretical analysis is quite toy. The authors use a model which is basically a bag of features, i.e. the result is a sum of cubed dot-products of features and \"filters\". This model is not very non-linear. However, it is understandable, given that theoretical analysis of non-linear models is challenging.\n", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We appreciate your positive feedback and suggestions that helped us improving our work. We hope our clarifications answer your questions. \n\n---\n\n**Q1a**: Consider DFR^Val.\n\n**A1a**: Thanks for pointing this out. We agree that all methods use the information from validation data for hyper-parameter tuning and model selection, including PDE and DFR^Tr. Yet, there is still a difference for DFR^Val that it also **trains the last layer on validation data**, which is updating the model parameters in addition to hyper-parameters. For a comprehensive comparison, we've updated our table (as in Table 2 of the attached PDF) to include DFR^Val, with a notation to distinguish whether the validation data is used solely for hyperparameter tuning or also for updating the final layer of the model itself. \n\n**Q1b**: The intuition for DFR is quite the opposite of the intuition in this paper.\n\n**A1b**: We believe that PDE and DFR approach the problem differently but don't contradict each other in their understandings. Our theory indicates that, for ERM, the learning of spurious features will quickly overshadow that of core features, resulting in the reliance on spurious features when making predictions. As in Theorem 2.2, the learning of spurious feature will have surpassed a threshold while the learning of core feature remains at the same scale as initialization. \n\nThat said, we don't intend to assert that ERM won't learn the core feature at all. Despite the dominance of spurious feature in prediction, the model will still learn the core feature at a minimal rate compared to spurious. We believe that there is no contradiction between us and the intuition of DFR to recover and amplify this learning of core feature through later interventions. To reflect this, we will revise the wording of Theorem 2.2 to make it more rigorous. \n\nHowever, in PDE, the case is very different as spurious feature in balanced dataset is not useful and its gradient remains 0. The model only tends to learn spurious feature in the expansion stage if without any further intervention, as the expansion data is not group-balanced. Hence, we need momentum from warmup, so that we use the core feature preserved in momentum to amplify the core feature gradient in expansion data so that it continues to dominate the spurious feature. Our arguments are also supported by synthetic experiments. In Figure 3a of our paper, there is an observable minimal learning of core feature for ERM. Meanwhile, in Figure 3c, the model does not learn spurious feature during warmup where the groups are balanced. \n\n**Q1c**: Efficiency comparison with DFR. \n\n**A1c**: We acknowledge and appreciate the efficiency of the fine-tuning approaches. We’ll clarify this in our manuscript to reflect that DFR retrains the last layer only. However, while it's true that re-training the final layer doesn't demand significant additional computation, it still requires first training a model using ERM. Meanwhile, PDE itself converges much faster to its optimal performance as compared to the ERM stage of DFR. **We still firmly believe that our statement on the efficiency of PDE is correct.** \n\n---\n\n**Q2**: Robustness to the choice of hyper-parameters. \n\n**A2**: We appreciate the suggestion and added ablation studies in Table 4 the PDF. Namely, PDE is robust within a reasonable range of hyperparemeter selections. Meanwhile, there are preferred choices for the hyperparameters.\n- The number of data added during each expansion cannot be too large as it will harm the performance. As similarly demonstrated in Figure 6 of Appendix A, gradual expansion is important to our method and adding all data at once will result in a worse performance. \n- There is a necessity to decay the learning rate after the warmup stage, while PDE is relatively robust to the extent of decay. \n- The number of times for data expansion depends on the early-stopping on the validation data. As in Figure 1 in the PDF, a smaller learning rate will result in more data expansions. However, a smaller learning rate does not necessarily result in a better performance.\n\nWe will add the ablation studies and corresponding discussion to the revised version of our work.\n\n---\n\n**Q3**: Isn't all of the minority data used in the warmup phase?\n\n**A3**: We'd like to clarify our methodology related to the usage of training data, particularly during the warmup stage. In this initial stage, we employ all data from the smallest group in the training dataset (waterbirds, land background), while for all other groups, we select a random subset equivalent in size to this smallest group. Post-warmup, we add additional data from the training dataset that were not seen during the warmup stage. At this point, new data from other small groups still remain (landbirds, water background). If possible, we aim to maintain balance among the remaining groups when enlarging the training dataset. Ultimately, the newly added data will exclusively come from the largest group (landbirds, land background). We will make sure to explain this more clearly in our revision.\n\n---\n\n**Q4**: Is the data not just sampled from the training distribution early in the expansion phase?\n\n**A4**: To clarify, our method uses the training data (and not the validation set) for warmup and expansion stages, with the intent to include all training data by the end of the expansion stage. While the data used in the expansion phase is drawn from the training dataset, remains new to the model, having not been used during the warmup stage. As we add new data, there's a minor increase in training loss, which quickly drops as training advances. Regarding the referenced figure, both average accuracy and worst-group accuracy are plotted on test data. As we use this figure to highlight the importance of momentum, we want to focus on model's actual performance as opposed to the training. We believe that the performance on test data will not always align with the trend of training loss as well. "}]

Author rebuttal:
We sincerely thank the reviewers for the feedback and especially the detailed suggestions that help us improve our manuscript. We appreciate that many reviewers commend our paper for its clarity, the alignment of our theoretical insights with empirical findings, the simplicity and novelty of our method, and the effectiveness of our method shown in experiments. In response to the questions raised, we've addressed each review with itemized answers. The updated additional tables and figures are organized into the 1-page PDF that we attach here. We hope this provides clarity and resolves any potential misunderstanding. Should there be additional comments or questions, we are more than happy to discuss them. 

Meta-review: This paper proposes a novel strategy to reduce the impact of spurious correlations 
in the context of imbalanced datasets, and shows improvement both in practice
and in theory (under a simpler model). The reviewers appreciate the contribution
and clarity of the exposition, and after a thorough discussion seem to have arrived at a consensus to accept the paper.
Decision: Accept (poster)