Spurious Correlation-Aware Embedding Regularization for Worst-Group Robustness
Authors: Subeen Park
JOOWANG KIM
Hakyung Lee
Sunjae yoo
Kyungwoo Song
Venue: ICLR 2026 Poster (2026)
Keywords: Spurious Correlation
Representation Learning
Embedding Regularization
Domain Generalization
Primary area: applications to computer vision, audio, language, and other modalities

Abstract: Deep learning models achieve strong performance across various domains but often rely on spurious correlations, making them vulnerable to distribution shifts. This issue is particularly severe in subpopulation shift scenarios, where models struggle in underrepresented groups. While existing methods have made progress in mitigating this issue, their performance gains are still constrained. They lack a theoretical motivation connecting the embedding space representations with worst-group error. 
To address this limitation, we propose Spurious Correlation-Aware Embedding Regularization for Worst-Group Robustness (SCER), a novel approach that directly regularizes feature representations to suppress spurious cues. We theoretically show that worst-group error is influenced by how strongly the classifier relies on spurious versus core directions, as identified from differences in group-wise mean embeddings across domains and classes. 
By imposing theoretical constraints at the embedding level, SCER encourages models to focus on core features while reducing sensitivity to spurious patterns. Through systematic evaluation on multiple vision and language tasks, we show that SCER outperforms prior state-of-the-art methods in worst-group accuracy. Our code is available at \href{https://github.com/MLAI-Yonsei/SCER}{https://github.com/MLAI-Yonsei/SCER}.

Published as a conference paper at ICLR 2026
SPURIOUS CORRELATION-AWALL EMBEDDING REGU- LARIZATION FOR WORST-GROUP ROBUSTNESS
Subeen Park', Joowang Kim², Hakyung Lee", Sunjae Yoo', Kyungwoo Song"
1 Yonsei University, Republic of Korea
2L CNS, Republic of Korea
3 Korea Development Bank, Republic of Korea
ABSTRACT
Deep learning models achieve strong performance across various domains but of- ten rely on spurious correlations, making them vulnerable to distribution shifts. This issue particularly severe in subpopulation shift scenarios, where models struggle in underrepresented groups. While existing methods have made progress in mitigating this issue, their performance gains are still constrained. They lack theoretical motivation connecting the embedding space representations with worst-group error. To address this limitation, we propose Spurious Correlation- Aware Embedding Regularization for Worst-Group Robustness (SCER), novel approach that directly regularizes feature representations to suppress spurious cues. We theoretically show that worst-group error influenced by how strongly the classifier relies on spurious versus core directions, as identified from differ- ences group-wise mean embeddings across domains and classes. By impos- ing theoretical constraints at the embedding level, SCER encourages models to focus on core features while reducing sensitivity to spurious patterns. Through systematic evaluation on multiple vision and language tasks, we show that SCER outperforms prior state-of the-art methods in worst-group accuracy. Our code is available htttt:///gtthub... wtt.:at...oo///aaaa....... Yonsei/SCER
INTRODUCTION
Deep neural architectures have transformed performance standards in many domains. However, they are still highly vulnerable to spurious correlations in real-world datasets (Yang al.] [023a]] Iz- mailov et al. 2022 Sagawa et al., 2020). Spurious correlations are patterns in the training data that are statistically ly associated with target labels but lack genuine causal relationship. Such correla- tions are often the result of biases in the dataset, causing models to fail when faced with distribution shifts (Ye et al., 2024). Recent studies have shown that this issue is particularly severe in subpopu- lation shift scenarios, where the distribution of specific subpopulations in the test data differs from that in the training data (Sagawa et al.) Yang et al.) 20236). Models trained with Empirical Risk Minimization (ERM) tend over-rely these spurious correlations, ERM minimizes the aver- age loss without considering subpopulation imbalances (Vapnik 2013] Izmailov et al., 2022). As result, models rely on easily learned but non-robust features, leading poor worst-group accuracy and unreliable performance across underrepresented subpopulations.
Numerous approaches have been developed handle this challenge, which can be broadly grouped into several categories (Yang et al. 2023b). Subpopulation robustness methods (Sagawa et al.) Duchi & Namkoong 2021 Liu et al. 2021 Yao al. 2022 Deng al. 2023) focus improving worst-group performance through various techniques such as reweighting, multi-stage training, or specialized loss functions. Domain invariant methods (Arjovsky al., 2019 [unn & Saenko 2016)
attempt to learn features that generalize across different domains by aligning feature distributions. Data augmentation approaches (Zhang et al., 2018) create synthetic training examples improve generalization. Class imbalance methods (Japkowicz, 2000; Cui et al.) 2019 Ren et al.) (2020) ad- dress the uneven distribution of samples across classes. However, these approaches primarily influ- ence model learning indirectly through sample reweighting or feature alignment, without explicitly
"Corresponding author.

---

Published as a conference paper at ICLR 2026
Input
y- Y1
Spurtoous Feature a 6ac)
Representation by
domatn de
O
y 31
domain dg
O' 4x.
domain de
1.
Peature Extractor
domain
O.G
Care Feature
Depresantations learmed by
Embedding
Cicore
Space
Pure
Classifler
Spurious Magnituds
WeiightSSpurious Alignment
Spuriaus Loss
Ourpur
Core Magnitude
x
Weight-Core Alignment
Core Loss
SCHX cargets Core Peature via embedding regulartaction
Figure 1: Overview of the Spurious Correlation Embedding Regularization (SCER) frame- work. Input data is encoded into embeddings, from which group-wise mean embeddings are com- puted for each label-domain pair. SCER decomposes these into Spurious Features, capturing do- main differences within each class, and Core Features, capturing class differences within each domain. Spurious Magnitude and Core Magnitude quantify the strength of domain-driven and label-driven variations. Weight-Spurious Alignment and Weight-Core Alignment measure how strongly the classifier aligns with each component. SCER constructs Spurious Loss and Core Loss from these terms to regulate the embedding space, reducing Spurious Loss to suppress domain- specific patterns while increasing Core Loss reinforce label-relevant signals, thereby improving robustness to domain bias.
constraining how spurious features are encoded within the embedding space. Consequently, spurious correlations may persist, limiting robustness under distribution shifts.
To address this limitation, we propose Spurious Correlation-Awar Embedding Regularization for Worst-Group Robustness (SCER), a novel method that directly regularizes feature representations to prevent reliance spurious features. As illustrated in Figure SCER operates the embedding level, imposing explicit constraints that guide the model toward learning more robust features. In this study, provide a theoretical relationship between worst-group error and the structural properties of the representation space under subpopulation shift scenarios, where spurious correla- tions exist between domain and label. We identify two key factors that influence worst-group error through our theoretical analysis. First, for the same-label samples across different domains, we reg- ularize their embeddings align on essential classification features. Second, for instances sharing the same domain but with different labels, we introduce additional constraints to reduce the impact of -emaai--ppecffc spurious cues. Through this approach, SCER prevents the model from overfitting to domain-level artifacts, enabling it to learn robust core features more effectively.
By structuring the model's 's representation space. SCER improves worst-group accuracy while main- taining strong overall performance. Through extensive experiments across image and text datasets, we demonstrate that SCER outperforms prior methods in worst-group accuracy. We also provide quantitative and qualitative evidence that embedding- constraints yield robust feature represen- tations even under spurious domain shifts, confirming the effectiveness of the SCER.
2 RELATED WORKS
2.1
SUBPOPULATION SHIFT
Machine learning models degrade under subpopulation shift, where the test subpopulation distribu- tions differ from those in the training data (Yang et al. (023b). ERM's focus on average training loss makes models susceptible to spurious correlations (Vapnik, 2013 Izmailov et al.) (022). To address this issue, various methods have been suggested, which can be broadly categorized as sam- ple reweighting and data augmentation strategies. GroupDRO (Sagawa et al.) reduces worst-group error by dynamically upweighting samples from groups with higher loss. LfF (Bahng et al. 2020) employs two-stage approach that first detects spurious correlations by training biased model and then debiases second model by reweighting the loss gradient. GIC (Han & Zou 2024) improves the accuracy of group inference by leveraging spurious attribute associations. For data augmenta-
2

---

Recent studies in the text and image domains have actively explored methods refine latent spaces, aiming to improve representation quality and better capture the underlying structure among sam- ples (Patil et al., 2023; Ming et al.). However, in the domain generalization setting, research has primarily focused on learning invariant features rather than explicitly modifying the embedding space. Previous work has attempted mitigate unintended correlations arising from background, lighting. or dataset-specine biases (Yang et al. 2023b). Hermann & Lampinen (2020) analyzed how networks differentiate robust and spurious features, while LfF (Bahng al. (020)) introduced a two- stage training approach to guide model learning. More recently. ElRep (Wen et al. 2025) applies norm penalties to final-layer representations with demonstrated practical effectiveness. However, these methods typically guide model learning indirectly rather than explicitly constraining spurious feature encoding in the embedding space (Patil et al. [023 Ming et al.). In contrast to previous ap- proaches, propose directly impose explicit loss function the embedding space, ensuring Published as a conference paper at ICLR 2026
tion, LISA Yao et al., 2022) enhances robustness by interpolating between subpopulation samples to encourage invariant representations. Similarly, PDE (Deng et al. 2023) gradually expands the training set, allowing models to first learn core features before incopporaiig more diverse samples. While prior research focuses primarily on sample reweighting or data augmentation, we propose SCER to directly regularize the embedding space and reduce reliance spurious features. By plicitly penalizing reliance on spurious cues at the representation level, SCER significantly improves robustness across various distribution shift benchmarks.
2.2 EMBEDDING SPACE
that learned representations minimize reliance on spurious correlations and improve generalization under distribution shifts. Our work addresses these theoretical gaps by providing motivation for embedding regularization and establishing clear connections to robust generalization.
METHODOLOGY
3.1 PROBLEM SETTING
We consider a classification problem where each data point E X' is associated with label y E y and domain E DD where y = {yı 1/2../2.... ./...... :ym is the set m classes and = [di, d2,..., ,d,] is the set of k domains. Each pair (y, d) E y D defines subpopulation, which may exhibit a spurious correlation if d is not causally related to y but appears disproportionately often with certain y values. We aim to learn classifier f X - V, which consists of a feature extractor fw : + R and a classifier f&z IRP - V, such that f(z) = (x)) A standard ERM objective aims to minimize
min Be [([(()), v)]:
where l is the loss and P is the full data distribution. Although ERM effectively reduces average error, it can fail on minority groups, leading to large misclassification rates in those groups.
Subpopulation Shift. We consider learning setting where performance should remain robust across all subpopulations defined by label-domain pairs (y, d). To address imbalances among these groups, we adopt the worst-case optimization framework of Group Distributionally Robust Opti- mization (GroupDRO), which can be formulated as
min max Q
Σ (y.d)EYXD
((p.d) Bladdd [e([(z), y)]
Here, py, represents the data distribution conditioned on subpopulation (v, d), g represents how much weight to give each subpopulation during training, and Q denotes the set of all possible ways to assign these weights. This minimax formulation ensures robustness across all subpopulations.
To further enhance robustness, we introduce a regularization approach motivated by the decompo- sition of worst-group error into spurious and core components within the embedding space. Rather than addressing group errors solely through loss optimization, we incorporate structural regular- ization that encourages the classifier to align with label-consistent Core Feature while penalizing dependence on domain-specific Spurious Feature. These regularization terms, formulated through directional correlations in the embedding space, are detailed in the following section.
3

---

Published as a conference paper at ICLR 2026
Input
Feature Extractor
Embedding Space
Classifier
Output
werf Agual
Spuriaw Pasture
Sngle Algenser
Spurioous Loss lxpor
Bepur
Spuriass Magnaiaad
11A
Lyrge
67's Au--)
CaPe
lererell
Acore
Core Mognitade
Core Loss
Waggle Algemeen
Figure 2: Overview of the SCER training framework. Input data encoded into embeddings, from which group-wise mean embeddings are computed to derive spurious and core directions. The framework combines worst- gorst-group classification loss with embedding regularization that penalizes spurious alignment and promotes core alignment.
3.2 SCER
We propose SCER, which enhances the generalization performance of classifiers by decomposing the worst-group error and applying embedding-leve regularization to each component, as shown in Figure 2 Let Femb fm (x) R2 denote the embedding vectors of input æ obtained by feature tractor Ju with parameters w. where is the embedding dimension. For each subpopulation defined by a label-domain pair (y, d), define its mean embedding as
p (v.d) Ez~Fy,.d [f (x)) E TR..
We compute mean embeddings representative vectors for each subpopulation, enabling decom- position of worst-group error into two primary components.
Spurious Feature. The embedding mean difference in feature representations across different do- mains within the same class. Since we assume that label-domain pairs (y, d) may exhibit spurious correlations, such differences should be minimized prevent the model from overfitting domain- specific artifacts. Formally, the spurious difference for a given class y is defined as
(y.d4.5) = =Iulvdd) ulldd)). Vy e V, di, d3 D, d: dd. spur
Core Feature. The embedding mean difference in feature representations between different classes within the same domain. This quantity captures the actual signal that the classifier should leverage for robust prediction. Formally, the core difference for given domain is defined
(36.5d)) () cure
V Vy, Mj y, d E D, y/ Mj-
spur E IR and A (3.,,,)) E Rº capture key structural variations in the data. To generalize these quantities across multiple classes and domains, we compute global directions using aggregated differences across groups. We define the Spurious Direction A E R as the average of differences mean embeddings across domains within the same class, and the Core Direction Agone E RP as the average of differences across classes within the same domain
The defined A (v.d..))
Aspar
Eyey
A
(u.d..s)] Spur
Acore = EEED A
(9L.).d)
To ensure proper interpretation based on theoretical insights, we normalize these direction vectors using the -nnorm, defined as || = Vo To Ev for v E IR?, where IRexP denotes the empirical covariance matrix of the embedding vectors. This normalization accounts for the geometric struc- ture of the embedding space, producing scalar quantities Spurious Magnitude || and Core Magnitude | for use in regularization. We also derive regularization terms by analyzing how classifier weights align with the Spurious and Core Directions. The key idea is that overreliance on Spurious Directions leads poor generalization minority groups, while alignment with Core

---

Published as a conference paper at ICLR 2026
Directions supports robust performance. Let 3 =18 18i..... ........ 27. , ,,, R denote the current weight matrix of the classifier fg. where each column 3j E R" corresponds to the weight vector for class j. During training, we the current classifier weights sures. To quantify the alignment between classifier weights and directional signals, we compute the average class-wise correlation with each direction using the E-norm. We refer these measures as Weight-Spurious Alignment and Weight-Core Alignment
compute the alignment mea-
A 13 A
(3). Acore)
cor(33, spur) =
(I)
1 A
These correlation terms measure how closely the classifier's 's decision boundaries align with spurious core feature directions, providing the basis for embedding-leye regulalaai regularization.
Using this definition, we define the Spurious and Core Loss as:
Lspur cor(33" Appur)] Appurly, Lcore = cor(33". Acore) A
The Spurious Loss Lypur penalizes alignment with spurious directions, scaled by the magnitude of spurious variation in the data. The Core Loss Leare encourages alignment with core directions, with the negative sign ensuring that more substantial core alignment reduces the overall loss. Our loss formulation penalizes alignment with spurious directions and encourages alignment with core directions. By introducing control parameters Aspur and Acore, we derive our embedding loss function:
Lembodding = Aspur Acare
The final SCER objective combines embedding loss with worst-group classification loss Lagge:
Loaal Luge + Lembedding
This loss formulation mitigates overfitting to domain-specinn biases by reducing intra-class variation across domains while preserving inter-class separability. SCER promotes robust decision bound- aries, enabling strong worst-group performance under distribution shifts. For additional implemen- tation details, please refer to Appendix A.I
3.3 THFORETICAL ANALYSTS OF SCER
We now present a theoretical analysis that formally decomposes worst-group error. The following theorem demonstrates that worst-group error can be decomposed into the classifier's reliance spurious versus core components, thereby providing theoretical motivation for SCER's regulariza- tion of these components, Detailed proof' are provided in Appendix A.2
Theorem 1 WWosst-group Error Decomposition). Consider classification problem where each embedding vector x E R is associated with a label y {y-11, [+1] and a domain D = {dR. dG). Each pair (y, d) E y D defines subpopulation. We assume that the data follows group- conditional Gaussian distribution (y, d) (n Σ) where Σ is positive definite. We further assume that A ( - 1.4) is constant across d, Apur p. (((dd)) -p(9)da) is constant across y. I(dn) (dd)) E. Fg =1) P(d e (dR, dG}) }. and P(y.d) )) 0 f.ror some pairs (y, d) under extreme spurious correlations. For classifier fa sign( x) with 8" E IRP,, the worst-group error can be decomposed as
Ewgg = {+ cor(B" A Appur || - cor(B" Acore)
||
The sign + Theorem reflects which subgroup constitutes the worst group. When A > 0, the unseen minority subgroups incur higher error, corresponding to the + sign; the sign arises in the rare case where seen subgroups are worst. In the typical ERM regime with spurious correlations, the sign applies, and we retain + for generality. See Appendix A.2 for the detailed derivation.
Remark
The Theorem shows that minimizing Euge requires reducing cor(3", Apur) Apur lbx: while simul- taneously increasing cor(3", Awwe) | A |s2. The Weight-Spurious Alignment term cor(3", A
5

---

Published as a conference paper at ICLR 2026
Table 1: Performance comparison on multiple datasets, each exhibiting different levels of spurious correlations. denotes the performance reported from Yang et al. 2023b) except for ColorMNIST. denotes the performance reported from Deng et al. (2023) for Waterbird and CelebA. tt denotes the performance reported from Wen et al. (2025) for Waterbird and CelebA. For the remaining datasets, the detailed experimental settings follow Yang et al. al. (2023b), as described Appendix BB. 1.2 SCER achieves the highest worst-group accuracy in Waterbirds, CelebA, MetaShift, and COlOT demonstrating its robustness in subpopulation shift scenarios.
Algorithm
Waterbirds
CelebA
MetaShift
ColorMNIST (p = 80%)
Avg Ace
Worst Ace
Avg Acc
Worst Acc
Avg Acc
Worst Ace
Avg Acc
Worst Acc
ERM
84.1 =1.
69.1 -47
95.1 -0.2
62.6 1.5
91.3 10.3
82.6 10.
38.2 124
30.9 22.7
GroupDROf
88.8 =18
78.6 -8.6=1.0
91.4 00.6
89.0 =077
#0.1
85.6 +04
73.5 +03
73.1 =0.1
LISA
92.8 +02
88.7 -066
92.6 92.6=0.1
86.3+1.2
89.5 +04
84.1 +0.4
73.8 +0.1
73.2 =011
ReSample/
89.4
77.7 =1.2
92.0 =0.8
87.4 10.8
10.1
80.4
73.7 =0.1
72.2 00.3
PDE
92.4
90.3 =03
92.4 -08
91.0
80.1
78.1
76.6
72.9 =0.3
EIRep't
92.9 =07
88.8 =0.7
92.8 =02
91.4100
80.9
+2.5
50.3 #06
46.5 44.3
SCER
=0.2
91.2 0.2
92.7
20.2
91.
91.3
86.7
10.8
74.1 10.1
73.6 102
measures how much the classifier weights align with domain-specific variations within the same class, indicating the dependence on spurious features. The Spurious Magnitude || suullls quantifies the divergence in the distribution of data between domains within the same label; thus, minimizing this component effectively reduces spurious correlations. contrast, the Weight-Core Alignment term cor( A captures how well the classifier weights align with the true labels' discrimi- natory directions that are consistent across domains. The Core Magnitude || ||x captures the separation between different label distributions across domains, and augmenting this term enhances the model's ability to detect core predictive patterns.
4
EXPERIMENT
4.1 EXPERIMENT SETTING
Datasets We evaluate on both real-world and synthetic datasets. For vision tasks, we use Wa- terbirds (Sagawa al.). CelebA (Liu al. 2015). MetaShift (Liang & Zou| 2022), and ColorM- NIST (Arjovsky et 2019)) For language tasks, we use CivilComments (Borkan et al. 2019) and
MultiNLI (Williams et al.) 2018). These datasets are widely adopted benchmarks for evaluating robustness to spurious correlations. Detailed descriptions and statistics are in Appendix B.1.1
Models Following prior work (Gulrajani & Lopez-Pazz Izmailov et al. 2022), we use pretrained ResNet-50 (He et al. 2016) for image datasets and pretrained BERT (Idrissi et al. (2022) for text datasets. We use SGD with momentum (Polyak 1964) for images and Adam (Loshchilov
Hut-
ter: (017) for text, following standard protocols (Yang et al. 20235). Training steps 5,00) for Waterbirds, MetaShift, and ColorMNIST, and 30,000 for CelebA, CivilComments, and MultiNLI.
Baselines We evaluate our method against various bias mitigation and imbalanced learning al- gorithms. Our comparison includes ERM; subpopulation robustness methods such as Group- DRO Sagawa al.),, LISA (Yao et al. 2022), PDE (Deng et al. 2023), and EIRep (Wen et al, 2025;; ; and class imbalance techniques such as ReSample (Japkowicz 2000). For ColorMNIST, we evaluate only methods that achieve worst-grous accuracy above 70%. We conduct experiments on a broader set of baselines, with detailed descriptions and complete results provided in Appendix 1.2
and Tables [1 [2 13, and 15 We also conduct a focused comparison against GroupDRO-ES, (Iz-
mailov al. 2022). n, the most competitive GroupDRO variant with early stopping, presented in Table
Evaluation Metries
Following prior work, we use Worst-group Accuracy (Worst Acc) as the pri- mary metric to assess performance on the most vulnerable minority group, and Average Accuracy (Avg Acc) for overall performance. Our objective is to minimize the gap between these metries.
RESULTS
5.1
QUANTITATIVE ANALYSIS
Image Dataset Table [ that SCER outperforms baseline methods across four benchmarks: Waterbirds, CelebA, MetaShift, and ColorMNIST, each presenting different spurious correlation

---

Published as a conference paper at ICLR 2026
Table 2: Performance comparison on ColorMNIST under increasing spurious correlation lev- els. SCER achieves the highest worst-group accuracy across different spurious correlation levels, demonstrating robustness to spurious feature reliance in ColorMNIST.
ColorMNIST
80%
P
90%
95%
P
90%
Algorithm
Avg Ace
Worst Acc
Avg Ace
Worst Ace
Avg Acc
Worst Acc
Avg Ace
Worst Ace
ERM
38.2 +2.4
30.9 +2.7
+7.4
7.8 +3.4
41.1 +13.0
10.1 +4,55
50.7 +8.2
8.5 =5.0
GRou
73.5 +0.3
73.1 +0.1
73.5 +0.0
72.7 +0.3
72.3 + 0.1
70.7 +0.2
50.3 +3.9
38.7 +0.8
LISA
73.8 10.1
73.2 10.1
73.3 10.1
10.2
72.6 10.4
71.4 10.6
53.1
10.2 18.1
ReSample
73.7
72.2 +0.3
72.6
+1.6
71.8 =0.1
70.7 =0.5
57.1 +3.0
45.4
PDE
76.6
72.9 +0.3
72.6
70.2
73.4
70.0 +0.3
56.3 +4.3
47.5 +8.9
SCER
74.1
73.6 1002
73.6
73.0
73.5 10.3
72.8 10.3
61.0 +2.0
56.0
challenges. SCER achieves the highest worst-group accuracy in four datasets, with 91.2% Wa- terbirds, 91.4% on CelebA, 86.7% on MetaShift, and 73.6% on ColorMNIST at p 80%. These results demonstrate that embedding-level disentanglement effectively mitigates spurious bias.
Image Dataset with Stronger Spurious Correlations Table 2 reports performance ColorM- NIST under varying levels of spurious correlation p. As increases, the dataset becomes increas- ingly biased, making classification more challenging and highlighting the detrimental impact of spurious correlations on standard models. SCER consistently achieves the highest worst-group ac- curacy across all values of p. attaining 73.6% at p = 80%, 73.0% at = 90%, 72.8% 95%, and 56.0% at p 99%, outperforming all baselines.
Table 3: Performance ColorMNIST with
In addition to the standard evaluation settings in Table we examine a more challenging scenario in which minority group is completely absent during training, creating label color distribution of 50-0-10-40, shown in Table g This extreme setting tests algorithms' ability to generalize un- der harsh spurious correlations when facing plete group absence. As shown in Table [ SCER achieves 59.6% worst-group accuracy in this treme setting, significantly outperforming all ex- isting methods. This result reveals limitations of existing approaches under extreme spurious cor- relations.
one group absent. SCER achieves the highest worst-group accuracy.
ColorMNIST (One Subpopulation Omitted)
Algorithm
Avg Acc
Worst Acc
ERM
52.9 +8.4
9.7 #5.4
GroupDRO
53.6 +2.5
44.1 +6.2
LISA
49.5 =2.4
13.5 #110
ReSample
+0.8
16.3 113.0
PDE
71.0 71.0-12.6
8.3 113.9
SCER
65.3 =1.0
59.6 110
In this extreme setting where an entire subpopulation is omitted from training, GroupDRO (Sagawal ct al.) suffers from instability with extreme minority groups, poor extrapolation to unseen subpop- ulations, and regularization-caacciy trade-offss that limit its effecctiveness. Class imbalance meth- ods (Japkowicz 2000) and LISA (Yoo et al.) (022)) fail as they rely on reweighting available samples interpolating between seen domains, making them unable to handle completely missing groups. Similarly, PDE (Deng et al. (002),, despite initially learning from balanced data, cannot general- ize entirely unseen group combinations. These results show embedding-level interventions essential when complete groups are absent, as this causes extreme spurious correlations. SCER's superior performance stems from directly regularizing the representation space, mitigating spurious correlations and enabling robust generalization to unseen subpopulations.
Table 4: Performance comparison without explicit bias labels.
Integration with Environment Inference Methods. A key advantage of SCER is its modular design, which en- ables seamless integration with existing frameworks with- out requiring explicit bias labels. We integrate SCER with the Environment Inference for Invariant Learning (EIIL) framework (Creager et al. (202T). replacing EIIL's second- stage IRM objective with SCER. Table 4 shows experi- mental results integrating EIIL with SCER where envi- ronment information is provided, simulating realistic sce- narios without environment labels.
ColorMNIST (Two train envs) Avg Acc
Method
IRM
54.8 +8.3
EIIL (IRM)
58.5 +1.8
EIIL + DRO
68.2 +7.0
EIIL + SCER
72.6 +5.55

---

Published as a conference paper at ICLR 2026
We adopt the data setup from (Creager et al., 2021), with two environments differing in the strength of spurious correlations, as detailed in Table g The environments inferred by EllL achieve over 95% agreement with actual spurious groups in ColorMNIST, providing reliable pseudo-labels for SCER training. As shown in Table 4 SCER maintains its effectiveness even with inferred envi- ronments, achieving the highest test accuracy of 72.6%. Notably, while GroupDRO suffers signifi- cant performance degradation under environmental misalignment, SCER remains robust due to its embedding-level regularization. This demonstrates SCER's practical applicability and adaptability to real-world scenarios where explicit bias annotations are unavailable. Even in scenarios where environment labels are increasingly noisy, SCER continues to demonstrate robust performance, as detailed in Table [4
Text Dataset Table 5 presents SCER's performance text-based datasets. SCER consistently achieves the best results on both CivilComments for multi-domain evaluation and MultiNLI for multi-class evaluation.
On CivilComments, SCER achieves the highest worst-group accuracy of 74.0%, effectively mitigating performance degrada- tion in minority subpopulations. On MultiNLI, SCER achieves the highest worst-group accu- racy of 76.8%, demonstrating balanced learning across differ- class distributions. These results confirm that SCER's em- bedding regularization approach generalizes effectively beyond vision text-based scenarios.
Table 5: Performance comparison on text datasets. from Deng et al. (202)) for Civilcomments. ti from Wen al. (2025) for Civilcomments. SCER achieves the best worst-group accuracy across multi-class and domain settings.
Algorithm
CivilComments
MultiNLI
Avg Acc
Worst Acc
Avg Acc
Worst Acc
ERM
85,4 #0.2
63.7 #1.1
80.9 #0.1
66.8 +0.5
GroupDRO LISA
81.8 #0.6
70.6 #1.2
81.1 #0.3
76.0 #0.7
82.7 #0.1
73.7 #0.3
80.3 #0.4
73.3 +10
ReSample
42.0000.0
73.3 40.55
77.2 1002
72.3 1:08
PDE#
86.3 11.7
71.5 10.5
69.1 10.3
65.8 10.6
EIReptt
79.0
70.5 10.5
69.0 13.8
66.8 15.22
SCER
81.7 10.4
74.0 11.0
80.4 10.1
76.8 10.5
SCER demonstrates robust performance across multi-domain and multi-class environments, high- lighting its versatility in handling subpopulation shifts across modalities.
Our quantitative analysis shows that SCER consistently surpasses existing methods across diverse image and text settings, maintaining strong worst-group performance even under severe spurious correlations, missing groups, and inferred environments. This demonstrates that embedding-leve regularization effectively reduces spurious bias and supports robust generalization across domains.
5.2 QUALITATIVE ANALYSIS
Table 6: Sensitivity analysis of embedding regularization on ColorMNIST. Worst-group accuracy remains stable across settings, with the best results observed when both loss terms are combined.
ColorMNIST (p = 95%)
both À
Asonc (fixed Aspur)
Aspur (fixed Acore)
Setting
Avg Acc
Worst Acc
Acore
Avg Acc
Worst Acc
Aspur
Avg Acc
Worst Acc
72.3 +0.1
70.7
+0.2
0.0
72.8 +0..4
71.6 +0.6
0.0
72.8 +-0.3
71.6 +0.4
0.5
73,0
#0.2
72.2 10.4
0.5
72.3 10.4
71.6 #0.3
1.0
73.3 +0.2
72.0 10.2
1.0
73.5 10.3
72.8 10.3
Embedding Regularization Analysis We conduct sensitivity analysis on ColorMNIST with 95% to evaluate our proposed loss components. As shown in Table we systematically vary Asore and Aspur according to Equation 2 Results show that each regularization component indepen- dently improves worst-group accuracy over the baseline, demonstrating that both Looe and L contribute to robustness against spurious correlations. Notably, their joint optimization achieves the highest worst-group performance, confirming that the two loss terms produce complementary effects. This validates our thcoretical insight that achieving optimal worst-grout generalization re- quires simultaneously maximizing alignment with core features while minimizing reliance spu-
8

---

Published as a conference paper at ICLR 2026
rious features. To further assess parameter sensitivity in large-scale scale setting, provide additional analysis on CelebA Table 18
ERM
GroupDRO
SCER
Florer Au
-30
-02
43
Lond
Landbird on Land
haterbirdd on Land
Landbind on Water
Waterbarr on Water
Figure 3: 1-SNE visualization on the Waterbirds dataset. ERM clusters samples primarily by back- ground rather than label. GroupDRO partially mitigates this but still exhibits background-based separation within each label. SCER produces label-aligee aligned, bakkground-invaiiann embeddings, ef- fectively suppressing spurious correlations.
Embedding Space Analysis Figure 3 shows t-SNE projections of embeddings learned by the three models on the Waterbirds dataset. ERM exhibits strong comain-basee clustering, with samples sep- arating predominantly by background rather than label. Samples from different backgrounds form distinct clusters, indicating heavy reliance on spurious features rather than core class characteris- tics. GroupDRO shows reduced but still visible clustering by background within the same label, demonstrating incomplete suppression of spurious correlations.
In contrast, SCER produces label-aligned embeddings where samples cluster primarily by label regardless of background. Waterbird samples from different backgrounds intermix to form a single unified cluster, while Landbird samples from different backgrounds aggregate similarly, maintaining clear separation between the two classes. This invarriat structure provides qualitative evi- dence that SCER effectively suppresses spurious correlations via embedding-leva regularization, resulting in representations that generalize robustly across distribution shifts.
Table 7: Comparison between Euclidean norm and Σ-norm on ColorMNIST. Our pro- posed Ennorm consistently achieves higher worst-group accuracy.
Component Analysis We investigate pro- posed E-norm by replacing the standard Euclidean norm in Equation [ As shown in Table 7 the E- norm outperforms the Euclidean baseline. This im- provement comes from the natural benefits of the distance, which captures the structure of covari- ance of the features and ensures the invariance of the scale in high-dimensional spaces (Ghorbani 2019 Mahalanobis 2018).
ColorMNIST (p 95%)
Norm Type
Avg Acc
Worst Acc
Euclidean norm
72.7 +0.8
70.0 +0.4
Ennorm (Ours)
73.5 +033
72.8 +0.3
Worst ACC vs Spurious Loss
Worst ACC us Core Loss
Capar cor( Aspar)
)
A
Leore
cor( (3",
A
A
Savrius Less
Care toss
Figure 4: Scatter plots showing worst-group accuracy spurious and core metrics. Spurious Loss shows negative correlation, while Core Loss shows positive correlation.

---

Published as a conference paper at ICLR 2026
Correlation Analysis between Objective Function Decomposition and worst-group accuracy Figure 4 demonstrates the relationship between worst-group accuracy and spurious/core metrics, learning. Spurious Loss exhibits negative correlations with Worst Acc, indicating that a stronger dependence of the classifier on domain-specific characteris- ties degrades the performance of the minority group, while Core Loss shows positive correlations, demonstrating that alignment with genuine label discriminatory features enhances the generalization of the group.
revealing insights into group-invaricn
Specifically, Weight-Spurious Alignment cor(3 ( Aspor) measures how much classifier weights align with intra-class domain variations, quantifying spurious feature dependence. contrast, Weight- Core Aligmment cor( A captures how well the classifier aligns with domain-consien consistent dis- criminative directions. Spurious Magnitude || lIx and Core Magnitude || |x quantify inter- domain distributional differences and inter-label separability, respectively. The consistent patterns observed across these metrics align with our theoretical decomposition, confirming that the regu- larization terms directly address the properties essential for worst-group robustness. These results establish that balancing core signal capture and spurious correlation suppression is fundamental to group invariance.
original
GrOuDRRO
SCER
Figure 5: Grad-CAM visualization comparison on the Waterbirds. ERM heavily relies on back- ground cues, and GroupDRO only partially captures the object, whereas SCER consistently attends to bird-specific features. This demonstrates SCER's ability to suppress spurious background corre- lations while focusing on label-relevant structures.
Grad-CAM Figure 5 presents comparative analysis of Brad-CAM (Selvaraju et al. 2017) visu- alizations for three methods applied to bird classification models. ERM exhibits dispersed activation patterns across the entire image, allocating considerable attention to background regions. Group- DRO shows concentrated activation on some regions of the bird, but does not fully utilize all parts that can be considered core features for classification. In contrast, SCER concentrates strong ac- tivation across structural features such as the bird's body, head, wings, and tail, while effectively suppressing dependence background or spurious elements. These results visually demonstrate that our mean embedding-basee based penalty approach comprehensively captures diverse discriminative features critical for classification while effectively suppressing spurious factors. Additional exam- ples are provided in Figure 6
Our qualitative analysis demonstrates that SCER effectively suppresses spurious features and learns representations focused on core signals, as evidenced by embedding structures, attention visualiza- tions, and interpretability analyses that support robust group-invariant behavior.
CONCLUSION
We propose Spurious Correlation-Aware Embedding Regularization (SCER), which addresses spu- rious correlations by decomposing worst-group eиror into spurious and core components at the embedding level. By suppressing alignment with spurious feature directions while promoting con- sistency with core feature directions, the model is prevented from overfitting to domain-specite artifacts. Through extensive experiments across image and text domains, we confirm that SCER consistently achieves superior worst-group accuracy, demonstrating that explicit embedding regularization addresses subgroup bias more directly than sample reweighting feature alignment approaches. Furthermore, SCER's 's modular design enables integration with environment-infereen methods, making it practically applicable to real-world scenarios without requiring explicit bias la- bels. This work presents a promising direction for robust learning under distribution shift and is expected to contribute to practical applications where fairness is critical.
10

---

Published as a conference paper at ICLR 2026
ACKNOWLEDGMENTS
This work was supported by the National Research Foundation of Korea(NRFF grant funded by the Korea govermnentMMSTTT8882224440422220
REFERENCES
Martin Arjovsky, Léon Bottou, Ishaan Guirajani, and David Lopez-Paz. Invariant risk minimization. arXiv preprint (rXXi:1907.0289) 2019.
Hyojin Bahng, Sanghyuk Chun, Sangdoo Yun, Jaegul Choo, and Seong Joon Oh. Learning de-biased representations with biased representations. In International conference on machine learning, pp- 528-539, PMLR, 2020.
Daniel Borkan, Lucas Dixon, Jeffrey Sorensen, Nithum Thain, and Lucy Vasserman. Nuanced metrics for measuring unintended bias with real data for text classification. In Companion pro- ceedings of the 2019 world wide web conference, pp. 491-500, 2019.
Kaidi Cao, Colin Wei, Adrien Gaidon, Nikos Arechiga, and Tengyu Ma. Learning imbalanced datasets with abbel-disribbution-aaaa margin loss. Advances neural information processing systems, 32, 2019.
Elliot Creager, Jörn- Hennikk Jacobsen, and Richard Zemel. Environment inference for invariant learn- ing. In International conference machine learning, pp.
PMLR, 2021.
Yin Cui, Menglin Jia, Tsung- Yi Lin, Yang Song, and Serge Belongie. Class-balancee loss based effective number of samples. In Proceedings of the IEEE/CVF conference computer vision and pattern recognition, pp. 9068-9266222 02889927,2222 2019.
Yihe Deng, Yu Yang, Baharan Mirzasoleiman, and Quanquan Gu. Robust learning with progressive data expansion against spurious correlation. Advances in neural information processing systems, 36:1390-1402. 2023.
John Duchi and Hongseok Namkoong. Learning models with uniform performance via distribu- tionally robust optimization. The Annals of Statistics, (9(3):1378-1)) 1406, 2021.
Hamid Ghorbani. Mahalanobis distance and its application for detecting multivariate outliers. Facta Universitatis, Series: Mathematics and Informatics, pp. 583-595, 2019.
Ishaan Guirajani and David Lopez-Paz. In search of lost domain generalization. In International Conference on Learning Representations.
Yujin Han and Difan Zou. Improving group robustness on spurious correlation requires preciser group inference. In International Conference on Machine Learning. pp. 2024.
PMLR,
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp- 770-778, 2016.
Katherine Hermann and Andrew Lampinen. What shapes feature representations? exploring datasets, architectures, and training. Advances in Neural Information Processing Systems, 33: 9995-10006. 2020.
Badr Youbi Idrissi, Martin Arjovsky, Mohammad Pezeshki, and David Lopez-Paz. Simple data balancing achieves competitive worst- worst-group-uee grwuupaccuraae In Conference Causal Learning and Reasoning, pp. 336-351. PMLR, 2022.
Pavel Izmailov, Polina Kirichenko, Nate Gruver, and Andrew G Wilson. On feature learning in the presence of spurious correlations. Advances in Neural Information Processing Systems, 35: 38516-38532, 2022.
11

---

error for "importance- Published as a conference paper at ICLR 2026
Nathalie Japkowicz. The class imbalance problem: Significance and strategies. In Proc. of the Int') Conf. on artificial intelligence, volume 56, pp. 111-117, 2000.
Polina Kirichenko, Pavel Izmailov, and Andrew Gordon Wilson. Last layer re- re-training is sufficient for robustness to spurious correlations. In The Eleventh International Conference on Learning Representations.
Kuo- Wei Lai and Vidya Muthukumar. Sharp analysis of weighted estimators in the overparameteriieed regime. In 2024 IEEE International Symposium on Information Theory (ISIT), pp. 3701-3706. IEEE, 2024.
Haoliang Li, Sinno Jialin Pan, Shiqi Wang, and Alex C Kot. Domain generalization with adver- sarial feature learning. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 5400-5409, 2018
Weixin Liang and James Zou. Metashift: a dataset of datasets for evaluating contextual distribution shifts and training conflicts. In International Conference on Learning Representations, 2022.
Tsung-Y Yi Lin, Priya Goyal, Ross Girshick, Kaiming He, and Piotr Dollár. Focal loss for dense object detection. Proceedings of the IEEE international conference on computer vision, pp. 2980-2988, 2017.
Evan Z Liu, Behzad Haghgoo, Annie S Chen, Aditi Raghunathan, Pang Wei Koh, Shiori Sagawa, Percy Liang, and Chelsea Finn. Just train twice: Improving group robustness without training group information. In Marina Meila and Tong Zhang (eds.), Proceedings of the 38th International Conference on Machine Learning. volume 139 of Proceedings of Machine Learning Research, pp. PMLR, 18 24 Jul 2021. URL https::/proceedd.nggg mlr wwwps/ms...reeeerrreem
1ic21f html
Ziwei Liu, Ping Luo, Xiaogang Wang, and Xiaoou Tang. Deep learning face attributes in the wild. In Proceedings of the IEEE international conference computer vision, pp. 3730-3738, 2015.
Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. arXiv preprint 1rX1v:171..01101, 2017.
Prasanta Chandra Mahalanobis. On the generalized distance in statistics. Sankhya: The Indian Journal of Statistics, Series A (2008-), 80:S1-S7, 2018.
Yifei Ming, Yiyou Sun, Ousmane Dia, and Yixuan Li. How to exploit hyperspherical embeddings for out-of-distribution detection? In The Eleventh International Conference Learning Repre- sentations.
Junhyun Nam, Hyuntak Cha, Sungsoo Ahn, Jaeho Lee, and Jinwoo Shin. Learning from failure: De-biasing classifier from biased classifier. Advances Neural Information Processing Systems, 33:20673-2066 2020.
Vardan Papyan, XY Han, and David Donoho. Prevalence of neural collapse during the terminal phase of deep learning training. Proceedings of the National Academy of Sciences, 117(40): 24652-24663, 2020.
Rajvardhan Patil, Sorio Boit, Venkat Gudivada, and Jagadeesh Nandigam. A survey of text repre- sentation and embedding techniques in nlp. IEEe Access, 11:36120-36144 2023.
Boris T Polyak. Some methods of speeding up the convergence of iteration methods. Ussr compu- tational mathematics and mathematical physics, 4(5):1-17, 1964.
Jiawei Ren, Cunjun Yu, Xiao Ma. Haiyu Zhao Shuai Yi, et al. Balanced meta-softmax for long- tailed visual recognition. Advances in neural information processing systems, 33:4175-4186, 2020.
Shiori Sagawa, Pang Wei Koh, Tatsunori B Hashimoto, and Percy Liang. Distributionally robust neural networks. In International Conference Learning Representations.
12

---

Published as a conference paper at ICLR 2026
Shiori Sagawa, Aditi Raghunathan, Pang Wei Koh, and Perey Liang. An investigation of why overparameterization exacerbates spurious correlations. In International Conference on Machine Learning. pp. 3346-8356 PMLR, 2020.
Mohamed El Amine Seddik, Cosme Louart, Mohamed Tamaazousti, and Romain Couillet. Random matrix theory proves that deep learning representations of gan-data behave as gaussian mixtures. In International Conference on Machine Learning, pp. 5573-8582, PMLR, 2020.
Ramprasaath R Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedantam, Devi Parikh, and Dhruv Batra. Grad-cam: Visual explanations from deep networks via gradient-based local- ization. In Proceedings of the IEEE international conference on computer vision, pp. 618-626, 2017.
Baochen Sun and Kate Saenko. Deep coral: Correlation alignment for deep domain adaptation. In European conference on computer vision, pp. 443-450. Springer, 2016.
Vladimir Vapnik. The nature of statistical learning theory. Springer science & business media, 2013.
Catherine Wah, Steve Branson, Peter Welinder, Pietro Perona, Serge Belongie, et al. The caltech- uesd mads-200-20111 dataset. Technical report, Technical Report CNS-TR-2011-001, California Institute of Technology, 2011.
Tao Wen, Zihan Wang, Quan Zhang, and Qi Lei. Flastic representation: Mitigating spurious corre- lations for group robustness. In International Conference Artificial Intelligence and Statistics, pp. 541-549. PMLR, 2025.
Adina Williams, Nikita Nangia, and Samuel Bowman. A broed-coverage challenge corpus for sen- tence understanding through inference. Proceedings of the 2018 Conference of the North Amer- ican Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume (Long Papers), pp. 1112-1122, 2018.
Yu Yang, Besmira Nushi, Hamid Palangi, and Baharan Mirzasoleiman. Mitigating spurious cor- relations in multi-modal models during fine-tuning. In International Conference Machine Learning, pp. 39365-39379. PMIR, 2023a.
Yuzhe Yang, Haoran Zhang, Dina Katabi, and Marzyeh Ghassemi. Change is hard: closer look at subpopulation shift. In Proceedings of the 40th International Conference on Machine Learning, pp. 39534-396677 2023b.
Huaxiu Yao, Yu Wang, Sai Li, Linjun Zhang, Weixin Liang, James Zou, and Chelsea Finn. Im- proving distri-iittiittionn robustness via selective augmentation. In International Conference Machine Learning. pp. 25407-25437. PMLR, 2022.
Wenqian Ye, Guangtao Zheng, Xu Cao, Yunsheng Ma, and Aidong Zhang. Spurious correlations in machine learning: A survey. arXiv e-prints, pp. arXiv-2402, 2024.
Hongyi Zhang, Moustapha Cisse, Yann N Dauphin, and David Lopez-Paz. mixup: Beyond empirical risk minimization. In International Conference on Learning Representations, 2018.
Bolei Zhou, Agata Lapedriza, Aditya Khosla, Aude Oliva, and Antonio Torralba. Places: A 10
million image database for scene recognition. IEEE transactions pattern analysis and machine intelligence, 40(6): 2017.
13

---

Published as a conference paper at ICLR 2026
A
METHODOLOGY
This section provides comprehensive algorithmic details and theoretical proofs for the SCER frame- work presented in the main paper. We first present detailed step-by-step procedures for SCER imple- mentation, including covariance computation, embedding regularization, and loss optimization, as summarized in the main text. Subsequently, we provide formal proofs for the theoretical foundations underlying our embedding regularization approach, including the decomposition of the embedding space into core and spurious components and the optimality conditions for robustness.
A.1 OVERALL PROCESS OF SCER
Recall from Section that we define the Spurious and Core Loss as:
Logur = cor( Aspur)l Leore = cor(13" Acore) Awwe/|ss
The Spurious Loss Lsprr penalizes alignment with spurious directions, scaled by the magnitude of spurious variation in the data. The Core Loss Leore encourages alignment with core directions, with the negative sign ensuring that more substantial core alignment reduces the overall loss. Our loss formulation penalizes alignment with spurious directions and encourages alignment with core directions. By introducing control parameters Asprr and Acares we derive our embedding loss function:
Lembedding
AspurhSsuur
Aeure
The final SCER objective combines embedding loss with worst-group classification loss Luge:
Lotall
Cuge + Lembedding
This loss formulation mitigates overfitting to domain-specion biases by reducing intra-class variation across domains while preserving inter-class separability. SCER promotes robust decision bound- aries, enabling strong worst-group performance under distribution shifts.
As shown in Equation 3] the total train- ing loss combines two complementary components. The first term, Luge, is the worst- classification loss derived from GroupDRO, which addresses subpopula- tion imbalances through robust optimiza- tion (Worst-Group Error-based Classifi- cation). The second term, Lembedding is an embedding-level regularization that explic- itly controls the alignment between classi- fier weights and Spurious/Core Directions in the feature space CEmbedding-basee Regularization). This regularization penal- izes the classifier's reliance domain- specific artifacts while encouraging align- ment with label-consistent features. The two components work synergistically: the clas- sification loss ensures robust performance across subpopulations, while the embedding regularization structures the representation space for better generalization. The com- bined objective is optimized via gradient- based methods (Loss Optimization). The details of each step are illustrated in Algo- rithm 1
Algorithm SCER Training Process
Require: Data (I, U, d), parameters (w, B), hyper-
params (n, Aspur Agore)
Initialize q 1, E+I
for each step do
Femb + (x) (1) Worst-Group Error based Classification Cly, Cly.fo(oen fa((Fm)))
((y.d) 4( (xp(nE(C(v.a)))
q 9/ (y.d)eDDD (y.d)
Luge E (,, (s (()))
(2) Embedding- Regularization
E = N E, (Temb, 5 t current classifier weights (#f (8j.A.p))
Remb) T
cor(3", Aqui) ES=
1187 |1gel1
cor( Acm)) Em [1;; A
(0) A.m)
Capur cor( Aspur)
A
Loure cor( Acore)
Aore
Lembedding >ssue Expur X Ceore
(3) Optimize
Listal = Luge + Lembedding
Update (w,3) 3) via gradient descent end for
Worst-Group Error-based Classification. The feature vector Zemb E IR,, where p is the embed- ding dimension, is used to compute classification losses for each group by comparing predictions

---

Published as a conference paper at ICLR 2026
with ground-truth labels. These group-wise losses are aggregated, and group-specific weights updated using an exponentiated gradient scheme. This reweighting emphasizes underperforming groups, helping to correct distributional imbalances and enhancing robustmess to worst-group rors.
Embedding-bawee Regularization. This step regularizes the learned representation by explic- itly distinguishing spurious and directions in the embedding space and aligning the decision boundary accordingly. To capture group-wise variability, a covariance matrix E E IR2x com- puted from the feature embeddings Temb- The core and spurious directions, denoted by core TRP and Aspur Re respectively, are derived from the groupwise mean embeddings. To assess the classi- fier's alignment with these directions, we use the current classifier weights 57 R2 for each class j. which are dynamically updated during training via gradient descent. Using the weight matrix 3 the spurious and core components -Srtr and Leor are computed. Their weighted combination forms the directional loss Lembedding- Minimizing this objective reduces the classifier's reliance on spuri- ous directions while encouraging alignment with core directions, guiding the representation toward meaningful semantic structure. This regularization promotes intra-class alignment across domains and enhances inter-class separation, thereby mitigating spurious correlations in the embedding space and reducing worst-group error.
Loss Optimization. The total loss is computed by combining the worst- masst-group-eree loss Lugs with the directional regularization loss Lembedding. Minimizing this objective ensures that the classifier per- forms well across all groups while learning representations that robust to spurious correlations. By aligning the embedding space with core directions and suppressing spurious ones, the method enhances robustness worst-group errors under distribution shift.
A.2 THEORETICAL ANALYSIS OF SCER
We now present theoretical analysis that formally decomposes the worst-group error. Following prior work (Yao et al. 20222,, we adopt a Gaussian mixture model to analyze how worst-group error arises from domain interactions.
In the main text, we decompose the classifier f into feature extractor fw and a linear classifier fs. where the representation given by the embedding Zemb fu) E TRP, Throughout this subsection, we conduct the analysis in the embedding space and, with slight abuse of notation, we overload the symbol x to denote the embedding:
x Zemb
Consequently, all distributions the form x (y, d) below are to be understood as distributions over embedding vectors.
A.2.1 BINARY SETTING
Consider a classification problem where each embedding vector IR is associated with label
y [v-1, and a domain D = [dn, dc). Each pair (v. d) E y D defines subpopulation. We assume that the data follows group-conditional Gaussian distribution x: (y. d) ~ N (H((v Σ)
where Σ is positive definite. We further assume that Ague (v++1.4) .d) (V- 1.4) is constant across
d, Aspur μ (v.dd)) ((uudo) is constant across y, Y(da) = (da) = Σ, PP +1) P(d e
{dR, da)) = 2. and Fy,d Rs 0 for some pairs (y, d) under extreme spurious correlations.
Proposition 1 (ERM solution under Cross-Entropp Loss). Under this assumption, the ERM solution with cross-entropy loss satisfies
8 P
À Apore + Aspurs
Proof. The ERM problem using cross-entropy loss as a logistic regression surrogate is min E[log(1 + exp(-v(8") x 8o)1)]
8.9g
2

---

Published as a conference paper at ICLR 2026
The population minimizer 8 satisfies the score equation
- [61] =0
((t)
Ite-
Under the Gaussian assumption with shared covariance 2, the Bayes classifier is derived from the log-likelihood ratio
p(ar 9+1) (fll+a) -(0-1)) - ((((vtt) + E ((v-1)), p( y-1)
log
which is linear in with normal vector E Σ- (1((0t)) (()-1)). Since logistic regression is Fisher- consistent under correct model specification, the cross-entropy minimizer /3r aligns with the Bayes direction:
8 MEET (w((r) μ 1)).
In the extreme shift setting, the effective class means correspond to ((strdd)) and μ ((0-1,da), Thus
8" x Σ' 2 - -((u-ido))).
Decomposing the difference:
( 1.dR) f 1.da) = (p(v12dR)) - p1v 1.dx)) + (ulo idda) ulu ido)) = -core + Aspur Å,
we conclude
8 X
Remark Our theoretical analysis relies on modeling the embedding distribution Rº under each subpopulation (y, d) using a Gaussian with a shared covariance matrix E. This assumption enables the tractable LDA-style derivation of Proposition [ and Theorem [ where the classifier direction admits the closed-form expression 9 Σ-' Importantly, this assumption is empir- ically well-grounded, [Seddik et al. (2020) show that deep embeddings exhibit a dominant shared covariance structure across groups, with group-specific differences appearing higher-order per- turbations. Furthermore, the neural collapse phenomenon indicates that within-class covariances converge toward one another training progresses (Papyan al. 2020).
Theorem (Worst-group error Decomposition). Under this assumption, for classifier fB (3 v) with 3 IRP, the worst-group error can be decomposed as
Ewge ( cor(8", A Appur I/ - cor( Acore) ||
Proof. Consider the linear classifier
f(x) sign(3 x 8o)-
For subgroup (y, d),
P(f(x) #y (v, $ (- plann)) + (o)
|| V ΣB
||
Using 3o = 3 E[xr under extreme spurious correlations, we obtain the following.
Under the extreme shift, the effective training distribution consists of (H+1, dR) and (3-1, dG). so
E[r] ((((++) p a(v-1.do)) and 3o - E[x]. Substituting into the subgroup error for-
mula gives P(f(x) # y (v. d)) H{ n (alv.4) -ull)))_(+ll) For (U41, dR): p(l.idd)) E[z)
1181 (f((v+rda)) - (do)) = Á. hence B(U+I,RR) of
Applying the same procedure
to all four subgroups:

---

Published as a conference paper at ICLR 2026
B(y+1, 46
A
(seen)
||
(2 A Anore) T
O(y-1,dR)
1
(unseen)
(22 - A
E(y+1, dc)
||
(unseen)
B(y-1, =
|T
(seen)
Let
A
== T
Then
F(y+1, ((-2),
Fly-1, 1. ((-2).
By Proposition f & Σ-I A, so Å 3t
|
> O. implying > O. Both seen subgroups
therefore have identical error $ -2) < ((0)
0.5. The unseen subgroups (y-1,dz), (V+1, de)
( A T
A - 9 Tells
have error + (w) where alignment A core reduces and thus (ww).
Note that w
so increasing the core
Ewge max 1 ((-2)) (ww).
Finally, writing Ä = Appue + Acore and substituting into w and 22
( + A 2A T
A
AT AT
181s
2 11
2 (Aupur Aeare)
TB
A A Tp
3
A AT 8
1
|
Since Ewge 4(max( -2, w)). we obtain
(+1
BT TTA Asprr 1 BT core
Ewge +b
||ا||
||
where corresponds (unseen group is worst) and corresponds -z (seen group worst). With Proposition and the identity
8 TT
cor(3", v) |/l|E
The claimed expression follows.
Remark The sign + appearing in Theorem rrffects which subgroup constitutes the worst group.
Specifically, since Ea +b (max( (2, w)) where = 2 > O and w (4 Acore)
A
+ sign: ATT 3 > so w > - and the unseen subgroups (y-1, dR), (041.de) are the
worst group.
sign: A
1 8
< 0, so
(y-1, de) are the = w and the seen subgroups (y+1, dR) (
worst group.
spar A > 0 in the typical regime, so the + sign (unseen groups worst) applies under ERM with spurious correlations.
By Proposition I
3* x Σ Σ-'Ā A implies AJ ATT 3 & A 15
A
Remark While the extreme correlation assumption is strong, such settings have been widely adopted in prior theoretical studies to isolate and analyze the effect spurious correlations a tractable manner (Yao et al.] 2022 Lai & Muthukumar 2024). Following this established tradi- tion, we present our analysis and worst-group error decomposition under these extreme conditions. Empirically, we demonstrate in Section 4 that SCER remains effective under less extreme cor- relation regimes, suggesting that the insights derived from this theoretical framework extend to more realistic settings.
4

---

Published as a conference paper at ICLR 2026
FROM THEORETICAL ANALYSIS TO EMBEDDING LOss
From Theorem
D the worst-group error under the Gaussian mixture model can be expressed as Euge = ( cor Asuur) Apurr / - 1/2 cor( Acore) ccorellx)
(4)
where (b-) is the standard Gaussian CDF.
Equation shows that two competing components govern the worst-group enror:
cor (3", Aspue) apur I|
and cor(s" Acore) |
spurious
core term
The first term increases the error by aligning with spurious directions, while the second decreases the error by aligning with core directions. Thus, minimizing Ewge requires simultaneously suppressing the spurious term and enhancing the core term.
Motivated by this decomposition, we define the following surrogate regularizers:
Lapur = cor(B, Aspur) A ||x;
Leare cor(3, Acore) Acore |x.
where / denotes the classifier weights. The embedding-level regularization is then given by
Lembedding Aspur fospur Acore Leare-
(5)
Remark. In Theorem [ derived Equation under the strict ERM setting with Gaussian
sumptions, where the optimal 3 has the closed form 8* E- In practice, however, / corre- sponds to the weight parameters of a deep neural classifier trained via stochastic optimization. While may not coincide with the ERM minimizer, the decomposition in Equation remains structurally valid: worst-group error is still determined by the trade-off between spurious and core alignment. This observation motivates upplying the embedding regularization loss in Equation deep models.
B EXPERIMENTAL SETUP
B.1 EXPERIMENTAL SETTING
This section provides comprehensive details of our experimental configuration for evaluating the SCER framework. We describe the datasets used for evaluation, the model architectures employed across different domains, the baseline methods used for comparison, the hyperparameter settings, and the evaluation metrics. These detailed specifications ensure reproducibility and provide com- plete context for the results presented in the main paper.
B.1.1 DATASETS
We conduct experiments on six benchmark datasets. Four image datasets, including Water- birds, CelebA, MetaShift, and ColorMNIST, and two are text datasets, including CivilComments and MultiNLI. These datasets are established benchmarks for evaluating model robustness against spurious correlations. Dataset characteristics and statistics are summarized in Table
Waterbirds Waterbirds (Sagawa et al.) is binary classification dataset constructed by superim- posing bird images from the CUB dataset (Wah et al.) (011)) onto backgrounds from the Places dataset (Zhou et al.) 2017). The task involves classifying images landbirds or waterbirds, where background type creates spurious correlations between land backgrounds and landbirds, and water backgrounds and waterbirds. We use the standard data splits from Idrissi et al. (2022).
CelebA CelebA (Liu et al. 2015) contains approximately 200,000 celebrity images. We formulate the problem as a binary classification task, classifying hair as either blond or non-blond, with gender serving as the spurious attribute. We follow the standard splits from Idrissi et al. (2022)
5

---

Published as a conference paper at ICLR 2026
Table 8: Statistics of datasets, For ColorMNIST, p denotes the spurious correlation level between label and domain attribute (color) in the training set. A higher indicates stronger spurious cor- relation, making color more predictive but misleading cue for classification. "One Subpopulation Omitted" represents extreme case in which one group combination is completely absent from the training data. Groups are defined as combinations of class labels and attribute values, and Max/Min group columns show the sample sizes of the largest and smallest groups, respectively, The disparity between these values indicates the degree of group imbalance, which tests model robustness under uneven data distribution.
Dataset
Data type
Attr.
Classes
Train
Val.
# Test
Max group
Min group
Waterbirds
Image
4,795
1,199
5,794
3,498
CelebA
Image
162,770
19,867
19,962
71.629
1,387
MetaShift
Image
2
2,276
349
874
789
196
ColorMNIST
80%
Image
30,000
10,000
20,000
12,228
3,002
90%
Image
2
30,000
10,000
20,000
13,756
1,469
95%
Imuge
30,000
10,000
20,000
14,518
731
99%
Imuge
30,000
10,000
20,000
15,111
One Subpopulation Omitted
Imuge
30,000
10,000
20,000
15,249
Two truin envs
Image
50,000
0
10,000
21,529
3,644
envl 80%)
Imuge
25,000
10,117
2,415
env2 (o 90%)
Image
35,000
11,412
1,229
CivilComments
Text
148,304
24,278
71,854
31,282
1,003
CivilComments groups)
Text
269.038
45,180
133,782
148,186
12,731
MultiNLI
Text
206, 175
82,462
123,712
67,376
1,521
MetaShift From the MetaShift benchmark (Liang & Zou 2022). ). we use the Cats vs. Dogs binary classification task, where background type serves the spurious attribute through correlations be- tween indoor backgrounds with cats and outdoor backgrounds with dogs. We adopt the unmixed version provided by the dataset authors.
ColorMNIST ColorMNIST (Arjovsky al.) 2019) is synthetic variant of MNIST where each digit assigned a color. The classification task groups digits into two categories: zero and four versus five and nine, with color serving the spurious attribute through red and green assignments. Training data exhibits strong color-class correlations, validation data balanced at 50:50, and test data reverses the correlation pattern 10:90. Following AArjovsky et al. (2019), we inject 25% label noise and evaluate under varying spurious correlation strengths of 80:20, 90:10, and 95:5 ratios to assess robustness.
To test the limits of group imbalance, we conducted additional experiments on ColorMNIST with severe distributional skew, in which one subpopulation completely omitted from training. Specifi- cally, we use 15,249 samples for Label 0 with Color 0, 3,000 samples for Label with Color 0, and 11,751 samples for Label with Color while Label 0 with Color contains zero samples. This ex- treme setting evaluates model performance when specific subpopulations are entirely missing during training, representing a challenging scenario for robust classification methods.
To test the generalizability of SCER, conducted additional experiments ColorMNIST using the environment inference methodology from Creager et al. (202T). Following the prior protocol with two environments in train, partition the dataset into 50,000 training samples and 10,000 test samples. The training set is further divided into two environments, each containing 25,000 samples that represent different spurious correlation patterns. We maintain consistency with primary perimental setup by using identical model architectures, training procedures, and evaluation metrics across all experiments.
CivilComments CivilComments (Brrkan et al. 2019) provides a binary text classification bench- mark designed to identify toxic language in online comments. Mentions of demographic groups serve spurious attributes. We use the standard WILDS splits.

---

Published as a conference paper at ICLR 2026
Table Hyperparameters and search spaces.
Model/ / Method
Hyperparameter
Default, Scarch Space
Base Models
ResNet
Learning rate
0.001, 10 Uniformm) -4, -2)
Batch size
108, 2Unfform(e,
BERT
Learning
10-3 10 10
Batch size
32, 2
Dropout
0.5, RandomChoice 0.1, 0.51)
subpopulation Robustness Methods
GroupDRO
7
0.01, 10 10
CVaRDRO
a
0.1. 10 10tniform-22.00
LIF
0.7. Uniform(0005,, 0.95)
JTT
First-stage step fraction
0.5, Uniform (0.2, 0.8)
A
10. 10
LISA
a
2. 10 (0Unffomm(-11)
Pselect
0.5, Uniform(0,
DFR
Regularization
0.1, Unifform (250
EIRep
81.
0.1. Uniform(O, 1)
Domain-Invanl Invariant Methods
IRM
X
100, 10 Uniformi -11,) 1,
Iterations of penalty annealing
500, 10 Uniforrm0
MMD
1. 10 Uniform([11 1,11)
Data Augmentation
Mixup
0.2, 10 UniformmO, 4)
Class Imbalance Methods
ReSample
Follow default implementation Follow default implementation Follow default implementation
WeWeight
Weight
Focal Loss
Y
0.5 10 Loiiformm()
CBLoss
B
0,9999, 1 - 10 U0fforrm(-0.--2 -2)
LDAM
max m
0.5, 10 Eniformm(
scale
30. RandomChoice (110, 301)
Proposed Method (SCER)
SCER
"
0.01, 10 10Mniform-33.-1)
Acore
Uniform(0, 1) Uniform(O, 1)
Aepar
CivilComments (4 groups) As additional experiment on the CivilComments (Borkan al., 2019) dataset, we use the coarse version for both training and evaluation to enable fair comparison with GroupDRO-ES (Izmailov al. 2022). which known the most competitive GroupDRO baseline and represents the sfate-of-the GroupDRO performance. In this setting, the spurious attribute is defined as the comment mentions least one the following categories, including male, female, LGBT, black, white, Christian, Muslim, or other religion. Otherwise, the spurious label is O. The presence of these eight categories is spuriously correlated with the comment being classified as toxic. This configuration forms four groups, with two classes for toxic and non-toxic comments, and two attributes representing the presence or absence the spurious attribute.
The MultiNLI dataset (Williams et al. 2018) addresses natural language inference by requiring models to classify the relationship between premise and hypothesis entailment, con- tradiction, or neutrality. The presence of negation words serves as confounding factor because they exhibit a strong correlation with contradiction labels. We adopt the data splits established in Idrissi et al. 2022).
MultiNLI
7

---

Published as a conference paper at ICLR 2026
Table 10: Summary of attribute availability settings across baseline methods.
Train Attr.
Valid/Held-out Attr.
Method
Known
Known
GroupDRO, LISA, CVaRDRO, ReSample, PDE, ElRep(with GroupDRO) JTT. LfF, CRT, SqrtReWeight
Unknown
Known (Val)
Unknown
Known (Ield-out)
DFR
Unknown
Unknown
ReWeight, ERM. Mixup, MMD, Focal Loss, CB Loss, LDAM, Balanced Softmax
B.1.2 BASELINES
For fair comparison, we followed the training protocol of Gulrajani & Lopez-Paz, performing ran- dom search over each method's joint hyperparameter space. Hyperparameter configurations follow prior work (Yang et al.) 2023b) for all baseline methods. We selected the hyperparameters that achieved the highest Worst Accuracy for each algorithm, then conducted three independent runs with different random seeds (0, 1, 2) to account for variance and ensure robustness of reported re- sults. Final average performance and standard deviation are reported across these runs. Complete hyperparameter details are provided in Table 9 which summarizes the hyperparameters used for each baseline. In addition standard optimization parameters such batch size, learning rate, and dropout, we include key weighting parameters y7, Aspar. and Acore to suppress spurious features and enhance core features. These values are chosen to encourage the model learn generalizable representations that generalize across distribution shifts.
We provide clear summary in Table indicating which baselines utilize group annotations each stage: training, held-out selection, and validation. Following the evaluation protocol established in prior work (Yang et al., 2023b), we adopt the Known Attributes setting, in which group annotations are available for both the training and validation sets. All baselines are evaluated under this uni- fied setting to ensure consistency and fair comparison. While some methods are designed to operate without group annotations during training or validation, all methods ultimately rely on group infor- mation for final model selection. Specifically, we select the best checkpoint for each method based on worst-group accuracy, ensuring alignment in model selection across all approaches. Thus, even when a method does not internally use group information during optimization, group annotations consistently used for model selection to ensure fair and comparable evaluation.
To comprchensively evaluate our method's robustness, we compare its performance against a diverse set of baselines spanning standard ERM, subpopulation robustness methods, conain-invariant ap- proaches, data augmentation techniques, and class imbalance methods. The primary baseline results appear in Tables
3 and 5 in the main text, with complete results for all baselines provided in Tables I1 12 13 and 5 Our baseline selection follows evaluation criteria established in recent stud-
1,
ies (Deng et al. 2023
Wen al., 2025), with particular attention to whether each method utilizes group annotations during training.
To ensure fair comparison, the main text reports only methods that rely on group annotations throughout the entire training and validation pipeline, while methods not satisfying this criterion are presented separately in the appendix to reflect differences in evaluation settings.
Empirical Risk Minimization ERM minimizes the average loss over all training samples without explicitly addressing group-specific performance disparities.
subpopulation Robustness Methods GroupDRO (Saaaa et al.) is a representative approach to distributional robustness that improves worst-group performance by assigning greater importance to groups with higher losses. CVaRDRO (Duchi & Namkoong 202T] extends this approach to the individual sample level, enabling more fine- ane-grained adjustment by assigning importance weights to individual samples. LiF (Nam et al., 2020) employs a co-training approach that simultancously trains a biased model and a debiased model, guiding the model to learn by distinguishing between biased and unbiased features. JTT (Liu et al.) 2021) is a two-stage strategy that identifies complex samples using information obtained from initial standard empirical risk minimization (ERM) training, then improves model robustness by increasing the weights of these samples in the second training stage. LISA Yao et al.] 2022) utilizes data augmentation techniques that mix data within and across at- tributes, encouraging the model to learn invariant features of attributes. DFR (Kirichenko et al.)) is an efficient method that improves robustness by retraining only the last layer using a balanced vali- dation set, rather than the entire model. PDE (Deng et al. 2023) employs strategy of progressively
8

---

Published as a conference paper at ICLR 2026
Table 11: Complete performance comparison on multiple datasets, each exhibiting different levels of spurious correlations. denotes the performance reported from Yang et al. (2023b) except for Col- orMNIST. denotes the performance reported from Deng et al. (2023) for Waterbird and CelebA. if denotes the performance reported from Wen et al. (2025) for Waterbird and CelebA. For the re- maining datasets, the detailed experimental settings follow Yang al. (2023)), as described in Ap- pendix B.I.2 SCER achieves the highest worst-group accuracy in Waterbirds, CelebA, MetaShift, and ColorMNIST, demonstrating its robustness in subpopulation shift scenarios.
Algorithm
Waterbirds
CelebA
MetaShift
ColorMNIST
(p 80%)
Avg Ace
Worst Ace
Avg Ace
Worst Acc
Avg Ace
Worst Ace
Avg Acc
Worst Ace
ERM
84.1 +1.7
69.1 +4.7
95.1 +0.2
62.6 +.5
91.3 +003
82.6 +0.4
38.2 +2.4
30.9 +2.7
Mixupt
89.5 10.4
78.2 =0.4
95.4 =0.1
57.8
91.6 90.3
81.0 10.8
41.2 1140
12.4 12.0
GroupDRO
88.8 +1.8
78.6
+0.6
40.7
91.0
45.4
73.5 40.3
73.1 10.1
IRM
88.4
74.5
94.7 +0.8
63.0
91.8
83.0
51.6
25.0
CVaRDRO
89.8 100.4
75.5
90.1
64.1 428
92.1
84.6
45.9
13.5
JIT?
+0.66
72.0 +0.
90.4
70.0 10.22
91.2
-3.6
69.1 +1.9
44.2
LFF!
87.0
75.2
81.1 #5.6
$3.0 +4.3
80.2
73.1 =1.6
74.0 44.1
4.1 42.33
LISA!
92.8 +0022
88.7 20.6
92.6
86.3 +1.2
80.4
84.1 1004
73.8 40.1
73,2 #0.1
MMD
93.0++0.1
83.9 +1.4
+-.7
24.4 +2.0
89.4 +0.1
85.9 +-0.7
41.7 +007
27.5 27.5+ +7.0
ReSample'
89.9
77.7 -12
92.0
87.4 10.8
91.2
40.4
73.7 40.1
72.2 40.3
ReWeight
91.8
86.9 00.7
90,9
89.7 10.2
91.7 10.4
85.6 40.4
74.1 40.1
7344 10.1
Seighe
88.3
78.6 0..1
93.6
82.4 +0.5
91.5 00.2
84.6 84.6
+0.7
66.7 +2.0
CBLoss
91.3
=0.3
90.7
89.4 10.7
91.7
80.4
73.6 40.2
72.8 72.8 10.2
Focal
89.3
94,9
59.1
91.7
81.5
40.7 44466
31.2
LDAM
87.3
71.
90.5
$9.6
90.1
83.6
33.3
26,9 #6.1
BSoftmax
88.4
74.1 =0.9
91.9
80.5
91.6
83.1 10.7
41.8
33.3 +2.5
DHRT
92.3++0.2 +0.2
91.0+05
91.9
90.4 +0.1
+0.4
85.4 +0.4
68.4
67.3 +0.2
PDE
92.4 +0.8
90.3 +03
92.4 40.8
91.0
87.4 +0.1
78.1 4.011
40.88 40.8
72,9 20.3
EIReptt
92.9 +0.7
88.8 0.7
92.8 +0.2
91.0
85.9
72.1 + 2.5
50.3 0.6
46.5
SCER
92.1++02
91.2 0.2
92.7 +0.2
91.4 +0.1
91.6 +0.3
86.7 +00.8
74.1 +0.1
73.6 +0.2
expanding training data, guiding the model to first focus on core features and subsequently learn spu- rious features. Finally, EIRep (Wen et al. 2025) introduces both Nuclear- and Frobenius-nome regularization to representation vectors, enabling robust group generalization representation learn- ing against spurious correlations by simultaneously emphasizing important features and maintaining diversity.
Invarian Methods IRM (Arjovsky et al. 2019) method for finding invariant predic- tors that work commonly across multiple domains. Specifically, it trains feature extractors so that fixed predictor becomes optimal in each domain, enabling the model to capture important charac- teristics effectively regardless of domain changes. This approach ensures robust performance even when domains change. MMD (Li et al., 2018) is a method that numerically measures the differ- ence in feature distributions across domains and trains to reduce this difference, thereby mitigating distribution mismatches. other words, it is an approach that aims to improve generalization per- formance by making feature representations across domains similar as possible.
Data Augmentation Mixup (Zhang et al. 2018) is technique that generates synthetic training data by linearly interpolating two randomly sampled pairs of inputs and their labels. This smooths decision boundaries and serves as regularization to improve generalization performance and pre- vent overfitting. It is a simple yet effective data augmentation method that artificially expands the diversity of the training data by learning intermediate points between samples in the data space.
Class Imbalance Methods ReSample (Japkowicz] 2000) mitigates class imbalance in training
data through resampling techniques. Re Weight (Japkowicz) 2000) assigns weights to the loss func- tion proportional to class frequency. SQRTWeight is another loss adjustment method that adjusts loss weights using the inverse square root of class frequencies, thereby increasing the influence of minority classes in imbalanced data. Focal Loss (Lin et al. 2017) focuses on difficult samples by reducing the contribution of easy-to-learn samples. CB Loss (Cui et al. 2019) calculates sample weights based effective class size. LDAM (Cao et al.) 2019) assigns distinct margins to each class to balance the decision boundaries. Balanced Softmax (Ren et al. 2020) modifies the softmax function itself to accommodate class imbalance.

---

Published as a conference paper at ICLR 2026
Table 12: Complete performance comparison on ColorMNIST under increasing spurious correlation levels. SCER achieves the highest worst-group accuracy across different spurious correlation levels, demonstrating robustness to spurious feature reliance in ColorMNIST.
ColorMNIST
80%
P
90%
95%
P
90%
Algorithm
Avg Ace
Worst Acc
Avg Ace
Worst Ace
Avg Acc
Worst Acc
Avg Ace
Worst Ace
ERM
38.2 +2.4
30.9 #2.7
+7.4
7.8 +3.4
41.1 +13.0
10.1 +4,55
50.7 +8.2
8.5 =5.0
GroupDRO
73.5 +0.3
73.1 +0.1
73.5 +0.0
72.7 +0.3
72.3 +0.1
70.7 +0.2
50.3 +3.9
38.7 +0.8
LISA
73.8 10.11
73.2 10.1
73.3 10.1
72.9 10.2
72.6 10.4
71.4 1006
53.1 17.2
10.2 18.1
ReSami
73.7
72.2
72.6
71.6
71.8 =0.1
70.7 =0.5
57.1
45.4 =10.0
Re Weightt
74.1 +00.1
73.4
73.6
72.8 +0.2
72.4 +0.5
70.7
CBLoss
73.6
72.8 +0222
72.7 +.0.2
71.7 10.1
71.8 10.3
70.8 +:03
PDE
76.6
72.9 10.3
72.610.2 102
70.2 10.7
73.4 10.4
70.0 10.3
56.3 14.3
47.5 18.9
SCER
20.1
73.6
10.2
73.6 10.1
73.0 20.1
73.5 10.3
72.8 10.3
61.0 #2.0
56.0 12.2
C ADDITIONAL EXPERIMENT
This section presents supplementary experimental results that further validate the effectiveness and robustness of the SCER framework.
C.I QUANTITATIVE ANALYSIS
Here, we present the complete results for quantitative analyses omitted from the main text.
Complete performance on Col- orMNIST with one group absent. SCER achieves the highest worst-group accuracy.
Table 13:
Image Dataset Table II reports the complete re- sults for all baselines image datasets. Across this comprehensive set of methods, SCER consistently achieves the highest worst-group accuracy, demon- strating its superior robustness compared to all com- peting approaches.
ColorMNIST (One Subpopulation Omitted)
Algorithm
Avg Ace
Worst Ace
ERM
52.9 18.4
9.7 +5.4
Image Dataset with Stronger Spurious Correla- tions Table [2 reports the complete results on Col- orMNIST under varying spurious correlation levels, including all methods achieving worst-group accu- racy above 70% in Table [1] SCER achieves the highest worst-group accuracy across most settings, confirming its robustness under subpopulation shift.
GroupDRO
53.6 +22.5
44.1 +6.2
LISA
49.5 +2.4
13.5
ReSample
49.1 +0.88
16.3 +13.0
ReWeight
52.1 +4.7
10.0
CBLoss
53.0
13.3 10000
PDE
1126
8.3+13.9
SCER
65.3 +1.0
59.6 +1.0
In addition to the standard evaluation settings in Table 12
we examine a more challenging scenario in which minority group is completely absent during training. Table 13 the complete results under this extreme setting, As shown, SCER achieves the highest worst-group accuracy, significantly outperforming all competing methods. Notably, class imbalance methods such ReWeight (Jap-] kowicz 2000) and CB Loss (Cui et al. 2019) exhibit substantial performance degradation when group entirely missing, similar to other approaches. In contrast, SCER maintains robust per- formance even under this extreme distributional shift, demonstrating its effectiveness in handling severe group imbalance.
Table 14: Performance compari- without explicit bias labels in noisy environments.
Integration with Environment Inference Methods. Follow- ing Table 4 we additionally experimented to verify whether SCER maintains robustness when the environment partitioning accuracy is imperfect, at approximately 70%. All other settings remain identical those in Table and the results are presented in Table Even under reduced environment partitioning accu- racy, EIIL+ SCER achieves an average accuracy 65.1 out- performing both EIIL + DRO at 54.6% and EIIL with IRM 53.7%. This suggests that SCER operates effectively even under imperfect environment partitioning conditions.
ColorMNIST (Two train envs)
Method
Avg Acc
IRM
54.8 18.3
EIIL (IRM)
53.7 +2.1
EIIL DRO
54.6 +5,9
BIIL SCER
65.1 +2.8
10

---

Published as a conference paper at ICLR 2026
Table 15: Complete performance comparison on text datasets. from Deng al. (0223) for Civilcomments. if from Wen et al. (0255 for Civilcomments. SCER achieves the best worst-grous accuracy across multi-class and do- main settings.
Text Dataset Table
15 presents the
complete performance comparison on text datasets, including CivilComments and MultiNLI. SCER consistently achieves superior worst-group accuracy across these benchmarks, demonstrat- ing its effectiveness beyond image domains. On CivilComments, SCER at- tains a worst-group accuracy of 74.0% Similarly, on MultiNLI, SCER achieves the best worst-group accuracy 76.8%.
Algorithm
CivilComments Worst Ace
MultiNLI
Avg Ace
Avg Acc
Worst Ace
ERM
85.4 +0.2
63.7 +1,1
80.9 #.11
66.8 +0.55
Mixup
84.9.10.
66.1 11.3
81.4
68.5 10.6
GroupDRO
10.6
70.6 11.2
81.1 10.3
76.0 10.7
IRM
85.0
63.2 40.8
77.8 10.6
63.3
CVaRDROI
83.5
68.7 +1.
70.1
61.5
JIT
83.3
64.3
80.9
69.1 #0.1
and 15 demonstrate that SCER consistently achieves supe- rior performance across diverse bench- mark datasets and in comparison with various state-of-the-art algorithms. No- tably, SCER achieves robust perfor- mance not only on vision tasks but also on natural language understanding tasks, confirming that our mechanism of separating spurious and core features at the embedding level generalizes effec- tively across domains.
Tables
LIFI
65.6
51.0
71.7..11
63.6 12.9
LISA
82.7
73.7 40.
80.4
13.3
MMD
84.6
54.5 11.
78.8
69.11.5
ReSample!
82.2
73.3 40.5
77.2
72.3 10.8
ReWeight
82.5
72.5 =0.0
8.0
68.8 #0.4
Seighee
83.3
71.7 =0.4
80.7
60.7
CBLoss
82.9
73.3
80.6
72.2 +0.3
Focalt
85.5
62.0
80.7
69.4 +0.7
LDAM
81.9+222
37.4
80.7 +0.3
+1.6
BSoftmax
83.8
+0.4
80.9 +0011
66.9
DFRt
83.0
102
81.7 10.0
68.5
PDD-
86.3
10.5
69.1 10.3
65.8
BIRepft SCER
79.0
10.5
66.8
81.7 10.4
74.0
80.4
76.8 10.55
Table 16: Performance comparison on Waterbirds, CelebA, MultiNLI, and CivilComments (4 groups) with GroUpDro-ES. from Izmailov al. (2022). Experimental and dataset settings also follow Izmailov et al. (2022). SCER achieves the best worst-group accuracy across all datasets.
Waterbirds Worst Acc
CelebA Worst Acc
MultiNLI Worst Ace
CimmlComments (4 groups)
Algorithm
Worst Acc
GroupDRO-EST
90.7 + 0.6 91.7 + 0.1
90.6 + 1.6
73.5
80.4
SCER
91.9 + 0.3
77.9
82.9
Deep Analysis with GroupDRO SCER proposes novel objective function that decomposes worst-group error into spurious and core components, then directly regularizes the embedding space to suppress spurious correlations while enhancing core features. To validate the effectiveness of this approach, we conducted an in-depth comparison with GroupDRO. Since early stopping is well known be essential for achieving optimal GroupDRO performance (Izmailov et al. (022). compared SCER with GroupDRO-ESS (Izmailov et al. 2022). which serves as the most competitive GroupDRO baseline. Note that for fair and direct comparison with GroupDRO-ESS we adopted the experimental setup from Izmailov et al. (2022) and used their reported results for GroupDRO-ES, which differ from the configuration in our main experiments following Yang et al. (2023)).
As shown in Table 16 SCER achieves consistent performance improvements over GroupDRO-ES across all datasets. Specifically, SCER demonstrates gains of 1.0% on Waterbirds and 1.3% on CelebA, with particularly notable 4.4% improvement on MultiNLI. This represents meaningful progress, given that most group-robust methods struggle to exceed worst-group accuracy in the low 70% range on this dataset. For CivilComments while Tables 5 and [5d follow the 16 group setting with two classes and eight attributes as in prior work (Yang et al. (2033)), we also conducted ex- periments using the 4 group setting with two classes and two attributes in Table [6] to allow fair comparison with (Izmailov et al. 2022), achieving a 2.5% improvement. Detailed information about the dataset configurations is provided Table g
Beyond quantitative gains, we provide qualitative evidence that SCER fundamentally restructures the embedding space. Figure shows through t-SNE analysis that while ERM and GroupDRO sep- arate clusters by background, SCER achieves label-based clustering independent of spurious fea- tures, confirming domain-invariant representations. Figure further demonstrates via Grad-CAM

---

Published as a conference paper at ICLR 2026
that SCER attends target objects rather than backgrounds, showing that our embedding regular- ization yields genuine improvements in representation learning.
We also conduct comprehensive quantitative analysis evaluate the computational complexity and scalability of the SCER framework. These experiments provide detailed insights into SCER's s adaptation mechanisms across various scenarios and assess its practical feasibility for real-world deployment.
Table 17: Average per-iteration train- ing time (seconds).
Complexity analysis While our method requires com- putational investment for correlation metric and mean embedding computations, it maintains computational costs comparable to established robust training methods. As shown in Table [7 our approach requires 0.0988 seconds per iteration, which is marginally higher than GroupDRO at 0.0915 seconds but faster than the two- stage DFR method 0.1025 seconds.
Method
Avg One Iteration (s)
ERM
0,0654
GroupDRO
0.0915
DFR
0.152
SCER
0.0988
Importantly. DFR's computational overhead stems from its inherent two-stage paradigm (Kirichenko) ct al.): first training with ERM to learn representations, then retraining only the final layer with group-balanced data. This approach suffers from critical limitations, including the requirement for high-quality, balanced validation data and complete failure when ERM cannot capture core features, such when minority groups are entirely absent from training data. In contrast, our single-stage approach achieves substantially better worst-group accuracy than these alternatives while avoiding DFR's structural constraints. Most importantly, method requires no balanced validation data or multistage training, delivering superior robustness with greater practical simplicity.
Table 18: Sensitivity analysis of embedding regularization on CelebA. Worst-group accuracy re- mains relatively stable across settings, with the best results observed when moderate Acore is applied.
CelebA
both À 0
Acone (fixed Aspur)
Aapur (fixed Awore)
Sctting
Avg Acc
Worst Acc
Accre
Avg Acc
Worst Acc
Agur
Avg Acc
Worst Acc
91.4 10.6
89.0 +0.77
0.0
92.5 10.1
91.1 10.2
0.0
92.0 10.3
91.0 :0.2
0.5
92.7 #0.2
91.4 #0.1
0.5
91.7 #066
90.9 #0.7
1.0
92.7
10.2
90.9 10.5
1.0
92.0 10.1
90.7 10.1
C.2 QUALITATIVE ANALYSIS
We provide comprehensive quantitative evaluations, including additional sensitivity analysis across hyperparameters.
Embedding Regularization Analysis addition to the results Table 6 we conduct sensitivity analysis for Acure and Àupur the large-scale real-world dataset CelebA. The results confirm the generalizability of our ColorMNIST findings beyond synthetic settings. CelebA exhibits even more pronounced performance improvements with appropriate hyperparameter values, highlighting the effectiveness of embedding regularization in mitigating spurious correlations in realistic scenarios, as shown in Table
D STATEMENT ON THE USE OF LARGE LANGUAGE MODELS
Large Language Models (LLMs) were used in limited capacity during the preparation of this research. Specifically, LLMs were used to check grammar and refine sentence structure after the initial draft was completed, primarily to correct awkward expressions and maintain consistency in writing style. However, all core research ideas, analytical methodologies, interpretations of the results, and conclusions were developed entirely by the authors. The LLM did not contribute to any creative content or academic judgments. This use of LLMs was conducted within limits that do not compromise the originality or academic integrity the research.
12

---

Published as a conference paper at ICLR 2026
Original
GrugDRO
SCER
Original
EBM
GruuDRO
SCER
Orderal
EXN
GroUDERO
SCBR
Figure 6: Comparison of Grad-CAM visualizations the Waterbirds dataset. SCER directs atten- tion to meaningful features, reducing focus on spurious regions.
13


Reviews:
[{"review_id": "rUI9WrZFDi", "reviewer": "Reviewer_mZvF", "summary": "This paper proposes Spurious Correlation-Aware Embedding Regularization (SCER) as a method to improve on worst-group robustness by proposing embedding-level regularization. The authors provide theoretical justification for decomposing worst-group error into spurious and core components. The method identifies spurious directions as differences in embeddings across domains within the same class, while core directions are differences across classes within the same domain and combines them into the SCER objective that penalize classifier alignment with the spurious direction while encouraging alignment with the core direction. Experiments on vision and language datasets show improvements over existing methods, particularly in extreme settings where a subpopulation is completely missing during training.", "questions": "Would it be possible to discuss how extending the error decomposition theorem beyond the two-class/groups assumption can be justified for the sake of generality?\n\nHow does SCER behave in terms of robustness and performance when group labels are noisy, for instance, when inferred groups are partially misaligned to 70-80% agreement instead of 95%? \n\nHow would you handle when multiple spurious or core directions exist? Can you provide evidence that consolidating into a single global direction is optimal?\n\nHow many training steps does SCER typically require compared to baselines? The per iteration time is provided, but not the full training cost", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The paper precisely defines core and spurious mean-embedding differences, aggregates them into global directions, and integrates their alignment with classifier weights into a composite loss term that is transparent and straightforward to implement.\n \nThe authors conduct comprehensive experiments 6 datasets across vision and language domains, comparing against 19 baseline methods. The settings also cover different spurious correlation strengths and extreme scenarios.\n\nNotably, the method substantially outperforms baselines when one subpopulation is entirely absent during training on ColorMNIST.\nThe authors conduct sensitivity and component analyses to support specific design choices and validate their theoretical decomposition.", "weaknesses": "When formulating the theorem for error decomposition, the paper is restricted to binary labels and two domains and assumes constant core and spurious differences between domains and classes. The paper does not discuss how violations of these assumptions affect the method's validity, particularly for multi-class problems, which appear in experiments like MultiNLI with 3 classes. \n\nIt can be argued that correctly specified group labels are required to operationalize SCER in terms of its component losses. While this is briefly explored with EIIL in a two-environment setup, it would be important to understand how SCER behaves when group labels are noisy. For example, how does robustness vary when EIIL’s inferred groups are partially misaligned to 70-80% agreement instead of 95%?\n\nThe paper computes group-wise mean embeddings but doesn't discuss key considerations, such as how many samples are needed for stable estimates, what happens when group sizes are highly imbalanced, and what happens if spurious and core features are not well-separated in embedding space.\n\nFurthermore, aggregating across all classes or groups into a single spurious and single core direction may lead to underfitting in settings with multiple heterogeneous spurious factors. A discussion and validation of how this is handled is missing.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "DgWaltNmfx", "reviewer": "Reviewer_NQJE", "summary": "The paper proposes a new method to improve worst-group robustness by encouraging models to rely on core features while suppressing spurious correlations in the embedding space. It introduces a regularization term that rewards alignment with class-discriminative directions and penalizes reliance on non-causal relations. Experiments on vision and language benchmarks show consistent gains in worst-group accuracy over existing methods.", "questions": "What do you think are the potential reasons for ElRep’s significant performance drop on MetaShift and ColorMNIST?", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "- The method is concisely proposed with theoretical analysis\n- The results are clearly explained with interpretable evidence", "weaknesses": "- The core idea is to encourage models to rely less on spurious correlations and more on invariant features. This has been explored in a substantial body of prior work both empirically and theoretically. The contribution here, while sound, does not clearly break new conceptual ground, considering the improvements over baselines appear modest.\n- I believe this work, as well as similar ones, should be applied to datasets with a larger number of groups. For example, where the number of groups is greater than 50, the spurious correlations are not very obvious to humans under the given group divisions.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "EwAkA51ktB", "reviewer": "Reviewer_4dbm", "summary": "This paper proposes a novel method for mitigating spurious correlations by developing clear constraints on the embeddings learned from different subpopulation groups. The core feature is calculated as the average difference of embeddings from samples with the same domain but different labels, while the spurious feature considers the difference for samples with the same label but different domains. Next, the alignment between classifier weights and the directions of the normalized features is calculated and used as an additional loss (constraint) alongside the common worst-group classification loss. Experiments are conducted on popular datasets for studying spurious correlations, and the proposed method is compared with many baseline methods. The results demonstrate that the proposed method works as designed and is very promising.", "questions": "(Q1) The authors are encouraged to respond to (W1) I described above. Additional simple experimental results in the appendix would be much appreciated and could have a big impact on my final opinion, such as figures of t-SNE plots labeling different subpopulations, especially in comparison to baseline methods that also operate in the embedding space and focus on invariant features. \n\n(Q2) (Minor) In Figure 1, instead of using mathematical notations for the proposed core/spurious features, plain language descriptions might improve readability. Personally, I had a difficult time understanding that part before reading the Methods section.", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "(S1) The proposed constraints on the embeddings are explicit and look novel to me, especially the correlation terms defined in Eq. (1), which are well supported by Theorem 1. I believe the research directions enlightened by this paper are very promising. Besides, the Introduction and Related Works sections also provide a good discussion about this paper's contributions, which places it well among the literature on spurious correlations. \n\n(S2) Comprehensive experimental settings provide strong evaluations of the performance of the proposed methods, although some improvements are marginal in Table 1. In general, I believe the evidence is enough to demonstrate that the proposed method is very promising. In particular, I really appreciate the results in Figure 3, which demonstrate that the proposed constraints on the correlation terms are working and work as designed. \n\n(S3) In general, this paper is well written and easy to follow.", "weaknesses": "(W1) While a schematic diagram/representation of the embeddings learned from the proposed methods is presented in Figure 1, a comprehensive evaluation of the structure and clusters of the learned embeddings, in comparison to baseline methods, is not included in this paper, which weakens the evaluation of a key contribution of this paper: the explicit constraints on the embeddings. In particular, it would be very helpful for readers to examine the differences between the embeddings learned by the proposed methods and those learned by works focusing on learning invariant features.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "9hlFH5lxIU", "reviewer": "Reviewer_vc9T", "summary": "This paper introduces a regularization scheme for GroupDRO, called SCER, which minimizes intra-class inter-domain alignment and maximizes inter-class intra-domain alignment in the embedding space. Theoretical results are presented which justify the worst-group error decomposition into these two alignment terms, and experiments are performed on a variety of vision and language benchmarks.", "questions": "1. How do the embedding space dynamics and the performance of SCER change under optimal early-stopping, and how does it compare to standard early-stopped GroupDRO?\n\n2. Is there any connection between the embedding alignment terms proposed by this paper and recent studies on neural collapse and group robustness [1, 2]? Specifically, the neural collapse (NC1) phenomenon suggests that the proposed $\\Delta_{spur}$ metric is naturally minimized during training as embeddings collapse to the class means. I’m also curious whether SCER has any relation to the “group mean alignment loss” of [2].\n\n3. There are several grammatical errors in the paper:\n\n    a. Line 24\n\n    b. Figure 1 lines 64-65\n\n    c. Line 76\n\n    d. Figure 2 lines 167, 171\n\n    e. Section 5.1 and 5.2 have repeated titles\n\n    f. Equation references are malformed in the Appendix\n\n[1] Papyan et al. Prevalence of neural collapse during the terminal phase of deep learning training. PNAS 2020.\n\n[2] Lu et al. Neural Collapse Inspired Debiased Representation Learning for Min-max Fairness. KDD 2024.", "limitations": "", "rating": 2, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 1, "strengths": "1. Embedding-space regularization is an under-explored technique for group robustness. More generally, the community’s understanding of feature learning in the presence of spurious correlations is incomplete -- so methods which probe embedding space phenomena to design robust algorithms are particularly welcome.\n\n2. The quantitative analyses and ablation studies are well-done, particularly under omission of one subpopulation and integration with the EIIL algorithm. The quantity and diversity of benchmark datasets is also a strength of the paper.", "weaknesses": "1. The comparison to previous methods (across all tables in the paper) is improperly organized and obscures the true performance differential of the proposed SCER method.\n\n    a. The number of previous methods in Table 1 is somewhat excessive: as this is not a benchmarking paper, there is no need to compare to 19 previous methods, especially older algorithms like IRM whose performance has been broadly superseded (a citation is sufficient).\n\n    b. Many of the compared methods utilize different data and annotation assumptions than SCER, and therefore do not represent fair comparisons. Specifically, SCER requires full group annotations on the training dataset since it is an extension of GroupDRO. Comparisons to methods like ReWeight (which does not use group annotations at all) and DFR (which uses group annotations only on a small held-out set) inflate the perceived performance of SCER relative to previous work. Overall, these methods should not be included in the comparison and/or whether group annotations are utilized in the training set, held-out set, or validation set should be clearly denoted.\n\n2. Since SCER is essentially a proposed regularizer for GroupDRO, the primary comparison should be to GroupDRO. However, the paper lacks evidence that SCER represents a substantial improvement over GroupDRO.\n\n    a. The comparison with this paper’s implementation of GroupDRO is somewhat obfuscated. It should be made clear that SCER with both regularization hyperparameters to zero reduces to GroupDRO, and the numbers from Table 6 and Table 11 included as such in Table 1. This is important because implementation details can vary performance: for example, the GroupDRO WGA in Table 1 for ColorMNIST is 73.1%, but in Table 6 one can see it decreases to 70.7% in this paper’s implementation.\n\n    b. The paper [1] which this paper draws GroupDRO numbers from does not represent the most competitive GroupDRO baseline. In particular, it is well-known that GroupDRO requires early-stopping to achieve the best performance [2, 3]. Compared to the early-stopped GroupDRO numbers of [3], the proposed SCER method barely improves performance. On Waterbirds, SCER improves from 90.7% to 91.2% (Table 1). On CelebA, SCER improves from 90.6% to 91.4% (Table 1). On CivilComments, SCER worsens from 80.4% to 74.0% (Table 5). On MultiNLI, SCER improves from 73.5% to 76.8% (Table 5). Keeping in mind that SCER tunes two additional hyperparameters compared to GroupDRO, or one additional hyperparameter including tuning the early-stop threshold, its improvement over GroupDRO is not very significant.\n\n    c. This is less important, but the Grad-CAM visualization in Figure 4 should also compare to GroupDRO. It is possible that the alignment of the gradient magnitude and the core feature is a property of GroupDRO and not of the SCER regularization scheme.\n\n3. Justification of theoretical assumptions:\n\n    a. In Theorem 1, it is assumed that the _data_ covariance matrices $\\Sigma^{d_R}=\\Sigma^{d_G}=\\Sigma$, but in the SCER algorithm, the _feature_ covariance matrices are utilized to compute the Mahalanobis norm (lines 203-204). Since neural networks are nonlinear models, the feature covariances may not be equal even if the data covariance matrices are. This observation has been confirmed empirically in the literature, e.g., [4] observed a greater spectral imbalance in the empirical covariance matrix of the majority group features compared to the minority group.\n\n    b. The extreme/complete correlation assumption that $\\mathbb{P}(y,d)\\approx 0$ for some groups $(y,d)$ is fine (as theory papers often decay the number of minority group data with dimension, e.g., [5]), but it should be justified more carefully. In particular, none of the datasets except ColorMNIST with an omitted subpopulation (Table 3) exhibit an extreme correlation.\n\n[1] Yang et al. Change is hard: a closer look at subpopulation shift. ICML 2023.\n\n[2] Sagawa et al. Distributionally Robust Neural Networks for Group Shifts: On the Importance of Regularization for Worst-Case Generalization. ICLR 2020.\n\n[3] Izmailov et al. On Feature Learning in the Presence of Spurious Correlations. NeurIPS 2022.\n\n[4] LaBonte et al. The Group Robustness is in the Details: Revisiting Finetuning under Spurious Correlations. NeurIPS 2024.\n\n[5] Lai and Muthukumar. Sharp analysis of out-of-distribution error for \"importance-weighted\" estimators in the overparameterized regime. ISIT 2024.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)