Out of Spuriousity: Improving Robustness to Spurious Correlations without Group Annotations
Authors: Phuong Quynh Le
Jörg Schlötterer
Christin Seifert
Venue: Accepted by TMLR ()

Abstract: Machine learning models are known to learn spurious correlations, i.e., features that have strong correlations with class labels but no causal relationship. Relying on these correlations leads to poor performance in data groups that do not contain these correlations, and poor generalization. Approaches to mitigate spurious correlations either rely on the availability of group annotations or require access to different model checkpoints to approximate these group annotations. We propose PruSC, a method for extracting a spurious-free subnetwork from a dense network. PruSC does not require prior knowledge of the spurious correlations  and is able to mitigate the effect of multiple spurious attributes. Specifically, we observe that ERM training leads to clusters in representation space that are induced by spurious correlations. We then define a supervised contrastive loss to extract a subnetwork that distorts such clusters, forcing the model to learn only class-specific clusters, rather than attribute-class specific clusters. Our method outperforms all annotation-free methods, achieves  worst-group accuracy competitive with methods that require annotations and can mitigate the effect of multiple spurious correlations. Our results show that in a fully trained dense network, there exists a subnetwork that uses only invariant features in classification tasks, thereby eliminating the influence of spurious features.

Published in Transactions on Machine Learning Research (03/2025)
Out of Spuriousity: Improving Robustness to Spurious Corre- lations without Group Annotations
Phuong Quynh Le Marburg University
Jörg Schlötterer Marbury University & University of Mannheim
jerry-chhuctterrrkkkkknaaaaraaa
Christin Seifert
cbristh.eefferfkhaiimmrrry de
Marburg University
Reviewed on OpenReview:
https: // openreview. net/ forum? id= EFeV
Abstract
Machine learning models are known to learn spurious correlations, i..., features that have strong correlations with class labels but no causal relationship. Relying these correlations leads to poor performance in data groups that do not contain these correlations, and poor generalization. Approaches to mitigate spurious correlations either rely on the availability of group annotations or require access to different model checkpoints to approximate these group annotations. We propose PRUSC, a method for extracting spurious-free subnetwork from dense network. PRUSC does not require prior knowledge of the spurious correlations and is able to mitigate the effect of multiple spurious attributes. Specifically, observe that Empirical Risk Minimization (ERM) training leads to clusters in representation space that are induced by spurious correlations. We then define supervised contrastive loss to extract subnetwork that distorts such clusters, forcing the model to learn only class-specific clusters, rather than attribute-class specific clusters. Our method outperforms all annotation free methods, achieves worst-group accuracy competitive with methods that require annotations and can mitigate the effect of multiple spurious correlations, Our results show that in fully trained dense network, there exists subnetwork that uses only invariant features in classification tasks, thereby eliminating the influence of spurious features.
1 Introduction
Deep neural networks tend to learn spurious correlations or shortcuts, i.e., misleading heuristics that work for the majority of training samples, but do not always hold (Sagawa et al. 2020a). Spurious attributes or Teatures are features that have no causal relationship with class labels. The existence spurious features data set itself not harmful, as long as a model does not rely on them for the classification. However, spurious fcatures often casy to learn, and if there is strong correlation with class labels, i.c., spurious correlation, models tend to rely these spurious correlations shortcuts, because learning these correlations is sufficient, to minimize the overall loss (simplicity bias) (Shah et al. 2020). This means, that in the presence of spurious correlations, deep neural networks under-learn invariant information and then lose their ability to generalize and perform poorly on data that does not contain the spurious correlations ! Spurious features occur naturally, and some of them, such as image background, can be easily detected, while others, such as
Shortcct learning also features and class labels. Ideally, aim for robust models that rely invariant features, L.e., they do rely on particular distribution of the training data.
(subscts of) non-spurious fcatures that do not fully describe the relationship between

---

Published in Transactions on Machine Learning Research (03/2025)
texture [Geirhos et al. 2018], superficial statistics (Wang et al. 2019 frequency bias (Wang et al. 2022 2023); require more effort. An example of natural spurious correlation of the background is the cous and camels classification task, where the background desert is spuriously correlated with the class camels and grass is positively correlated with cows (Beery et al. 2018). These features seem to be casy to learn for deep networks, but relying on them makes nelworks perform poorly in minorily groups such as camels on grass.
To improve the performance for minority groups, mitigation strategies use group label information, obtained either by manual labeling or by approximation using misclassification information. fround-truth group labels have been used during training to minimize the worst-group loss (Sagawa et al. 2020a) or to retrain only the last layer (Kirichenko et al.) 2023). However, obtaining group labels either requires prior knowledge of the spurious attributes and a time-consuming annotation effort, or may be impossible for certain types of attributes. For example, neural networks tend to rely on low frequency components for regression tasks (Rahaman et al. 2019). Wang et al. (2022 2023) showed that classification models also rely on frequency (both texture-basod and shape-basod). However, frequency attributes are only implicit and not visible to the human eye, unlike explicit spurious attributes, such background (e.g., grass or desert). A common proxy estimate for missing group labels is to use misclassified examples from the training data (i.e., hard cases), based the assumption that models that are influenced by spurions correlations fail to classify instances where the spurions correlation
is absent (Liu et al., 2015 Nam et al.) 2020 Zhang et al. 2022 Park et al. 2023). However, this assumption is not guaranteed to hold. Furthermore, in the case of zero training loss, where the model fits the training data perfectly, hard cases are not available. Careful control of early stopping is then required to obtain a sufficient number of hard cases without degrading the Empirical Risk Minimization (ERM) training. Consequently, these methods also require access to full training information and are not applicable to fully trained models.
In this paper, we propose Pruning Spurious Correlations (PRUSC), method for extracting spurious-free subnetwork from a fully trained dense network that requires no prior knowledge of the spurions correlations, i... no ground-truth group annotations, PRUSC defines potential spurious groups based on the representation of instances in latent space. Basod on the obsorvation that instances with the same spurious attributes lic close logether in representation space, we can identily groups of instances with the same spurious feature. We apply clustering to the representations of the penultimate layer and introduce a novel application of supervised contrastive loss to train subnetwork such that clusters (likely) induced by spurious attributes are distorted. By distorting the clusters induced by the spurious feature, the subnetwork cannot rely on learning and grouping samples based on similar spurious attributes and is, therefore less influenced by spurious correlations. In summary, the contributions of this paper are:
We present PRUSC, an approach to mitigate the effects of spurions correlations. Our method relaxes the assumptions of previous work: PRUSC does not require prior knowledge of spurious correlations, and it does not require any group annotations in either the training or the validation set. We show how to extract a subnetwork optimized for unlearning data manifolds induced by spurious attributes. We base our solution contrastive learning, and introduce effective way to sample contrastive batches based on representation clustering.
We evaluate PRUSC on benchmark datasets and show that it has superior worst-group accuracy to annotation-fre free approaches and is competitive with approaches that require annotations. We show that PRUSC is robust to multiple spurious correlations and achieves the largest improvement over all spurious attributes.
We evaluate our approach the CelebA dataset [Lu et al. 2015) with potentially multiple spurious attributes, the ISIC skin cancer dataset (Codella et al. 2019 realistic collection of images used for melanoma detection, and the artificial benchmark Waterbirds (Sagawa et al. 2020a). Our method outperforms all approaches that do not require group annotations across all benchmarks ks and has the highest overall score on 2 out of 3 benchmarks. Our code is available at: htttss///githu/ cww htap:ttub..ommmiix-ggoo/////// /atp/aiiiiigoppuue.......
2
Related Work
In this section, we review related work on mitigating spurious correlation. Approaches to mitigate spurious correlations can be broadly categorized based on assumptions: methods that require prior knowledge of
2

---

Published in Transactions on Machine Learning Research (03/2025)
spurious correlations, such group labels or human annotations, and methods that do not require prior knowledge.
With prior knowledge of spurious correlations, training regimes can directly mitigate undesired attribute influences or relations in the data and consequently yield high accuracy, both on average and in the minority
group. Data-centric approaches such as UV-DRO (Srivastava et al., 2020) and L.ISA (Yao et al. 2022)
assume full information about the type of the spurious correlations on all instances and augment the training data with additional samples to eliminate the spurious correlation. GROUPDRO (Sagawa et al. 2020a instead uses the existing group labels directly to minimize the loss for the worst-grous performance. Requiring only a small subset of group labeled training data, BARACK (Sohoni et al. 2022 predicts other instances' groups and DFR (Kirichenko et al., 2023; Izmailov et al. 2022] retrains part of the model with a small, spurious-free free dataset. Group annotation or tuman-in-the--oop annotations is, however, costly and difficult to achieve in practice. In particular complex spurious attributes, such as as fluency bias (Wang et al. 2023) or texture and shape bias (Geirhos et al. 2018 remain challenging, as they are difficult for humans to detect.
Assuming no group annotations, Sohooi et al. (2020); SSoo et al. (2022) and Creager et al. (021 focus automatically inferring appropriate groups before training a robust model (e.g., GROUPDRO) with pscudo-group labels. Several methods follow a two-stage training approach, where the first stage is commonly
used to identify hard cases from ERM training. These hard cases are instances that the ERM model fails to classify, presumably because ERM relies on spurious correlations, with the hard-to-classify cases not exhibiting the spurious correlation, i.e,, chas-conflicting samples. Not utilizing the misclassified cases, SPARE (Yang et al. 2024) infers groups labels by clustering model outputs in early epochs, then applies importance sampling to upsample smaller clusters (minority groups) and downsample larger clusters (majority groups)
during the remaining epochs. Approaches differ in the second stage: Liu et al. 2021 Nam et al. (2020)
upweight hard cases in an additional training run, while Yaghoobzadeh et al. (0021) exclusively fine-tune hard
cases. CNC (Zhang et al. 2022) and DCWP (Park et al. (2023) utilize the set of hard cases for contrastive learning to define contrastive batches and retrain the whole network (CNC) or to extract a subnetwork (DCWP) with contrastive loss. Different from the two-stage approaches, DEDIER. (Tiwari et al., 2024) uses knowledge distillation to find overconfident predictions in early network layers d, and then uses this signal to weight the distillation loss on per-instance level. SELF (LaBonte al., 2024) finds small set of hard cases by measuring the degree of disagreement in predictions between an ERM and an early-stop ERM model, and then applies last-layer fine-tuning with this set. The effect of methods that rely on ERM misclassifications for hard cases depends on both the quantity and the quality of the misclassifications. In cases where ERM models almost perfectly fit the training data (training accuracy close to 100%), there are not enough hard cases (i.e., misclassifications) and these methods fail. To obtain a sufficiently large set of hard cases, methods either use a carefully controlled early stopping criterion (Liu et al., 2021 or classify based on earlier layers in the networks, assuming that these layers still contain information about misclassified samples, likely originate from learned spurious correlations (DEDIER (Tiwari et al.) 2024 Therefore, methods that rely on misclassification cannot directly mitigate spurious correlations of a trained ERM model unless trainers have access to the entire training process of that model.
3 Problem Setting and Example
In this section, we describe spurious correlations in machine learning more formally (Sec 3.1 and show the core idea of our method PRUSC for mitigating spurious correlations in simple example (Sec 3.2
3.1 Problem Setting
Suppose a data set D consists of data where each sample has some attributes of the input x that are correlated with the target labels y E yV With prior knowledge of spurious attributes a E A, we can partition the data set DD into groups g E G according to G = A x }/ i.e., each group is defined by the combination of the label and the corresponding spurious attribute (Sagawa et al. 2020b
A model trained with Empirical Risk Minimization (ERM) on the class labels may be able to separate the classes well but rely on spurious attributes to do so, resulting in mismatch between the intended and the
3

---

Published in Transactions on Machine Learning Research (03/2025)
Figure 1: Simple network on the two moons dataset. Decision boundaries (black lines) depend on model capacity and loss. Left: Standard ERM with cross-entropy loss with nearly linear decision boundary shows strong dependence on the x-coordinate (nearly linear vertical decision boundary). Center: Pruned network (masking) using only 50% of the weights at test time shows less dependence on the x-coordinate. Right: Masking, and using our contrastive loss (cf. Sec. 5.2.1 results in non-linear decision boundaries with large margins.
model-learned solution (Geirhos et al. 2020 Spurious attributes typically appear in the majority of the dataset. Thus, by using these attributos, ERM model can minimize the training loss for the expected average population, but still show high errors on the minority group that lacks these spurious attributes. Furthermore, in cases where the spurious attributes are easy to learn (e.g., background instead of the object), ERM prioritizes learning those attributes over more complex ones and then loses the ability to generalize. Our goal is to mitigate the learning of spurious correlations by addressing both statistical (highly unbalanced group distributions) and geometric (the tendency to learn simpler features) issues.
3.2 Motivating Example
To illustrate the problem and motivate our approach, we consider a simple 2-D classification task using the two moons dataset (Pezeshki et al. 2021), ) with one attribute that is strongly correlated with target labels, causing neural networks to under-learn other invariant attributes. The two moons dataset D consists of input pairs (x, y), where (z,, 22) E ? and y {0. 1} (cf. Fig. 1 The dataset contains a strong relation between I1 and the corresponding label y. We train three different models on this dataset, showing the effect of smaller networks and ii) our contrastive loss formulation (Sec. 5.2.1) A simple feed-forward network (5 ReLu layers, 500 hidden units per layer, 100 epochs, cross-entropy loss) can separate the two classes very well (Fig, | left). The decision boundary is nearly linear because the cross-entropy loss optimizes for class separation and for the model it is sufficient to rely on the correlation between the x-axis values of x and to minimize the loss. We train the same network architecture with random masks (50% of weights in each layer) and use the same fraction of random weights for lesting, i.e., we randomily prune the network to 50% of its weights. We observe curved decision boundaries (Fig. [ center), suggesting that smaller network within dense network is more robust to invariant features. Combining masked training and our contrastive loss (Sec. 5.2.1 we obtain curved decision boundary with large margin (Fig. right), suggesting that the model uses information from both the x- and y-coordinates.
The above example strengthens two hypotheses for mitigating spurious correlations. First, network capacity seems to have an influence. Both, the dense network and the smaller subnetwork with randomly pruned connections perform well on the dataset, but the pruned network has better decision boundaries, i.e., depends less on the spurious feature. However, this does not suggest to train sparse architecture from scratch, because small architectures are difficult to train, whereas larger models have a higher ability to learn information and improve performance beyond the bias-variance regime (Nakkiran et al., 2021 Frankle & Carbin 2018). Instead, it suggests to first train a large network with sufficient capacity to encode all relevant information ion, and then extract a subnetwork after training that does not rely on spurious forrelations. The post most-training approach is also consistent with our realistic scenario that the existence of spurious correlations becomes apparent only after training (and possibly later the lifetime of model). Thus, extracting a subnetwork without spurious correlations less expensive than retraining the whole model from scratch. This observation is our main motivation. Second, our contrastive learning scheme improves decision boundaries
4

---

Published in Transactions on Machine Learning Research (03/2025)
by learning more information from the training data than just the simplicity bias features. Our batches of contrastive instances serve guide for the models to know which feature relations to focus less on.
4 Background: Subnetwork Extraction
In this section, we outline the approach for the extraction of subnetworks (Csordás et al.) 2020). i.c., subnetworks responsible for a specific task. The extraction method is also utilized by [Zhang al. (2021)
solve out-of-distribution generalization by probing subnetworks serving different functional subtasks from pre-trained model. Additionally, Park et al. (2023) use this approach to train subnetwork that more robust to a specific set of hard cases, therefore, mitigating the effects of spurions correlations in learning. In generally, we aim to extract a sparsely connected subnetwork from a dense network with a binary mask indicating which weights to include (the subnetwork) and which to discard.
Masking Models. Given trained neural network f. for layer in L hidden layers of a neural network f. we introduce a Icarnable paramcter II = ( ,,,,... x²) corresponding to the weights of each layer W = (w", w2 ,.., wL,. Each element T acts as a logit indicating the probability keeping the correspouding weight w/,x- The weight w/,K connects the jth ncurnn from the (I - - to the kth -eurnn from nth llyye, we write each weight and its corresponding logit as W; and n; for short. During training, network parameters in hidden layers become (w1 @ n² w O x² ..... w O Th).
The Mask Training. To train the binary mask, we freeze the model weights W = (w", w² w?.... wh) and train only the added parameters II = (π nT ..... x with each π. equal to 0.9 initially, i... high keep probability. Modular loss Lmod (Csordás et al., 2020) aims to extract set of sparse weights (the subnetwork) that retain the performance of the original dense network on the classification task.
Lmod = LCE + Σ T
(1)
where LcE is the cross-entropy loss, E, ग. is sparse regularization, which keeps logit n, small unless it is needed for the task and a is responsible for the strength of the sparse regularization.
The Gumbel-Signoid trick
(Jang et al. 2017 is applied during training to the logit
a( log(log U1/ log U2) with U1, U(O, 1),
(2)
where T is the temperature and (x) is the sigmoid function. We obtain the binary mask by
m, = [1 11,2 - si]stop + 8i.
(3)
where the threshold is set to 0.5 by default, 1 is the indicator function and [[lstop is the stop gradient operator. Accordingly, the binary mask m, 1, if the probability s; to keep this weight exceeds 0.5 and m: = 0 otherwise.
The final parameters of the resulting subnetwork are defined as W' = =(w1 @ 5m', wa c m² w @ m²).
5 Our Approach
We hypothesize that instances with the same spurious attribute are nearby in representation space, and that spurious attributes induce clusters (cf. illustration in Fig A). We verify this hypothesis for multiple attributes the CelebA dataset (Sec. 5.1p. Our approach centers around two key ideas: First, identify clusters in the latent space and move instances with the same spurions attributes away from each other and samples of the same class closer to each oth. Second, we reduce the representation capacity of the net network to subnetwork that is less prone to spurious correlations. Specifically, we learn a task-oriented subnetwork (Soc. 5.2) with a task-specific contrastive loss (Sec. 5.2.1p that optimizes the representation space. We describe the specific data selection and sampling procedure for negative and positive samples in the contrastive loss in Sec. 5.2.2 Finally, we present the overall training procedure in Sec. 5.3
5

---

Published in Transactions on Machine Learning Research (03/2025)
Figure 2: Embedding space (I-SNE) of ERM on CelebA for predicting hair color. Colors represent class labels (left), attributes (center), and k- k-means cluster labels (right). Spurious attributes are strongly correlated with class labels (e.g., female - blond hair) and sub-manifolds are defined by spurious attributes within class (non-blond, beard and young). k-means tends to cluster based on attributes with high purity.
5.1 Analysis of ERM Representation Space for Multiple Spurious Attributes
In the presence of spurious attributes, ERM tends to learn them as predictive features yielding decision boundaries that are aligned with the spurious feature (as exemplified on the two moons dataset in Fig. D We analyze the behavior of ERM in the presence of multiple spurious features on the CelebA dataset. We use an ERM model for predicting hair color (blond or not blond) having a training accuracy of 95%.
To analyze the feature space, we select three additional attributes (gender, age, presence of beard) as potential spurious attributes. For a brief description of the CelebA dataset see Sec. 6.1 and Appendix A.1 for detailed information. Fig. shows the I-SNE [Van der Maaten & Hinton 2008} projection of a random subset of CelebA training data representations from the penultimate layer"] The two classes are well separated in representation space (Fig. _ left). Interestingly, there is a strong correlation between spurious attributes and classes in the representation space. Female correlates with blond hair, whereas the cluster for non-blond hair (orange cluster, left in Fig. 2 corresponds to young people and people with beard. Furthermore, within class (non-blond hair), the two attributes partition the data into manifolds, even though the attribute labels are not used during training. Finally, unsupervised k-means clustering in representation space tends to cluster by attributes with high purity (Fig. 2 right). This means that when optimizing for class separation, instances from the same attribute group a, E A (e.g., persons with beard, young persons) lie close together in representation space, even though the group labels are not used during training.
Tab. shows the purity of classes (blond non-blond hair) and spurious attributes (male or female) in clusters obtained from k-mcans clustering on represent attisssntations ations of the CelebA training set. Each cluster is assigned a label based on the most frequent class/ spurious attribute. Purity is calculated as the number of correctly matched attribute and cluster labels divided by the total number of instances that cluster. The table demonstrates that k-means clustering effectively groups samples by class label, but achieves even higher purity for spurious attributes. Fig. 2 (right) demonstrates k-means clustering (k =) well defines clusters based on not only one but multiple spurious attributes. The critical case is when the model can potentially separate both, classes and attribute groups, i.e, when the model has high predictive performance and the latent space shows clear clusters for spurious attributes. Such a model will i) make predictions based on spurious attributes and ii) not generalize well samples with attribute combinations that do not align with the spurious correlations.
Table 1: Purity measurement accord- ing to class labels ()/purityy and spurious attributes (A-purity) of un- supervised k-means clustering with different values of k.
V-purity
A-purity
4
95.55%
96.48%
8
97.01%
97.32%
16
97.00%
97.80%
Yang et al. 2024) investigate and provide a theoretical proof using two-layer fully connected neural network trained on the CMNIST dataset (which has spurious correlation between each class and specific color) to mimic the carly learning process of neural networks. The authors show that each group (defined by label
We discard samples with more than of the selected attributes for easier interpretation.
6

---

Published in Transactions on Machine Learning Research (03/2025)
A
B
E
a
3
Figure 3: Assumption and overall idea. A. Instances from the same class (same colour) lie in different clusters apart in feature space. Instances with the spurious feature (solid border) are nearby in feature space, i.e., the spurious feature induces clusters and primarily defines the shape of the data manifold. B. Contrastive loss: For an anchor x, instances from the same class but different clusters constitute positive samples '=, instances from the same cluster negative samples x- Positive samples are moved closer to the anchor, negative samples away from the anchor. In each iteration multiple samples act anchors (with their positive and negative samples). C. Goal and effect of our contrastive loss. The spurious feature does not define data manifolds and cannot be used as discriminative feature between classes. Note: Our approach does not assume (and need) knowledge about the spurious feature(s).
and its correlated spurious feature) can be separated early in training based on the model's outputs. Our observalions ions empirically show that even with a complex neural network, which is assumed to learn simple functions early in training and gradually build complexity over time, the representation space can still be separated by spurious features after training once spurious correlations are learned.
Based on these observations, our goal to ensure that spurious altributes do not define data manifolds and cannot be used discriminative attributes.
5.2 Task-oriented Subnetworks
To ensure that spurious attributes do not define data manifolds and cannot be used as discriminative attributes, we introduce a task-oriented subnetwork, and contrastive loss that optimizes the representation space. More specifically, we extend the approach for extracting subnetworks (ef. Sec. 4 by incorporating task-specific loss function to prevent the network from lcarning a grouping of samples based on spurions attributes. The loss for our (ask-oriented subnetwork is defined as
L Lmod + BLaasks
where Linod is the modular loss (cf. Sec. the loss terms.
4 Ltask the task-specific loss, and / is a hyper-parameter to balance
The choice of Tatakk in combination with a mask-training dataset Drask defines the specific task the subnetwork is optimized for. For example, Csordás al. (2020) finds a subnetwork responsible for a particular class (Drask contains examples of this class, Leask = 0), and Park al. (2023) identify a subnetwork that is more robust to spurious feature (Dtask contains instances misclassified by ERM, and Liask is debiasing term). Details on these mothods can be found in Appendix A.2
We detail our choice of Lrack and Drask in Sec. 5.2.1 and 5.2.2 respectively. Fig. 3 illustrates the training procedure of our task-oriented subnetwork. Based on the assumption that instances the same class (same color) lie different clusters, and that spurious attributes induce clusters, we define contrastive loss L which moves samples of the same class in different clusters closer together in representation space, and pushes samples of the same cluster apart.
7

---

Published in Transactions on Machine Learning Research (03/2025)
5.2.1 Contrastive Learning
We use a variant of contrastive loss (Khosla et al., 2020) to mitigate the tendency of ERM to form clusters based on spurious attributes. Different from the original work, our rastivevive batch defined not only by the class labels, but also by the location of the samples in representation space. Fig. 3 outlines the idea.
For each contrastive batch we randomly sample set of anchors and for each anchor a corresponding set of positive and negative samples. We define positive samples samples of the same class as the anchor, but from a different cluster than the anchor. We define negative samples as samples from the same cluster (independent of their class label). For anchor (z.y) belonging to cluster с C C, the positive set of size P is ((xt,yt) | (at, yt) E C (CAcAg yt = yy and the negative set of size is {(TT))) (a "uTT." .8) E (2,, y)}. The contrastive loss term defined with respect to the anchor (2.y) is
Looa (2)
Σ log Σ Expxp(= 2
(5)
2T/T) + ΣN exp(2 T an /T)
where T > O is a scalar temperature hyper- parammerr,, 2,²t.² x are the representations obtained from the penultimate layer of the corresponding inputs x; xt and x- from anchor, positiues and negatives respectively. We follow the finding of (Khosla et al. 2020) that locating the summation outside the log improves performance over summation inside the log. This loss forces the representations of samples of the same class across clusters to become more similar and more similar the anchor, while samples representing potentially spurious attributes are pushed away from the anchor.
5.2.2 Data
PRUSC does not require prior knowledge of spurious attributes and is applicable in the presence of multiple spurious attributes. We do not assume that group annotations are available but automatically infer groups to construct the de-biasing dataset Dtask- Based on the assumption that the representation space of ERM has regions corresponding to spurious altributes (cf. Sec. 5.1 and Fig. 2 we use unsupervised clustering to construct the spurious groups for our dataset Drask.
Given an ERM model, we calculate the embeddings of the training data D and apply k-means clustering on the embeddings. Each sample in the training set is assigned to a cluster e, (r, y, e) E E X x y x C, with input domain X, set of class labels V, and set of clusters C. We define the number of classes in the classification task as | = K and the number of clusters [C|=k.
We distinguish clusters based on their class purity. A cluster is called i-dominant for class i if at least 90% of its samples belong to class i, i.., cluster is i-dominant, if 3i E 3/ such that (((==)))) > 0.9. Otherwise, C is neutral. A cluster must. fall into one of the K + categories that are either neutral or i d-mminatnt for some i y and multiple different clusters may be i-dominant for the same class i. C# is the set of i - dominant clusters (of the same class i) and CN is the set of neutral clusters.
[c]
Based on the observation in Section 5.1 that cluster purity is higher w.r.t. attributes than w.r.t. to classes, we derive that minority instances in an i-dominant cluster are typically hard cases. Hard cases contain a spurious altribute, but not the spurious correlation, i.e., are assigned to a class other than the spuriously correlated one. Accordingly, the set of hard cases h; obtained from the representation space contains samples with labels assigned to clusters dominated by a different class:
h4 ((.,v,)) | CE ー
(6)
To construct a class- balanced dataset Drask for subnetwork training, we determine a fixed number p of samples for cach class. We draw p samples for class i as follows: First, we take all samples from the hard cases sot h,- Second, we draw the remaining samples for class i uniformly from all i-dominant and neutral clusters. That. is, from cach cluster in C and CN, we draw an equal number of p- hal samples having class label i. We obtain a subset of the training data Drask that is class balanced with p samples for each class.
[cl+|ON]
In the exceptional case [bs] |C] = (no neutral cluster exists and all clusters are dominant for some class other than i) we only use hi which may result less than p samples. We never encountered such situation our experiments.
8

---

Published in Transactions on Machine Learning Research (03/2025)
By sub-sampling Daask we intend to construct subset from the training data without spurious correlations. In order to mine hard cases without prior knowledge of spurious correlations access to ERM misclassifications from pre-training, we sample minority cases from i- dominant clusters. These instances are supposed to be anti-correlated. In addition, we draw remaining samples for class uniformly from all i-dominant and neutral clusters to construct balanced dalasel, i.e,, without spurious correlation. Drawing an equal number of samples per class results in a balancing, both along (spurious) features and classes. We also use that balanced dataset Drask for fine-tuning the extracted subnetwork after pruning, in order to avoid re-learning the spurious correlations. The ablation results in Sec. 7.1 confirm that fine-tuning with Drsk is an effective performance improvement.
5.3 Overall Training Procedure
Given a trained ERM neural network the dataset D, perform three steps sequentially.
Representation Clustering and Sub-sampling. First, we extract the representations learned by the ERM model as the output of the last hidden layer of f: femb- We apply k-mcans to fems(x) with a pre-defined We determine cluster labels (i (i-dominant and neutral) and construct the subnetwork mask-training dataset Drssk as described in Sec. 5.2.2 In our experiments, we set p, the number of samples for each class such that Prask typically contains 10% of the samples of D.
Binary Mask Training. We freeze all weights W of f and train only the mask II with the mask-training dataset Drask (cf. Sec. [). To construct contrastive batches, we randomly sample multiple anchors æ from different clusters along with their correspouding positives at and negatives = We accumulate the per-batch batch constrastive loss by summing over Loon of all anchors in the batch. The mask II = (π , , π b is updated through the final loss term
L= LCE + a Σ Te + BLoon
(7)
where cross-entropy loss LCE and sparse regularization E; कy correspond Lmod for extracting a sparse subnetwork, and Lcon is the contrastive loss to mitigate the formation of spurious attribute clusters.
Fine-tuning. After binarization of the trainable mask II, the resulting binary mask m defines the parameters of the subnetwork as W = (w1 @ m= w² @ m m²:.., w @ m-). We fine-tune the extracted subnetwork on the dataset Drask with cross-entropy loss for a few epochs.
6 Experiments
We evaluate the effectiveness of PRUSC on three datasets (CelebA, Waterbirds, and ISIC) in comparison to state-of-the-art methods (Sec. 6.4b We further examine the impact of PRUSC in the presence of multiple
potential spurious correlations Secc. 6.5
6.1 Datasets
Following previous work, our classification target is hair color (blond or non-blond hair) on the CelebA dataset (Liu et al.] 2015), where the potential spurious attribute is gender (male or female). Majority groups are blond females and non-blond males with proportions of 44% and 41% of the data. The minority groups are non-blond females (14%) and blond males (1%). We extend the common sctup to multi-spurious dataset by incorporating additional attributes (no beard, young, heavy makeup, wearing lipstick, pale skin) that also pose potential spurious correlations based on their heavily imbalanced distributions.
CelebA.
We for all experiments and show that numher of clusters not critical
for approach,
long as it large enough in section 7.2
9

---

Published in Transactions on Machine Learning Research (03/2025)
standard spurious correlation benchmark. This artificially constructed dataset pastes bird segmentations from the CUB dataset (Wah et al. 2011) onto backgrounds from the Places dataset (Zhou et al., 2018). The birds labeled as either waterbirds or landbirds are placed on backgrounds of either water or land. The task is to classify the types of birds: y = (waterbird, landbird) and pasting 95% waterbirds on a water background and 95% landbirds on a land background results spurious correlations in the standard training set since the models tend to rely on the background instead of the actual bird characteristics. Accordingly, the spurious attributes are A = {water background, land background).
Waterbirds. Waterbirds was introduced by Sagawa et al. (2020a)
ISIC. ISIC Skin is a real-world medical dataset provided by the International Skin Imaging Collaboration ISIC (ooellaa et. al., 2019). The objective is to distinguish benign (non-cancerous) cases from malignant (cancerous) cases. Using the source code of Rieger et al. (2020) we retrieved 20, 394 images for the two classes benign (17, (87, and malignant (2,513) from the ISIC Archive Nearly half of the benign cases (8, 349) have colored patches attached to patients' skin, whereas no malignant case contains such patches. In this realistic dataset, the group "malignant with patches" is missing from the training set. To create this missing group during the validation procedure, we programmatically add colored patches to subset of malignant cases outside the area of the lesion, following Naula et al. (2021).
Details about the distributions of spurious attributes in each dataset are provided in Appendix A.1
6.2 Baselines
Empirical Risk Minimization (ERM) represents conventional training without any procedures to improve worst-group accuracy.
We compare PRUSC with two approaches that require group annotations: GROUPDRO (Sagawa et al., 2020ab requires knowledge about spurious attributes and group annotations to train from scratch, while DFR (Kirichenko et al.) 2023) requires group-balanced subset for its retraining phase. To meet this requirement in the ISIC dataset, create artificial samples for the missing group (malignant with patches) according to Nauta et al. (2021). We use these artificial samples during the training of GROUPDRO and DFR. To analyze the importance of a group-balanced subset in DFR, we implement DFRehass which is a version of DFR that uses our automatically inferred class-balanced subset Drask from clustering instead of annotated
data.
In addition, we compare to methods with relaxed annotation requirements. CxC Zhang et al. 2022 and DCWP (Park et al. 2023) work with ERM misclassified cases instead of group annotated samples. These methods sometimes necessitate retraining the ERM with early stopping criteria to obtain sufficiently large set of misclassified samples. SPARE (Yang et al. 2024) shows that groups in G can be separated carly in ERM training if the spurious correlations are strong enough. It infers groups from early training epochs and from these inferred groups samples group-balanced mini batches during training of the remaining epochs. DEDIER (Tiwari et al. 2024), is an early readout mechanism with a distillation model. This method identifies misclassifidd cases from early layers of the network by design. However, both SPARE and DEDIER. require group information during validation to tune the hyper- syper-parametess and determine the early epoch or layer from which to take the readout. Early-stop disagroement SELF [LaBonte et al. 2024) fine-tunes the last layer weights with a small held-out dataset chosen by maximizing the disagreement between ERM model and an early-stop ERM model. SELF requires neither group annotations nor class labels for the held-out dataset, however, it does require group annotation during hyper-parameter
tuning. In contrast to the above methods, PRUSC does not require any group annotations, neither for (re- (re))training, nor for fine-tuning hyper-parameters (cf. annotation requirements in the first column of Tab. 2).
DCWP, DFR, DFRciass and PRUSC, first require an ERM model that is affected by spurious correlations before taking action to mitigate those spurious correlations. For fair comparison, we use the same ERM model, which is the ERM baseline reported in Tab. 2 for all methods. Since the training accuracy of this baseline ERM is about 99%, DCWP needs access to the full training process of the ERM model and uses earlier model checkpoints to obtain a sufficient amount of misclassified samples.
httpss//www. www.:iiiiii-accie.coom
10

---

Published in Transactions on Machine Learning Research (03/2025)
Table 2: Performance overview, comparing methods with different assumptions on availability of annotations (Annot.) of the spurious features in training (Tr) and/or validation set (Val). ** denotes results reported in respective papers, n.a. denotes that results are not available in the original paper. denotes the presence of artificially created missing group in ISIC, otherwise DFR and GROUPDRO would not be applicable. We bold the highest WGA without any group annotation requirement and underline the best among all methods, We report the mean and standard deviation over 5 runs.
Annot. Tr/Val
CelebA
ISIC
Waterbirds
AVG
WGA
AVG
WGA
AVG
WGA
X/X
ERM (baseline)
90.1
49.711.
86.910.2
34.410.8
91.510.9
74.912.8
VIV
DFR Kiriebenko et al. 2023
91.3=0.3
88.3+1.1
87.4+1.3"
77.7+2.4
93.7+0.4
91.5+1.0
VIV
GROUPDRO (Sagawa et al.
2020s
93.0=0.1
88.7
87.7
59.0+0.4
90.3+0.0
87.0:10.1
X/V
CNC** Zhang et al.
93.9+0.1
88.9+1..
n.a.
n.a.
92.0+0.6
89.940.8
X/V
DEDIER Tiwarr ct
2024
93.210.1
89.611.7
n.a
n.a
12.1
89.810.5
X/V
SELF LaBonte
2024
I1.
83.9+0.9
n.a
n.a
n.a
93.0+0.3
X/V
SPARE Yang et al.
2024
91.1=0.1
90.3+0.3
11.8
n1. 96.240.6
91.6#0.8
X/X
DCWP
Park al.
2023
88.9
73. 73.00.2.0
86.411.7
45. 15.1
1.4
66.413.5
X/X
DFRelass
84.9
67.7 10..8
74.1
48.816.0
91.010.8
75.511.2
X/X
PruSC
91.0=0.3
89.7+0.
86.1+0.4
75.1=1.7
89.4+2.9
82.5+2.2
Hyper-parameter Setup. We use ResNet18 and ResNet50 pre-trained on ImageNet. We train the models for epochs using SGD with a constant learning rate of 10-3, momentum decay of 0.9, weight decay of 10-2, 2, and batch size 32 64. All images are resized 224 224. We use ResNet50 evaluate the predictive performance in Tab. 2 to compare with the original work for CNC, DEDIER and SELF. We use ResNet18 to investigate the robustness of our model to multiple spurious correlations (Sec. 6.5) and in the ablation study (Sec. 7.1 For the subnetwork extraction approaches (DCWP and PRUSC), we report results with pruning around 50% of the network parameters and discuss the impact of other pruming ratios in Sec. 7.3
6.3
Evaluation Metrics
We report worst-group accuracy (WGA), the accuracy of the group (attribute-label combination) that a particular method performs worst. WGA is the standard metric for evaluating whether models use spurious correlations (Sagawa al. 2020a). A significant drop between average accuracy and worst-group accuracy is strong indicator of reliance on spurious correlations. To assess the impact the group with the fewest. training samples (usually the group with a spurious attribute uncorrelated with target label), report minority-group accuracy (MGA) for each spurious altribule a,, The difference between WGA and MGA is that the former is defined per method, and the latter is defined on the data. Since the minority group is usually the group with a spurious attribute uncorrelated with a target label, performance is mostly lowest on this group and WGA-MGA, but not necessarily. To assess group fairness, ie., whether all groups have similar prediction errors, we measure the unbiased accuracy gap (UAG). Unbiased accuracy (UA) the accuracy on a balanced test set, sampled such that there is no correlation between the attribute o; and a target label. UAG is the difference between the overall (official) test set accuracy and the UA. A small UAG indicates that groups are treated equally, regardless of whether they are minority or majority groups.
6.4 Predictive Performance
Our main results on predictive performance are shown in Tab. PRUSC shows the best performance in terms of worst-group accuracy (WGA) among methods that do not require any group annotations and shows comparable performance in most cases (CelebA and ISIC) to ofate-of-the-art methods that rely on annotations. Notably, our method successfully improves the accuracy of the worst group malignant with patch ISIC, although it has never seen an example of this group during training. Our performance is even comparable GROUPDRO and DFR which explicitly need (artificial) minority group samples during training.
11

---

Published in Transactions on Machine Learning Research (03/2025)
Table 3: Minority group accuracy (MGA) across multiple spurious attributes A = {male, no beard, heavy makeup, wearing lipstick, young, pale skin) on CelebA with classification target hair color. ERM (baseline), DCWP and PRUSC are trained with class labels only, GROUPDRO and DFR in addition on attribute a; male to mitigate the spurions correlation on gender. We report the accuracy of the minority groups defined in the training dataset. We bold the highest MGA among all methods.
Annot.
male
no beard
heavy makeup
wearing lipstick
young
pale skin
Tr/Val
x/X
ERM (baseline)
48.5+1.0
42.3=1.3
79.0+0.6
79.1
88.2+0.4
87.1
V/V
GROUPDRO
30.3
86.0=0.7
88.9#0.6
90.1=0.5
89.8+0.8
38.2+0.3
V/V
DFR
86.110.2
78.0=0.5
83.9
83.940.4
86.0:t..1
81.810.2
X/X
DCWP
82.
80.0=0.9
77.5
77.410.5
82.1
80.
X/X
PruSC
91.0+0.3
92.1++02
89.9
89.3+0.3
90.5=0.1
89.3+0.7
The DFRsssss method mimics DFR but replaces the group-balanced subset (which requires manual annotations) with
subset (class annotations are readily available). DFR falls short compared to DFR in all cases by large margin, highlighting the importance of group annotations in DFR retraining. Using the same retraining dataset as DFRahss: the success of our method PRUSC emphasizes the necessity of distorting spurious clusters, which we achieve through a contrastive loss. We empirically analyze the impact of individual components of our method in an ablation study (Sec. 7.1..
On Waterbirds, there is a clear gap between annotation-free methods and those that rely on annotations. Using only annotations in the validation set, SELF even outperforms methods that require more annotations. The reason is peculiarity of the Waterbirds dataset: its validation set is spurious- equal quantities of land and water backgrounds, which gives an advantage free,, i.e., each class has methods that rely the validation set. This advantage becomes particularly apparent for DFRcluss By sampling a class-balanced subset from the validation set instead of the training set, La Bonte et al. (2024) report a WGA of 92.6 for DFR,ass:: which is comparable to the performance of DFR on a group- balanced subset obtained from label annotations. For truly annotation-fee methods (such as ours) a spurious- free validation is not available, as its construction itself requires annotations.
6.5 Robustness to Multiple Spurious Correlations
In this section, we analyze how PRUSC performs in the presence of multiple spurious correlations within a dataset. Since we do not explicitly use information about the spurious attribute during subnetwork training, we expect the representations to be agnostic to any spurious feature. We use the CelebA dataset which has potential set of spurious attributes A (Seo et al. 2022 (see Appendix A.3 for details). Specifically, we investigate six potential spurious attributes a; E A= = (male, presence of beard, heavy makeup, wearing lipstick, young, pale skin]. For each attribute ag, we create group set G {(blnnd, a:), (uon-blond, a:), (blond, non-a;), (non-blond, non-a;)}. We train an ERM model to predict hair color and evaluate the performance separately for each group in G. We then train all methods for correcting spurious attributes. GROUPDRO and DFR are trained on balancod groups for the attribute malc!"
In Tab. 3 we report the performance of the minority group across multiple spurious attributes. Details of the data distribution and accuracy values for each individual group & {bblond, a;), (non-blond, ai), (blond, non-o:), (non-blond, non-a;)) are provided in the Appendix, Tab. 7 and Tab.
LaBonte et al. 2024 report their method annotation-free in their result tables, but state in the limitations section that they use "a small val validation datasct with group annotations for model selection" and call for future work to completely remove this assumption.
7 "CRoUDROO and DFR are single-attribute methods, and cannot be trained for multiple attributes at the same time. Since the spurious attrihutes in CelehA are themselves strongly correlated (male, beard) there are still performance improvements for attribute groups not specifically trained for.
12

---

Published in Transactions on Machine Learning Research (03/2025)
Table 4: Multiple spurious attributes. We report unbiased accuracy (UA), worst-group accuracy (WGA) and unbiased accuracy gap (UAG). We bold the highest WGA and underline the smallest UAG for cach spurious attribute. higher is better, lower is better.
male
no beard
heavy makeup
wearing lipstick
young
pale skin
ERM
48.511.0
1.3
79.010.6
79.1
88.210..
87.110.6
WGA t
GROUPDRO
86.7
86.0=0.7
88.9
88.3+0.5
85.7=0.4
87.4+0.7
PRUSC 88.9
89.6+10.5
74..1
88.7=0.2
82.0+0.2
89.0#0.4
ERM
82.0#1.1
78. 7=0.3
88, 88.4
88.0+0.8
91.3+0.2
91.0+0.3
UA
GROUPDRO
88.520.4
91.7=0.2
91.011.0
91.4+0.2
90.510.4
90.3
PRUSC
90.4+0.2
91.1=0.2
90.0#0.5
90.2+0.3
90.0+0.5
89.8+0.1
ERM
7.0+0.2
9.4+1.1
0.9
0.9
2.0=0.4
2.0+0.3
UAG 4
GROUPDRO
2.2+0.2
2.010.2
0.810.1
1.4:0.2
1.0
1.110.11
PRUSC 0.5+0.1
1.0+0.1
0.1
0.1
0.8
0.3+0.0
The ERM models usually fail for the minority group represented by the target and uncorrelated attributes. For example, for the attribute gender = male, the ERM model only has 48.5% accuracy for the group (blond hair, male) since in the training set there are only 1, 102 samples in the group (blond hair, male) while the group (non-blond hair, male) contains 53, 483 samples. For the ERM model Tab. 3 the minority group accuracy indeed always equal worst group accuracy, which not the case for the other methods. For example, GROUPDRO has accuracy of 88.7% on the minority group (blond hair, male), but worst-group accuracy of 86.7% for the group (not blond hair, not male). PRUSC has the largest. improvement over the baseline across all spurious attributes. In particular, on the spurious attribute "male", PRUSC even outperforms GROUPDRO and DFR which were trained with known group labels for this attribute.
In Tab 4 we compare GROUPDRO that relies on group annotations, with PRUSC that does not require prior knowledge of spurious attributes. Both methods show an increase of worst-group accuracy (WGA) over the ERM baseline for all groups, suggesting that both methods can successfully mitigate certain spurious correlations. However, in CelebA's official training and test split, the groups identically distributed. The WGA indicates whether method has successfully reduced the impact of biasing features, but because the average accuracy is driven by the majority group, it may mask performance declines in the remaining groups. That is, we cannot determine from these two measures alone whether a model treats all groups equally. Our approach, PRUSC, is on par with GROUPDRO in terms unbiased accuracy (UA) and WGA. This indicates that both mothods can effectively unlearn spurious correlations and maintain strong performance even when there is distribution shift from highly imbalanced training set to balanced test set. However, PRUSC shows a smaller UGA in all cases and is therefore more consistent across different spurious attributes and shows more balanced performance across groups.
7 Detailed Analysis
In this section, we show an ablation study to evaluate the effectiveness PRUSC components, including the pruning and fine-tuning steps, as well as the use of contrastive loss (Sec 7.1p. We also analyze the impact of
number of clusters (Sec 7.2 the impact of varying pruning ratios (Sec 7.3 the performance with different
choices of contrastive batch in PRUSC (Sec(7.4) and the representation space after mitigation (Sec 7.5
7.1 Component Ablation
We analyze the impact of each component of PRUSC. Tab. 5 shows an overview of our settings. Specifically, we evaluate the impact of pruning, i..., subnetwork extraction and finc-tuning on performance, along with the
effect of applying contrastive loss at each step. If the contrastive loss is omitted during pruning (ConLoss Pr x), we train the subnetwork with cross-entropy loss and sparse regularization. Without the cantrastive loss
13

---

Published in Transactions on Machine Learning Research (03/2025)
Table Ablation study on CelebA (target: blond hair, spurious attribute: male). Effect of pruning (Pr) and finetuning (Ft) steps with (V) and without (X) our contrastive loss, cells marked 'n.a.' show no loss values because there is no corresponding pruning or fine-tuning stage. v last means fine- iin-tuning last layer only.
Setting
Fine-tuning ConLoss Pr ConLoss Ft WGA
AVG
UA
WITH SUBNETWORK EXTRACTION
V
67.7
83.7
80.4
(PRUSC)
X
89.6
90.1
90.5
41.8
91.5
73.9
X
n.a
43.7
89.6
74.4
WITHOUT SUBNETWORK EXTRACTION
n.a.
67.4
88.8
81.6
n.a.
78.9
87.9
87.2
(DFRelace
last
n.a.
79.5
83.8
85.5
during fine-tuning (ConLoss Ft x), we fine-tune the subnetwork or full model with cross-entropy loss alone. In setting 4, 'Finetunnigg column refers to fine-tuning for 2 to epochs after extracting subnetwork.
Subnetwork Extraction. Settings to 7 investigate whether one can improve the worst-group accuracy only by fine-tuning and take advantage of designing a class-balanced subset (Dtask) without a subnetwork. We skip the Binary Mask Training step (ef. Sec. 5.3) and either fine-tune the whole network (settings and 6), only the last layer (setting 7) for the same number of epochs with Drask. The latter setting is equivalent to DFRelass with Dask We observe no significant improvement when fine-tuning the whole network compared to fine-tuning only the last layer (setting vs. 7), fine-tuning the whole network even results in worse WGA,
Fine-tuning. Without fine-tuning after subnetwork extraction (setting 4) the performance of the subnetwork drops significantly, similar performance as before pruning. This finding aligns with previous work on modular subnetworks (Csordás et al. 2020 Zhang et al.) 2021 Park al.) 2023), which require retraining or fine-tuning the extracted network after pruning to achieve better performance.
Contrastive Loss during Pruning. Settings 2 and 3 show the effect of the contrastive loss during pruming (ConLoss Pr), which is the key feature of PRUSC. In setting 3, we prune the networks using the cross-entropy loss and sparse regularization, iee, vanilla, pruning. Vanilla pruning leads to slightly better AVG but keeps the WGA the same as before pruning, while PRUSC (setting 2) significantly increases the WGA, showing that the contrastive loss helps to find the correct subnetwork that is robust to spurious correlations. The contrastive loss forces the pruning process retain only those connections that contribute to an optimized reprosentation, ensuring that all spurious attributes are distributed in the representation space.
Contrastive Loss during Fine-tuning. Fine-tuning with contrastive loss (ConLoss Ft) reduces the WGA significantly compared to fine-tuning without contrastive loss, both with (setting 1 vs. 2) and without subnetwork extraction (5 vs. and 7). In all those settings, we fine-tune with the balanced subset Dtask- Kirichenko et al. (2023) showed that ERM models can effectively learn all features, but spurious features contribute more strongly the prediction, due to the data distribution. Hence, during fine-tuning without. subnetwork extraction, the model only has to adjust the classifier weights to rely more on invariant features. This is also confirmed by the increased WGA in setting 7 (fine-tuning only the last layer vs. fine-tuning all layers in setting 6). Adding the contrastive loss (setting 5), the model now has to optimize dual objective: classification and representation, which is likely confusing rather than helpful, in particular as the balancod subset is comparably small. Similarly, adding the contrastive loss during fine-tuning after the extraction of a sub-network (setting 1) hurts performance. In this setting, the representation space has been optimized by the extraction of the subnetwork already and fino-tuning is only supposed to adjust the weight magnitudes after pruning. We hypothesize that forcing the network to also re-learn the representation space does not effectively align the remaining weights, but rather distorts weights and representations.
14

---

Published in Transactions on Machine Learning Research (03/2025)
Contrastive Learning with (pseudo) Group Labels. Instead of using contrastive learning with clusters defined by representation clustering, we can use annotated group labels. The clusters in Dinsk by ground truth group labels correspond to spurious attributes, i.e. the two clusters in CelebA with respect to the spurious attribute gender. This approach does significantly improve the WGA (from 50.2% to 84.8%, see Tab. 10 in the Appendix), highlighting the potential of using annotated or pseudo-group labels. However, contrastive learning with ground truth labels does not outperform PRUSC with representation clustering (WGA 89.6%). We hypothesize that this is due the alignment between our contrastive loss and contrastive batch sampling, which is particularly beneficial in the representation space. Further results and discussion are presented in App. A.5
7.2
Impact of the Number of Clusters
80
goostgoupp Accuracy
85
90
80
Accorscy (16)
80
75
Aggrage Min Ve
WGA
AVG
10
12
25
1.8
20
22
24
26
50
60
70
80
Number of clustars
(a)
(b)
Figure 4: Analysis of PRUSC wrt. the number of clusters and pruning ratio on the CelebA dataset. (a) Average, minimum and maximum worst group accuracy (WGA) among runs across varying numbers of clusters. (b) WGA and average accuracy (AVG) of varying pruning ratios.
We analyzed the impact of the number of clusters l in k m-maass clustering on the CelebA datasct. Fig. 4a shows the average WGA logether with minimum and maximum among 5 runs across different choices of k. For performance is unstable and generally low. The effectiveness of PRUSC depends on the purity of clusters with respect to classes and spurious features. For lower values of k, both, class purity ()-purity) and spurious purity (A-purity) are also lower (cf. k vs 8, Tab. []. With lower cluster purity, it is more likely that each cluster contains samples from all groups in G, making it. harder for method to sample more group-balancee subset, which in turn affects WGA performance. For k 8, the performance of PRUSC remains rather stable, indicating that k is not a critical hyper- hyper-paramett parameter of our method, as long as it is sufficiently large to obtain pure clusters, both in terms of classes and spurious features.
7.3 Impact of the Pruning Ratio
Fig. 4b shows the average and worst-group accuracy of our model CelebA for different pruning ratios. We test on the standard setup of classifying hair color and considering gender as spurious attribute. While the average accuracy does not change much as long the pruning ratio is below 70%, the worst-group accuracy varies more significantly. Submetworks the regime of small gap between average and worst-group accuracy are less prone to spurious correlations. On the CelebA dataset, the most effective pruning ratio is around 40% to 50%. When pruning beyond 70% of the parameters, the model scems fail to predict accurately, both, grosp and average accuracy drop.
7.4 Choice of Positive and Negative Samples in Contrastive Learning
We study the effectiveness of our sampling choices for contrastive batch learning in PRUSC. Recall our default setting: for random anchor, we sample positives from different clusters having the same class label as the anchor and negotives from the same cluster (regardless of class label). We compare this choice with sampling negatives from the same cluster, but different. class labels (Negative Ablation) and classic
15

---

Published in Transactions on Machine Learning Research (03/2025)
supervised contrastive learning (SupCon) (Khosla et al. by class labels. The results in Tab. G show that both choices are less effective w.r.t. worst-group accuracy. Sampling negatives from the same cluster as the anchor but with a different class label (Negative Ablation) is affected by the purity of clusters. As shown in Fig. 2 and Tab. | k-means clustering on the representations of converged ERM model leads to high purity clusters in terms of class labels. There- fore, cluster may not contain sufficient number of negatives with different class labels. Early stopping could solve this problem, but requires access to the model training procedure, and requires additional op- timization of the stopping criteria to balance model performance and the amount of misclassificaatons. Instead, SupCon aims to bring the representations of the same class closer together, tightening the samples within the same cluster, which has high potential to increase the spurious correlation instead of reducing it. Furthermore, SupCon primarily helps to separate classes (cf. increased average accuracy), which ERM models may already do sufficiently well (cf. Fig. 2 left and Fig. 5 left). Our contrastive batch is defined not only by class labels as in SupCon, but also by the positions of the samples in the representation space based on clustering.
2020 , where positines and negatives are defined only
Table 6: Average accuracy (AVG) and worst-group accuracy (WGA) of PRUSC with different sampling choices for the contrastive batch.
AVG
WGA
Default Negative Ablation
91.0%
89.7%
90.2%
73.1%
SupCon (Khosla et al., 2020
92.2%
61.5%
7.5 Representation Space after Mitigating Spurious Correlations
In this section, we discuss how the mitigation strategy of DCWP and PRUSC affects the representation space. In Soc. 5.1 we showed that ERM tends to learn and cluster samples based on spurious attributes during training (Fig. 2p. In Fig. 5 (left), we visualize the embedding space of the non-blond hair class in the test set of the same FRM model, which reveals again ERM learns based on spurious attributes. Despite achieving high average test accuracy (90%), the fact that ERM relies on spurious correlations in its predictions leads to a low worst-group accuracy (49.7%) and lack of generalization ability.
DCWP mitigates spurious correlations by extracting a subnetwork by training a contrastive loss, and sampling positives and negatives from bias-couflict (early-stop ERM misclassified) and bias-align (early-stop ERM correctly classified) cases. In terms of performance measures, DCWP successfully improves worst-group accuracy (73%), but the model still tends to group samples by the female attribute (Fig, 5 center). We hypothesize that defining bias-conffict based on ERM misclassified cases is not reliable. When spurious altribute (female) is easy to learn and strongly correlated with class (blond hair), the model learns easily and predicts correctly based on the spurious attribute, even in the early stages of training.
PRUSC climinates this reliance on spurious fcature learning. While the average accuracy remains high (91%) through cross-ent loss, the use of contrastive learning with a distortion of spurious clusters the representation space eliminates the reliance on spurious features and thus improves generalization. Our pruned model mixes up samples with different attributes within one class showing its independence from these spurious attributes (Fig. 5 right). Additional visualizations are shown in Appendix A.4
8 Conclusion
In this paper, we introduced PRUSC (Pruning Spurious Correlations) to extract spurious-free subnetwork from fully trained dense neural network. Our method does not rely on annotations of spurious attributes, but instead optimizes the representation space to produce class-induced clusters rather than clusters induced by spurious attributes (as observed in the representation space of the dense network trained with ERM).
PRUSC achieves competitive worst-group accuracy across several benchmarks without requiring any annotation of spurious features at all (not even the validation set). While PRUSC outperforms all annotation-froe methods in comparison, we found that the spurious-fres validation set of the Waterbirds dataset might lead to overestimation of the performance of methods that rely annotations in the validation set. Our
16

---

Published in Transactions on Machine Learning Research (03/2025)
Figure 5: Test set representation space CelebA for predicting hair color. The visualization shows data points of the class 'non-blond hair', colors represent the spurious attributes. All models achieve high average test accuracy (over 88%, see Tab. 2 While ERM (left) and DCWP (center) cluster samples by spurions attributes, PRUSC (right) mixes samples with these attributes, obtaining a better worst-group accuracy.
truly annotation-fre free approach not only eliminates the annotation overhead and makes PRUSC applicable implicit and unknown spurious features, but also results in subnetworks that are robust multiple spurious correlations.
By extracting the subnetwork from fully trained network without extensive retraining of the weights, we show that even dense networks, that are heavily influenced by spurious correlations, contain smaller parts that learn invariant attributes relevant to the task. In future work, we plan to analyze the extracted subnetworks detail, order to quantify the information (encoded features and relations) that retained and also the information that is lost. In particular, we aim to identify whether relevant information lost and how retain it (and vice versa for irrelevant information) in order further improve performance.
17

---

Published in Transactions on Machine Learning Research (03/2025)
References
Sara Beery, Grant Van Horn, and Pietro Perona. Recognition in terra incognita. In Proceedings of the European conference computer vision (ECCV), pp. 456-473, 2018.
Noel Codella, Veronica Rotemberg, Philipp Tschandl, M. Emre Celebi, Stephen Dusza, David Gutman, Brian Helba, Aadi Kalloo, Konstantinos Liopyris, Michael Marchetti, Harald Kittler, and Allan Halpern. Skin Lesion Analysis Toward Melanoma Detection 2018: A Challenge Hosted by the International Skin Imaging Collaboration (ISIC), 2019.
Flliot Creager, Jörn-Henrik Jacobsen, and Richard Zemel. Environment inference for invariant learning. Tn International Conference on Machine Learning, pp. 2189-2200. PMLR, 2021.
Róbert Csordás, Sjoerd van Steenkiste, and Jürgen Schmidhuber. Are Neural Nets Modular? Inspecting Functional Modularity Through Diffcrentiable Weight Masks. In International Conference on Learning Representations, 2020.
Jonathan Frankle and Michael Carbin. The lottery ticket hypothesis: Finding sparse, trainable neural networks. arv preprint arXiv: 01888003033333 2018.
Robert Geirhos, Patricia Rubisch, Claudio Michaelis, Matthias Bethge, Felix A Wichmann, and Wieland Brendel. Imagenet-trained CNNs are biased towards texture; Increasing shape bias improves accuracy and robustness. In International Conference on Learning Representations, 2018.
Robert Geirhos, Jörn-Henrik Jacobsen, Claudio Michaelis, Richard Zemel, Wieland Brendel, Matthias Bethge, and Felix A Wichmann. Shortcut lcarning in deep neural networks. Nature Machine Intelligence, 2(11): 665-673, 673, 2020.
Pavel Izmailov, Polina Kirichenko, Nate Gruver, and Andrew G Wilson. On feature learning in the presence of spurious correlations. Aduances in Newral Information Processing Systems, 01335566-38538 2022.
Eric Jang, Shixiang Gu, and Ben Poole. Categorical reparameterization with gumbel-softmax. In International Conference Learning Representations, 2017.
Prannay Khosla, Piotr Teterwak, Chen Wang, Aaron Sarna, Yonglong Tian, Phillip Isola, Aaron Maschinot, Ce Liu, and Dilip Krishnan. Supervised contrastive learning. In Advances in neural information processing systems, volume 33, pp. 18661-18673, 2020.
Polina Kirichenko, Pavel Izmailov, and Andrew Gordon Wilson. Last Layer Re- ve-TTranning is Sufficient for Robustness to Spurious Correlations. In The Eleventh International Conference on Learning Representations, 2023.
Tyler LaBonte, Vidya Muthukumar, and Abhishek Kumar. Towards Last-layer Retraining for Group Robustness with Fewer Annotations. In Advances in Neural Information Processing Systems, volume 36, 2024.
Evan 7 Liu, Behzad Haghgoo, Annic S Chen, Aditi Raghunathan, Pang Wei Koh, Shiori Sagawa, Percy Liang, and Chelsea Finn. Just Train Twice: Improving Group Robustness without Training Group Information. Marina Meila and Tong Zhang (eds.), Proceedings of the 38th International Conference on Machine Learning, volume 139 of Proceedings of Machine Learning Research, pp. 6781 6792. PMLR, 18 24 Jul 2021.
Ziwei Liu, Ping Luo, Xiaogang Wang, and Xiaoou Tang. Deep learning face attributes in the wild. In Proceedings of the IEEE international conference on computer vision, pp. 3730 3738, 2015.
Preetum Nakkiran, Gal Kaplun, Yamini Bansal, Tristan Yang, Boaz Barak, and Tlya Sutskever. Deep double descent: Where bigger models and more data hurt. Journal of Statistical Mechanies: Theory and Experiment, 2021 (12) 2021.
Junhyun Nam, Hyuntak Cha, Sungsoo Ahn, Jacho Lee, and Jinwoo Shin. Learning from failure: De-biasing classifier from biased classifier. Advances in Neural Information Processing Systems, 33:20673 20684, 2020.
18

---

Published in Transactions on Machine Learning Research (03/2025)
Meike Nauta, Ricky Walsh, Adam Dubowski, and Christin Seifert. Uncovering and correcting shortcut learning in machine learning models for skin cancer diagnosis. Diagnostics, (241)) 2021.
Geon Yeong Park, Sangmin Lee, Sang Wan Lee, and Jong Chul Ye. Training Debiased Subnetworks With Contrastive Weight Pruning. In Proceedings of the IEEE/CVE Conference on Computer Vision and Pattern Recognition, pp. 7929 7938, 2023.
Mohammad Pezeshki, Oumar Kaba, Yoshua Bengio, Aaron C Courville, Doina Precup, and Guillaune Lajoie. Gradient starvation: A learning proclivity in neural networks. In Advances in Neural Information Processing Systems, volume 34, pp. 1256-1272, 2021.
Nasim Rahaman, Aristide Baratin, Devansh Arpit, Felix Draxler, Min Lin, Fred Hamprecht, Yoshua Bengio, and Aaron Courville. On the spectral bias of neural networks. In International conference on machine learning, pp. 5301-5310 PMLR, 2019.
Laura Rieger, Chandan Singh, W. James Murdoch, and Bin Yu. Interpretations Are Useful: Penalizing Explanations to Align Neural Networks with Prior Knowledge. In Proceedings of the 37th International Conference on Machine Learning, pp. 11. JMLR.org, 2020.
Shiori Sagawa, Pang Wei Koh, Tatsunori B. Hashimoto, and Percy Liang. Distributionally Robust Neural Networks. In International Conference on Learning Representations, 2020a.
Shiori Sagawa, Aditi Raghunathan, Pang Wei Koh, and Percy Liang. An investigation of why overparameteri- zation exacerbates spurious correlations. In Hal Daumé III and Aarti Singh (eds.), Proceedings of the 37th International Conference on Machine Learning, volume 119 of Proceedings of Machine Learning Research, pp. 8346 8356. PMLR, 13-18 18 Jul 2020b.
Seonguk Seo, Joon- Young Lee, and Bohyung Han. Unsupervised learning of debiased representations with pscudo-attributes. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 16742 16751, 2022.
Harshay Shah, Kaustav Tamuly, Aditi Raghunathan, Prateek Jain, and Praneeth Netrapalli. The pitfalls of simplicity bias in neural networks. In Advances in Neural Information Processing Systems, volume 33, pp- 9573 9585, 2020.
Nimit Sohoni, Jared Dunnmon, Geoffrey Angus, Albert Gu, and Christopher Ré. No subclass left behind: Fine-grained robustness in coarse-grained classification problems. In Advances in Neural Information Processing Systems, volume 33, pp. 19399, 2020.
Nimit Sharad Sohoni, Maziar Sanjabi, Nicolas Ballas, Aditya Grover, Shaoliang Nie, Hamed Firooz, and Christopher Re. BARACK: Partially Supervised Group Robustness With Guarantees. In ICML 2022: Workshop Spurious Correlations, Invariance and Stability, 2022.
Megha Srivastava, Tatsunori Hashimoto, and Percy Liang, Robustness to spurions correlations via human annotations. In International Conference on Machine Learning, pp. 9109 9119. PMLR, 2020.
Rishabh Tiwari, Durga Sivasubramanian, Anmol Mekala, Ganesh Ramakrishnan, and Pradeep Shenoy. Using Early Readouts to Mediate Featural Bias in Distillation. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 2638 2647, 2024.
Laurens Van der Maaten and Geoffrey Hinton. Visualizing data using t-SNE. Journal of machine learning research, 9(11), 2008.
Catherine Wah, Steve Branson, Peter Welinder, Pictro Perona, and Serge Belongie. The Caltech-UCSD Birds-200-2011 Dataset. Technical report, California Institute of Technology, Jul 2011.
Haohan Wang, Zexue He, Zachary Lipton, and Eric P Xing. Learning robust representations by projecting superficial statistics out. In International Conference on Learning Representations, 2019.
19

---

Published in Transactions on Machine Learning Research (03/2025)
Shunxin Wang, Raymond Veldhuis, Christoph Brune, and Nicola Strisciuglio. Frequency shorteut learning in neural networks. In NewrIPS 2022 Workshop on Distribution Shifts: Connecting Melhods and Applicalions, 2022.
Shunxin Wang, Raymond Veldhuis, Christoph Brune, and Nicola Strisciuglio. What do neural networks learn in image classification? A frequency shorteut perspective. In Proceedings of the IEEE/CVF International Conference on Compuler Vision, pp. 1433 1442, 2023.
Yadollah Yaghoobzadch, Soroush Mchri, Remi Tachet des Combes, Timothy Hazen, and Alessandro Sordoni. Increasing Robustness to Spurious Correlations using Forgettable Examples. In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume, pp- 3319 3332, 2021.
Yu Yang, Eric Gan, Gintare Karolina Dziugaite, and Baharan Mirzasoleiman. Identifying spurious biases early in training through the lens of simplicity bias. In International Conference on Artificial Intelligence and Statistics, pp. 2953 2961. PMLR, 2024.
Huaxiu Yao, Yu Wang, Sai Li, Linjun Zhang, Weixin Liang, James Zou, and Chelsea Finn. Improving out-of-distribution robustness via selective augmentation. In International Conference on Machine Learning, pp. 25407 25437. PMLR, 2022.
Dinghuai Zhang, Kartik Ahuja, Yilun Xu, Yisen Wang, and Aaron Courville. Can subnetwork structure be the key to dut-of-distribution generalization? In International Conference on Machine Learning, pp. 12356 12367. PMI.R, 2021.
Michael Zhang, Nimit S Sohoni, Hongyang R. Zhang, Chelsea Finn, and Christopher Ré. Correct-N-Contrast a Contrastive Approach for Improving Robustness to Spurious Correlations. In International Conference on Machine Learning, pp. 26484-26311 PMLR, 2022.
Bolei Zhou, Agata Lapedriza, Aditya Khosla, Aude Oliva, and Antonio Torralba. Places: A 10 Million Image Database for Scene Recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 40(6): 1452-1464, 2018. doi:
20

---

Published in Transactions on Machine Learning Research (03/2025)
A Appendix / supplemental material
A.1 Datasets
CelebA. Tab. shows the distribution of group = {(blond, az), (non-blond, az), (blond, non-a:), (non- blond, non-a;)) training dataset and testing dataset. This shows the imbalanced distribution of the group (target, correlated attribute) and (target, un-correlated attribute). The models can both depend on the easier to learn features and the dominance of training data to suffer from spurious correlations. Generally, the minority group (group having smallest portion in training set) is the group having worst accuracy in test. (with respect to baseline ERM).
Table 7: Data distribution in CelebA dataset of groups 9 E G = y x A forming by target class blond or non-blond hair y E y and spurious attribute a E A. Gray color highlights the minority group smallest. p(g) among groups the training set.
Training distribution
Testing distribution
Attribute a
Group g
Quantity
p(g) (%)
p(t
a) (%)
Quantity
p(g) (%)
p(t ()) (%)
0,
1,858
2.1
7.8
14, 415
44.3
76.4
0,
53, 483
71.7
98.0
13, 391
41.1
97.9
male (M)
1,
18,417 417
24.7
92.2
4, 4,463
13.7
23.6
y = 1, a =
1, 102
1.5
2.0
285
0.9
2.1
0,
21, 229
28.5
98.9
5, 369
16.5
98.8
0,
812
45.3
63.7
22, 437
68.9
82.7
no beard (NB)
1,
243
0.3
1.1
64
0.2
1.2
1,
19,276 276
25.9
36.3
4, 684
14.4
17.3
y 0,
53, 885
72.3
89.5
18, 620
57.2
92.3
heavy makeup (HM)
0,
156
1.6
8.1
9, 186
28.2
74.2
1,
6,317
8.5
10.5
1, 1,0
4.8
7.7
1,
13, 202
17.7
91.9
3, 198
9.8
25.8
= 0,
53, 480
71.7
93.6
16, 386
50.3
94.9
wearing lipstick (WL)
0,
1,561
2.1
9.0
12,
35.1
74.7
= 1,
3, 678
4.9
6.4
880
2.7
5.1
1,
15, 841
21.2
91.0
3, 868
11.9
25.3
0,
16, 262
21.8
31.4
6,473
19.9
89.2
0,
3, 257
4.4
14.3
21,333
65.5
84.3
young (Y)
1,
35, 456
47.6
68.6
780
2.4
10.8
1,
19, 585
26.3
85.7
3,968
12.2
15.7
0,
53, 664
72.0
74.7
26, 758
82.2
85.8
0,
1,377
1.8
49.9
1, 048
3.2
76.1
pale skin (PS)
1,
18, 139
24.3
25.3
4,418
13.6
14.2
1,
1,380
1.9
50.1
330
1.0
23.9
Waterbirds. Waterbirds is artificial dataset generated based on the code from (Sagawa al. 2020a. Similar to the CelebA dataset, the training set of Waterbirds suffers from both high class imbalance and significant spurious correlations, with waterbirds typically appcaring in water backgrounds and landbirds in land backgrounds. Interestingly, the validation and test sets of Waterbirds are balanced in terms of these spurious correlations, with equal numbers of waterbirds and landbirds appearing in both land and water backgrounds. Therefore, we expect that all methods tuning hyper-parameters on the validation set will have access distribution information reflective of the test set, leading high overall performance and improved worst-group accuracy. We show details number of samples in each group in Tab.
21

---

Published in Transactions on Machine Learning Research (03/2025)
Table 8: Distribution of samples in dataset Waterbirds and ISIC. Gray color highlights the minority group. Waterbirds has similar distribution between validation and test sets, and they are both spurious- free within a class (i.e,, equal number of samples with land water background). In ISIC, we construct a group balanced validation set for the purpose of training DFR which requires held-out group-balanced dataset.
Dataset
Groups
Train
Validation
Test
waterbird on water
1057
133
642
waterbird on land
56
133
642
Waterbirds
landbird on water
184
466
2255
landbird on land
3498
467
2255
Benign with patch
5526
60
2763
Benign without patch
6314
60
3158
ISCC
Malignant with patch
0
60
821
Malignant without patch
1571
821
A.2 Trsk-oriented Subnetworks
Csordás al. (2020) aims to highlight sets of non-shared weights solely responsible for individual class k by training mask with class k removed from the training dataset (Drask = D {(z, = k}) and need not add any other loss term. The weights solely responsible for this class will be absent from the resulting mask. Results show that the performance the target class drops significantly, while only a small drop in performance is observed for non-target classes. This indicates a heavy reliance on class-exclusive, non-shared weights in the feature detectors. Interestingly, [Sorráss al. (2020) also shows the misclassification behaviors based on 'shared features' (or possibly spurious features), e.g., when "airplane' removed, images are often misclassified "birds" or "ships," both of which commonly feature blue background.
Zhang et al. 2021 investigate the capability of finding subnetwork for a specific task within a trained dense network by extracting it using specially designed held-out dataset tailored for that task. Specifically, the authors train a dense network on the Colored MNIST dataset, which contains strong spurious correlation between digits and colors, with each digit being strongly correlated with a particular color. Using the same technique as Csordás et al. (2020), but while pruning, Zhang et al. (2021) train the network with a dataset balanced in terms of colors and digits to isolate subnetworks that solely predict either digits or colors. Surprisingly, even though the trained dense network suffers significantly from spurious correlations, it is still possible to extract a subnetwork that predicts only the digits with high accuracy without extensive retraining on a balanced (or non-spurious) dataset.
Park et al. 2023) construct the mask training data by upweighting the complex cases (called by bias-conflict samples) defined by the misclassified cases of the ERM model, aiming to extract a of weights more robust. those cases, therefore climinating bias in the network.
A.3 Multiple spurious attributes
CclebA - Dataset contains multiple spurious attributes. We analyze 7 chosen attributes having annotations from dataset, namely: male, no beard, heavy makeup, wearing lipstick, young, pale skin, big nose. Treating each attribute independently, we show their potential of being spurious attribute by comparing the worst-group accuracy from set set G ((blond, a,), (non-blond, a;), (blond, non-a;), (non-blond, non-a;)) for each attribute a, and the ERM average accuracy. A fair model should treat all groups equally and minimize the influence of distribution. As shown in Tab. (ef. ERM), we see some gaps between worst-group and average accuracy, and at the same time, the unequal accuracy among groups.
Model performance across multiple spurious attributes. Tab. 9 shows more details on the accuracies of each group and compares among more methods, namely ERM, DCWP, GROUPDRO, DFR, and PRUSC. Interestingly when considering the group having the worst performance by ERM, our method outperforms
22

---

Published in Transactions on Machine Learning Research (03/2025)
all others by considerable margins. We argue that cluster sampling stage can find hard cases effectively than methods using ERM failures such as DCWP.
Table 9: Details of group accuracy (%) with respect to multiple attributes in CelebA datasct. Average accuracy of baseline ResNet18 is 89.4%. After pruning, subnetwork has keep-ratio 45.5% of ResNet18 weights. We color the column corresponding to the minority group with respect to the combination of target and spurious attribute.
T = blond hair (T-0, A=0)
(T-0, A=1)
(T-1, A=0)
(T-1, A=1)
ERM
78.5
98.9
98.9
50.2
DCWP
75.9
92.7
97.0
81.8
male
PruSC
90.5
90.8
89.6
91.2
GROUPDRO
88.5
91.1
93.5
88.8
DFR
84.9
85.6
94.9
85.6
ERM
99.4
85.7
37.5
96.7
DCWP
93.9
81.6
78.1
96.4
no beard
PruSC
92.3
89.6
92.2
89.7
GROUPDRO
95.8
89.1
85.9
94.9
DFR
90.4
82.4
78.1
96.6
ERM
92.6
79.6
88.9
99.3
DCWP
87.4
76.9
92.7
97.8
heavy makeup
PruSC
88.9
92.7
89.2
89.9
GROUPDRO
89.4
90.6
91.5
94.1
DFR
84.4
83.1
93.8
97.6
ERM
94.6
79.4
82.1
99.1
DCWP
88.9
76.9
90.0
97.5
wearing lipstick
PruSC
88.7
92.2
89.8
89.7
GROUPDRO
89.6
89.9
90.0
94.0
DFR
84.9
82.6
91.8
97.4
ERM
90.9
87.5
92.1
96.7
DCWP
84.7
83.7
95.3
96.3
young
PruSC
81.4
92.8
94.1
88.8
GROUPDRO
84.5
91.3
95.1
92.9
DFR
79.1
85.4
96.0
96.4
ERM
88.5
83.5
95.8
97.9
DCWP
84.1
79.2
96.2
95.2
pale skin
PruSC
90.2
88.7
89.6
90.6
GROUPDRO
89.8
87.5
93.3
92.7
DFR
84.0
81.9
96.4
96.1
ERM
85.8
95.3
96.7
86.9
DCWP
81.3
91.4
96.2
94.4
big nose
PruSC
90.4
89.4
89.4
93.3
GROUPDRO
89.2
91.3
93.1
94.7
DFR
82.8
87.2
96.3
96.9
A.4 Feature Visualization
23

---

Published in Transactions on Machine Learning Research (03/2025)
Figure 6: Embedding space of ERM (first row), DCWP (second row), and PRUSC (third row) CelebA for predicting blond hair color. We visualize the train set embedding space with colors representing the class of non-blond hair (left column) and three other attributes: female, young, and beard (center column). To show how the approach works on test set, we visualize the class non-blond hair of test set, coloring above spurious attributes (right column). In both train and test embedding, ERM clusters well spurious attributes even those attribute labels are not used during training; showing its priority to learn spurious features. DCWP leveraging the ERM-misclassífied cases can not miligate the influence of attribute female, shown by a high purity cluster of samples with attribute female in both train and test representation space. Our pruned model mixes up samples in the same class, climinating the biased learning behavior.
24

---

Published in Transactions on Machine Learning Research (03/2025)
A.5 PruSC with (pseudo) Group Labels
In this section, we study the effectiveness of PRUSC with annotated spurious altributes and other pseudo group labeling approaches such as using minority and majority groups based on ERM predictions Nam et al. (2020): Park et al. )0223) Zhang ct al. 2022).
We conduct experiments on the CelebA dataset using the same frozen ERM model ʃ trained as defined Sec. [.3 The saper-parameters remain identical to those used for training the original PRUSC, Regarding the changes, first, we do not perform representation clustering and sub-sampling. Instead, we design Drask to be (pseudo) group-balanced and fine-tune the model after pruning with this Drask- Second, the contrastive loss defined during the binary mask training stage remains the same, but with the new clusters definition, as described in the following:
Annotated group labels. Each cluster defined based on the spurious attribute (gender), resulting in exactly two clusters for CclebA. We sample an equal number of data points from each group G=yxx A in the training sel, creating group-balanced Drask-
Pscudo labels by ERM predictions. Following previous work (Nam et al. 2020 Park et al. 2023
Zhang et al. 2022,, we track the ERM predictions on the training using an early epoch checkpoint during the training process (early-stopped ERM). Samples misclassified by the early-stopped ERM form the minority group, which is assumed to be less influenced by spurious correlations, while the remaining samples form the majority group. We construct Drask by sampling an equal number of data points from each combination of groups, {minority, majority] x y. There are exactly two clusters defined by the minority and majority groups, and Drask is also constructed class balanced.
Table 10:
Details of group accuracy (%) in CelebA dataset when applying PRUSC with different defined clustering methods for constructing Dtask. Average accuracy of baseline ResNet18 is 89.4%. After pruning, subnetworks has keep-ratio around 50% of R8sNet18 weights.
91
G2
G3
94
Baselines
ERM GROUPDRO (fine-tuning on balanced subset)
78.3
98.9
98.9
50.2
88.5
91.1
93.5
88.8
PRUSC's contrastive loss with pseudo-labels
Annotated group labels
84.8
90.4
94.8
87.7
ERM predictions
68.7
92.3
91.4
64.9
PRUSC
90.5
90.8
89.6
91.2
Table shows the behavior of our subnetwork extraction method with different clustering definitions. The results show that our method can be used in combination with wide range of clustering methods, all of which improve the worst group accuracy. In particular, the contrastive loss is specifically designed to push apart samples with identical spurious attributes, i.e., samples grouped within the same cluster. Consequently, clusters defined by ERM predictions show limited improvement, as ERM misclassifications tend to divide the dataset. into broad minority and majorily groups, failing to effectively distinguish between different. spurious attributes. On the other hand, clusters defined by ground truth group labels significantly increase the worst-group accuracy (50.2% 87.7%). Furthermore, the subnetwork achieves similar group performance when applying GROUPDRO from scratch.
We hypothesize that the success of PRUSC comes from the alignment between the clustering method and the defined contrastive loss. The contrastive loss aims at reshaping the latent space by pulling and pushing samples according to specific criteria. In this context, defining positive and negative samples by representation distances poses the best choice as it directly reinforces the representation structure learned through the contrastive objective. Meanwhile, other sampling approaches that do not use the representation distance information could disrupt this alignment.
25


Reviews:
[{"review_id": "jYcl7W4rbW", "reviewer": "Reviewer_Syh8", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths\n=======\nThe method is clearly described and generally well motivated, and it makes intuitive sense. It is significant that it does not require annotating which instances possess the \"spurious attribute,\" because as pointed out in the paper, the features that a neural network learns to rely on that do not generalize well may not be legible to humans, so annotation may not be feasible. The experiments are thorough and appropriate, and they show the proposed method strongly outperforming other annotation-free methods while performing comparably to methods that require annotation. Overall, this appears to be a useful method and a valuable contribution to the literature.\n\n\nWeaknesses\n==========\nWhile the method is clear, the nature and scope of the problem it aims to address are not fully clear. Specifically, I'm not totally sure what the authors mean by \"spurious correlation\" or related phrases like \"spurious attribute\" and \"spurious feature.\" In the abstract, a spurious correlation is defined as \"features that have strong correlations with class labels but no causal relationship,\" while in the problem setting in section 3.1 it's defined as different groups having \"different levels of bias correlated to an input feature of x and the label of y,\" which I don't understand.\n\nSpurious correlation is a term that can potentially take on somewhat different meanings, and it's not clear what's intended here. For example, the motivating example in Figure 1 describes feature x1 (on the x-axis) as a spurious feature, while the correlation between being female and being blonde in the CelebA dataset is also described as spurious. But in CelebA, it's possible in principle to classify the blondness of images correctly without knowing whether the faces are male or female, whereas in Figure 1, x1 is not an optional feature; the other feature x2 is insufficient to separate the classes. So I'm not sure in what sense x1 is spurious, or whether it's the same sense in which being male/female is spurious in CelebA.\n\nOverall, I found the Intro somewhat imprecise and muddled. I think it's probably more accessible to readers who are already familiar with the related literature, but it would be helpful to make clear up front what it meant by terms like \"shortcuts\" and what the groups are that are referred to in terms of group labels/annotations. These things only become somewhat clear later in the paper.\n\nRelatedly, while I find Figure 3 very clear and helpful, I wonder when this scenario will and won't occur. For example, is it necessarily the case that decision boundaries cut cleanly in between clusters? Is it possible to have clusters in feature space along a spurious feature axis but have the decision boundary determined primarily by non-spurious features?\n\nOverall, while I feel reasonably confident that the method is valuable, I think the paper would benefit from more precise discussion about the problem it aims to address. After all, the presence of water is pretty useful for identifying water birds vs. land birds, and many people would say that these things are causally related, even if the presence of water doesn't directly cause a bird to be a water bird.\n\nOne other major point: how is k chosen in the k-means clustering? Did I miss this somewhere?", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper addresses the problem of neural networks being prone to relying on spurious correlations, such as when a neural network learns to distinguish images of water birds vs. land birds based on whether there is water or land in the background, rather than features of the birds themselves. Give a pre-trained dense neural network which has learned spurious associations, the authors propose a method to extract a subnetwork which relies less on the spurious associations to predict a label. A noteworthy feature of their method is that it does not require annotations with respect to spurious features, unlike many methods in the literature. In experiments, their method substantially outperforms other annotation-free methods and performs comparably to methods which require annotations.\n\nAs a caveat to this review, I have little expertise in the literature on reducing learning of spurious correlations in neural networks, so I am taking the authors at their word regarding the novelty of their method.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "- A more precise definition of spurious correlation, and more careful use of related language like spurious feature, spurious attribute, and shortcut.\n- In section 4, the description of $\\Pi$ doesn't quite make sense. The $\\pi$ values are described as Bernoulli random variables, logits, and probabilities, which are all different.", "broader_impact_concerns": "None", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "L4HDPebjvB", "reviewer": "Reviewer_8JeD", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Pros:\n- The paper is well-motivated, addressing the challenge of group annotation costs, and is well-organized.\n- The experiments are comprehensive, covering multiple spurious correlations, and analyzing various hyper-parameters.\n- The paper begins with clear hypotheses, which guide the reader logically to straightforward solutions.\n\nCons:\n\n**Limited Novelty**\n- My main concern lies in the limited novelty of the paper. The two hypotheses—(a) subnetworks are more robust to spurious correlations, and (b) spurious features tend to cluster more closely than class labels—are not entirely new.\n- Hypothesis (a) has already been demonstrated by [1], while hypothesis (b) has been thoroughly explored in disentanglement, clustering, and domain-aware contrastive learning approaches (see [2,3,4,5]). Thus, the methodology largely builds on existing high-level intuitions from previous work.\n\n**Missing clarification**\n- There is a lack of clarity regarding the importance of sub-sampling. The authors introduce this concept in Section 5.2.2 without fully explaining its purpose. Several questions arise:\n   1. Why not simply fine-tune on the original dataset $D$? \n   2. For $D_{task}$ which contains hard cases, why is subnet masking/FT necessary? Could we not just fine-tune the ERM without subnet masking? This choice is not adequately justified, and no ablation experiments address it. Is this approach simply adopted from the debiasing idea in [1]?\n- Table 5 raises additional concerns:\n    1. I’m surprised that setting 5 performs worse than settings 6 and 7. Could the authors explain this discrepancy? Does the contrastive learning approach only work effectively when paired with subnet fine-tuning?\n    2. Why does the vanilla subnet fine-tuning in settings 3, 6, and 7 perform significantly worse than full-model fine-tuning? This seems to contradict the assumptions in Figure 1.\n    3. Why is there such a large difference in performance between training with and without $ConLoss_{FT}$  (settings 1 and 2)? How exactly does $ConLoss_{FT}$ differ from standard fine-tuning? The variation across the different combinations of modules seems excessive, making the results harder to interpret.\n \n\n\n- [1] Training Debiased Subnetworks with Contrastive Weight Pruning\n- [2] SelfReg: Self-supervised Contrastive Regularization for Domain Generalization\n- [3] Domain Generalization via Contrastive Causal Learning\n- [4] Feature Stylization and Domain-aware Contrastive Learning for Domain Generalization\n- [5] Domain Generalization Using a Mixture of Multiple Latent Domains", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper tackles the problem of OOD generalization problem on datasets with spurious correlation. The major contributions are presented as follows:\n- It is motivated by the cost of group annotations for spurious samples and authors propose an annotation-free framework to improve robustness to spurious correlation.\n- They have two hypothesis: (a) masked subnet is more robust to spurious correlation, (b) ERM is prone to end up with a spurious-specific space than class-specific. \n- Authors propose to extract a subnet for finetuning and also reduce spurious correlation by spurious-aware contrastive learning with clustering. \n- Experiments show large improvement on multiple spurious correlations.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "Please refer to the weakness", "broader_impact_concerns": "N/A", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "VggXS4u7hF", "reviewer": "Reviewer_54Z3", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "## Strengths:\n\n- The proposed method does not rely on group-annotations, while still\n  performing on par with methods that do need group-annotations.\n- The authors motivate their method extensively and inform the reader about the\n  thought process that went into developing all the components of their method.\n  This was greatly appreciated!\n- The authors perform clear experiments which show the usefulness of their\n  method. The additional ablation analysis also further strengthens their\n  results.\n- The code that is provided is clearly written and allows for easy reproduction of the results.\n\n## Weaknesses:\n\n- The motivation that is given is mostly intuitive and although the empirical\n  speak for themselves, a more theoretical justification would have been\n  appreciated. \n- A minor weakness is that the method relies on setting the number of clusters\n  by hand. This is an additional hyperparameter and can probably be tuned using \n  cross validation approach. An automated approach would be even better, but as this \n  paper introduces a new approach I see this only as a minor weakness.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "## Main contribution\nThe authors propose a new method for the training of neural networks such that\nthe network does not rely on spurious correlations. Networks that do rely on\nthese spurious correlations often have poor generalizability. Reducing the\nreliance on spurious correlations would thus result in better performance.\n\nCurrent methods that try to solve the same problem rely on the presence of\ngroup labels. However acquiring these labels can be costly or requires\nknowledge of what spurious correlations could exist. The method that the author\npropose circumvents this reliance on group labels and requires no ground-truth\ngroup annotations. The authors clearly motivate their approach and shed light\non their thought process when developing this method. They also supply\nextensive background information of previous proposals and the different methods\nthey have employed when developing their method. \n\nThe experiments and the code provided\nshow the performance of their method. The results show that their method\nbeats several other baseline\nmethods on standard datasets in the field. This method looks like a valuable\ncontribution to battling the problem of spurious correlations in\nclassification. Finally, the authors provide an ablation analysis showing \nthat each of the components in their method actually contributes to the performance. \n\n## The method\nThe authors summarise their method in 3 steps. Given a trained network $f$ the\nfollowing steps are performed:\n\n1. The representations of the penultimate layer are calculated and clustered.\nThese clusters are labelled as $i$-dominant or neutral, depending on if class\n$i$ is presented more than $90\\%$ of the time in that cluster. From these\nrepresentations and cluster labels, a class-balanced dataset is sampled which\nshould result in a dataset that is $10\\%$ the number of samples of the original\ndataset\n\n2. The weights of the network are frozen and a binary mask over the original\nweights is learned. This binary mask is learned through a contrastive learning\napproach, while trying to maintain a good performance on the original task\n\n3. After the mask is learned, the weights of the subnetwork are again finetuned\nusing the original dataset and a cross-entropy loss. \n\n## Experiments\n\nThe authors provided experiments on 3 datasets: The CelebA dataset, the\nWaterbirds dataset and the ISIC Skin dataset. Their method is compared with 2\ntypes of baseline models: 2 methods that require group annotations and 2\nmethods that do not need group annotations. \n\nVarious different metrics are reported to measure the performance with respect\nto different goals. The reliance on spurious correlations is measured by the\nworst-group accuracy (WGA), the performance on the smallest group is measured\nby minority-group accuracy (MGA) and the group fairness is measured by the\nUnbiased Acurracy (UAG). \n\nLooking at the WGA results. It can be see that the method of the author\nperforms best amongst all the methods that do not need any group annotations.\nThe methods that do need group annotations perform better in some cases only. \n\nThe method by the author also seems to perform the best amongst their baselines\nwith respect to the MGA metric \n\nFinally, the authors show that their method also performs well using the UAG\nmetric.\n\n## Ablation\n\nIn the ablation study the authors systematically analyse all the components\nof the their method. Specifically, they show that the pruning, contrastive learning\nand finetuning after pruning all improve the WGA AVG and UA metrics. \n\nFurthermore, they investigate the influence of their samplings scheme for the \npositive and negative samples in the contrastive learning step. \n\nFinally, it is shown empirically that the spurious features are mixed using \nt-SNE visualization of the representations in the penultimate layer.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "# Requested Changes\n\n- In the experiment section, the WGA scores are reported over multiple runs using the mean \n  and standard deviation, while the\n  MGA and UAG scores are only reported using single runs. It would be good to see\n  the results of these metrics over multiple runs as well and with the mean and standard deviation.\n\n- There is a related work section on Modular Networks and a background section\n  on Subnetwork extractions. In my opinion, these can be merged.  \n\n## Minor changes\n\n- The acronym ERM is used without introduction, but later specified in Section\n  6. Either specify it at the start or not at all.\n\n- In Table $4$ the methods are suddenly along the columns, while before they\n  are along the rows. It would be more consistent to have the methods still as\n  rows and the attributes along the columns.", "broader_impact_concerns": "I believe no broader Impact concern is needed.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper proposes PruSC, a novel method for mitigating the effect of spurious correlations in image classification without relying on group annotations. In response to reviewer feedback, the authors improved their definitions and terminology (e.g., of spurious correlation, feature, attribute, shortcut, etc.); clarified the purpose and importance of sub-sampling in the method; improved table 5; linked to some relevant theoretical work; conducted additional experiments and ablation studies. While some concerns remain, the paper addresses an important and clearly stated problem in machine learning which the reviewers believe is of interest to TMLR audience, and contains a thorough empirical evaluation of the proposed method with fairly strong results, thus warranting acceptance of this paper.

Below I copying reviewer comments. I highly recommend for the authors to see whether these additional experiments and edits can be added to the final version of the paper: (1) Consider adding experiments assessing the impact of pseudo-labeled group annotations (e.g., comparing the proposed approach with ground truth group annotations or alternative pseudo-labeling methods); (2) Provide more convincing evidence for the benefits of contrastive learning, particularly when combined with pseudo-group annotations. This could involve additional experiments or clearer analysis of existing results.
Decision: Accept (poster)