Severing Spurious Correlations with Data Pruning
Authors: Varun Mulchandani
Jung-Eun Kim
Venue: ICLR 2025 Spotlight (2025)
Keywords: Spurious Correlations
Data Pruning
Primary area: alignment, fairness, safety, privacy, and societal considerations

Abstract: Deep neural networks have been shown to learn and rely on spurious correlations present in the data that they are trained on. Reliance on such correlations can cause these networks to malfunction when deployed in the real world, where these correlations may no longer hold. To overcome the learning of and reliance on such correlations, recent studies propose approaches that yield promising results. These works, however, study settings where the strength of the spurious signal is significantly greater than that of the core, invariant signal, making it easier to detect the presence of spurious features in individual training samples and allow for further processing. In this paper, we identify new settings where the strength of the spurious signal is relatively weaker, making it difficult to detect any spurious information while continuing to have catastrophic consequences. We also discover that spurious correlations are learned primarily due to only a handful of all the samples containing the spurious feature and develop a novel data pruning technique that identifies and prunes small subsets of the training data that contain these samples. Our proposed technique does not require inferred domain knowledge, information regarding the sample-wise presence or nature of spurious information, or human intervention. Finally, we show that such data pruning attains state-of-the-art performance on previously studied settings where spurious information is identifiable.

Published as a conference paper at ICLR 2025
SEVERING SPURIOUS CORRELATIONS WITH DATA PRUNING
Varun Mulchandani & Jung-Eun Kim
Department of Computer Science
North Carolina State University
[vmmul cha, jung-eun. kim weimeenccu...m eecss.ecsuuuu
ABSTRACT
Deep neural networks have been shown to learn and rely spurious correlations present in the data that they are trained on. Reliance on such correlations can cause these networks to malfunction when deployed in the real world, where these correlations may no longer hold. To overcome the learning of and reliance on such correlations, recent studies propose approaches that yield promising results. These works, however, study settings where the strength of the spurious signal is significantly greater than that of the core, invariant signal, making it easier detect the presence of spurious features in individual training samples and allow for further processing. this paper, we identify new settings where the strength of the spurious signal is relatively weaker, making it difficult to detect any spurious information while continuing to have catastrophic consequences. We also discover that spurious correlations are learned primarily due to only a handful of all the samples containing the spurious feature and develop a novel data pruning technique that identifies and prunes small subsets of the training data that contain these samples. Our proposed technique does not require inferred domain knowledge, information regarding the sample wise presence nature of spurious information, or human intervention. Finally, we show that such data pruning attains state-of- of- the-art performance on previously studied settings where spurious information is identifiable]
INTRODUCTION
Deep neural networks have shown promising results a variety of benchmarks and applications. However, their reliability and by extension, ability to solve increasingly challenging problems remains questionable these networks have been shown to exhibit several failure modes in practice. Of these, the inability to adapt to distributional shifts due to the reliance on spurious correlations is of strong concern it makes them unreliable for deployment in the real world. Spurious Correlations are correlations that a network learns between simple, weakly predictive spurious features present in network
fraction of the training data and the class label. These correlations may prefer them over strongly predictive invariant correlations when making prediction. Thus, in the event of distribution shift where spurious features either no longer exist or become correlated with a different task, which is common phenomenon in the real world, these networks begin to
problematic
malfunction (Arjovsky al. 2019 Sagawa et al. 2020a Nagarajan et 2021 2020).
Geirhos et
To promote the learning of invariant features, several recent works have proposed techniques that yield promising results. These works heavily rely on sample-wise environment labels, where each class is spuriously correlated with spurious feature from one environment. Such labels are also referred to as group labels. For instance, (Sagawa et al. (2020a) aim to prioritize samples from environments that have high training risk to promote invariant feature learning, with the assumption that such
"Corresponding author.
Code httpr:/twttttrrooo//////o//e///n 2wpur2___purrii ousiaaaPrrunng http://aspprrrouaaaaarrun..

---

Published as a conference paper at ICLR 2025
labels are available. Kirichenko et al. (2023) work with stronger assumption, where they assume that overrepresented environments contribute to the learning of spurious correlations and simply re-train the last layer of a spuriously biased network
dataset where all environments represented. Such solution makes strong assumptions regarding which samples contain spurious features associated with each class. |Maayer et al. (2023) follow similar technique of utilizing human supervision to determine the strength of spurious signals in each training sample and fine- tune-tune biased network on samples where spurious signals are weaker. Attaining sample-wise environment labels and information regarding sample-wing wisp presence of spurious features during training, however, unrealistic. To overcome this problem, Liu et al. (202I); Zhang et al. (2022); Pezeshki al. (2024) aim to infer the sample-wise presence of spurious features based on biased network's s outputs and they follow this with further processing such as up- upiweigg of samples without spurious features or aligning representations of samples with and without spurious features, typically at the cost of overall testing accuracy.
equally
All past works that study spurious correlations and aim to promote invariant feature learning perform experiments on settings where the strength of the spurious signal is significantly higher than the core, invariant signal. This makes easy to identify the presence of spurious features in each sample within the training set, which is, however, not always the case. In this paper, we aim to tackle novel settings where the strength of the spurious signal is relatively weaker, making it impossible identify such information while rendering existing techniques ineffective. We then show that spurious correlations are learned primarily due few key samples the training set and present novel data pruning technique to identify and prune a small subset of the training data that contains these samples. Finally, we show that such data pruning attains state-of-the-an performance on previously studied settings where information regarding the sample-wist presence of spurious features is easily identifiable. We summarize our contributions below:
Contributions
We identify settings where is impossibleddificult to identify sample-wise presence of spurious features. This renders past approaches that mitigate spurious correlations as ineffective.
We discover that spurious correlations are learned primarily due to handful of all the samples containing spurious features, through extensive empirical investigation. Based on this insight, we propose simple and novel data pruning technique that identifies and prunes small subset of the data that contains these samples.
We show that such data pruning attains state-of-the-art results even in previously studied settings where information regarding the sample-wise presence of spurious features is identifiable.
2 BACKGROUND AND RELATED WORK
Consistent with past literature, we study the supervised classification setting where S = {I,, ) denotes the training dataset of size N and network is trained to learn mapping between x; (input) and y; (class label) using empirical risk minimization (Vapnik) 1998). Every training sample E S contains a core feature (c:) that represents its class (y;). A fraction of all samples within class contain the spurious feature (a:) associated with that class. Core (or invariant) features represent the class label y/ and are semantically relevant to the task. They are also fully predictive of the task, as they are present in all samples. Spurious features do not represent the class labels and are semantically irrelevant to the task.
Spurious Correlations. The correlations a network learns between spurious features and class labels. Such correlations are undesirable as they can disappear during testing or become associated with different task, causing these networks to malfunction (Arjovsky et al. [2019] Nagarajan et al.) 2021 Shah et al. 2020 Tsipras et al.) 2019 Sagawa et al. 2020b a Kirichenko et al. 2023 Jain al. 2023 Ye et al. 2022). To make a network robust against spurious correlations, Sagawa et al. (2020a) suggest using information regarding sample-we wise environment labels during training. Since such information is often unavailable, more recent approaches attempt to infer group or spurious information with the help of an ERM-trained model, followed by subsequent sample up- ue-weighting
class-wise representational alignment (Sohoni al. 2020
Liu al. 2021 Zhang al. 2022
2

---

Published as a conference paper at ICLR 2025
(1) Female Samples Without Spurlous Feature
(2) Femalle Samples With Spurious Feature
(1) Samples With Spurious Fea
(3) Male Samples Without
(4) Male Samples With
Spurious Feature
Spurious Foature
Samples Without Spurious Feature
Training with no sparious samples 10%)
Training with spurious
Training with spurious samples (50%)
Unstectifiuble Switing IWD-DD1.
Unidentifable Setting
Identi Setting
samples (10%)
164
(MD=5.1 2nd
2nd
Group Tall
(a) Training with just a few Male samples with spurious fea- tures causes heavy reliance spurious features. indicated by low test accuracy on Female samples with Spurious Features.
(b) Training with greater regularization may not help identify samples with the spurious feature.
Figure I: Spurious information is often unattainable.
Ahmed et al.) 2021] Creager et al. 2021 Pezeshki al. 2024). Recent work has also shown that model learns both general and spurious features and just re-training the last layer with balanced data where strong spurious correlations do not hold can improve robustness (Kirichenko et al. (023). Moayeri et al. (2023) perform a similar re-training, where they fine-tune the final layers of trained (biased) models on those samples with minimal spuriosity, where spuriosity is determined with human supervision. Idrissi et al. (2022) train network on a balanced train set where all environments equally represented, thereby promoting invariant feature learning.
Simplicity Bias and Spurious Correlations. Recent work has shown that deep neural networks have a greater affinity towards simpler features than more complicated ones (Shah et al. 2020 Morwani let al., 2023] Tiwari & Shenoy] 2023). [hah al. (2020) show that in the presence of two fully predictive features, a network would choose to fully ignore the more complicated features in favor of the simpler set of features. In settings where the simpler feature is not fully predictive (Spurious), the network still relies strongly on these features (Kirichenko et al. 2023), leading to the learning of and reliance on spurious correlations.
Feature Difficulty. Consistent with deep learning literature (specifically, those works concerned with spurious correlations), difficulty of learning a feature is determined by the following three factors: (1) Proportion (or Frequency) of training samples containing the spurious feature (Sagawa et al., 2020b Shah et al. 2020 Kirichenko et af., (023), (2) Area Occupied and Position (if it is centered or not) in the training sample (Moayeri al. 2022 and (3) The amount of noise in its signal (Sagawa et al. 2020b Ye et al., 2022). A feature which is present in a large portion of all training samples, occupies lot of area, centered, and has little to variance, is easy to learn. On the other hand, feature which is present in a small portion of all training samples, occupies little area, is not centered, and has a lot of noise/variance, is hard to learn.
Our work is concerned with two different widely studied areas in deep learning: out- -ittt--ddittiitti generalization and data pruning.
Data Pruning. A large part of deep learning's 's recent success has been attributed to large datasets that models are trained on. However, with increasing datasets sizes, computational costs have risen significantly. This raises an important question: Are all samples in a dataset equally important for attaining good testing accuracy? In other words, is it possible reduce samples in the training data without impacting generalizability? Recent work has shown that it is possible to remove a large fraction of a dataset without sacrificing test accuracy. Most approaches create sample-wise metrics that measure how important it is to maintain a given sample in the training set and remove the least important samples by their definition (Paul al. 2021 Toneva et al., 2019 Feldman & Zhang 2020
Sorscher et al. 2022 He et al. 2023
While existing works aim to prune samples to reduce computational costs, our work focuses on pruning samples to mitigate the learning of spurious correlations during training. This direction, to the best of our knowledge, novel.
3

---

Published as a conference paper at ICLR 2025
3 SPURIOUS INFORMATION IS OFTEN UNATTAINABLE
Suppose one wants to build a gender classifier using the CelebA dataset (Liu et al. fraction of the training samples in the Male class contain eyeglasses (10%), which takes the form of the spurious feature in our setting. None of the samples in the Female class contain eyeglasses in the training set. Thus, the Male class is spuriously correlated with eyeglasses. Note that in our setting, only a small minority of all Male samples contain the spurious feature, making the strength of the spurious signal relatively weaker. This is different from settings studied in literature, where the spurious feature is present in a majority of the samples (often 95-97% of all samples of that class) and the strength of the spurious signal is significantly higher than the strength of the core, invariant signal. We observe that under normal training conditions without the existence of spurious samples, test accuracy of Female samples with the spurious feature is high as shown in Fig. Ta (Left), Introduction of just few Male samples with the spurious feature is capable of reducing the test accuracy of Female samples with eyeglasses significantly, as shown in Fig. Ta (Middle and Right).
2015). A small
In settings where the strength of the spurious signal is significantly greater than the strength of the core, invariant signal, it becomes easy to differentiate between groups of samples containing spurious features and those that do not (Liu et al.] 2021 Zhang al., 2022 Yang et al. (024). This is because the network relies very strongly on the spurious features during training, making it casy to differentiate between such groups based on representational differences or on network's ease of learning these samples. For instance, Liu et al. (20TT) Zhang et al.] (2022) train network with high regularization (greater weight decay) such that a network correctly classifies samples with easy spurious features early during training and misclassifies those without, as shown in Fig. [b (Right) The identifiable setting shown is the CelebA setting studied Liu et al. (202T). . our setting, however, the spurious feature is not present in a majority of the training samples and so the strength of the spurious signal is relatively weaker. This makes it impossible to identify which samples contain spurious features using past approaches, even when 50% of all male samples contain eyeglasses see Fig. 1b (Left and Middle). Thus, we identify the primary failure mode existing approaches as follows:
* Attaining information regarding the presence of spurious features becomes difficult when the strength of the spurious signal not exceptionally greater than the strength of the core, invariant signal in the training set.
Based on this observation, we propose the following problem statement:
Problem Statement: How does one overcome spurious correlations in settings where attaining spurious information is difficult impossible?
EXPERIMENTAL DESIGN
In this paper, we study both settings: those where information regarding spurious information is identifiable and those where such information is unidentifiable. To support our claims, we study both Vision and Language Tasks. We also move beyond simple binary classification tasks that have been the focus of all recent works that study spurious correlations. Below, we detail the experimental design for this paper. Please note that we provide additional details in the Appendix.
TESTBED
We create a testhed based on the CIFAR-10 dataset (Krizhvsky 2009) where we synthetically introduce spurious features a fraction of one class' (cl) training samples.
CIFAR-10S.
We introduce the same spurious feature in all samples of different class (e2) during testing.
The degree of spurious feature reliance is estimated by measuring the number of samples of
(c2) that are misclassified as (cl) during testing. Since spurious features are synthetically introduced, this setting allows us to:
Vary the strength of the spurious feature relative to the core, invariant feature.
Accurately compute the difficulty of learning the core, invariant feature of individual training samples before the introduction of spurious features.
4

---

Published as a conference paper at ICLR 2025
51 (Hardest Spurious Feature)
$2
53 (Easiest Spurious Feature)
Spurious Misclasoiicatios (98)
100
Easiest Core
100
Hardest Core
Spurious Misclassification (%)
Easiest Core
Spufious Misllasifiiations (%9
Hardest Core
80
8
00-
60
40
20-
20
20
Anne
125
30
20
Epochs (c)
Epochs
(a)
(d)
Figure 2: Introducing spurious features in 100 samples with the easiest core features (Easiest Core) causes little to no reliance on spurious features, indicated by low Spurious Misclassifications Introducing the same spurious features in 100 samples with the hardest core features (Hardest Core) causes heavy reliance on spurious features, indicated by high Spurious Misclassifications.
Note that in this setting, the spurious feature takes the form of a line running through the center of the images of class cl and we vary the strength of the spurious signal by increasing the region the spurious feature covers or by increasing the proportion of samples that contain the spurious feature. We note that in each experimental setting studied, only vary of the two factors.
UNIDENTIFIABLE BENCHMARKS
CelebA. (Luu al. 2015) We utilize the same experimental setting studied in Sec. 3 where we build a gender classifier in which the Male class is spuriously correlated with Eyeglasses The degree of spurious feature reliance is measured based the testing accuracy of Female samples with Eyeglasses.
Hard ImageNet. (Moayeri et al. 2022) The Hard ImageNet dataset is a 15 class classifica- tion task where all classes have spurious feature associated with them and certain classes share similar spurious features. For instance, both Ski and Dog Sled classes contain similar spurious features (Snow, Pecple, Trees, and Hi s)) In our experiments, measure the degree of spurious feature reliance by measuring the numbers of Dog $ ed samples that are misclassified Ski during testing. Note that in our experimental setting, no Skis are misclassified as Dog Sled.
STANDARD BENCHMARKS (IDENTIFIABLE)
Waterbirds. (Sagawa et al. 2020a The Waterbirds task is binary image classification
task where the goal is to classity an image of a bird as landbi rd or waterbs rbb rd. We use the same setting studied in previous works (Sagawa et al. 2020a Kirichenko al.) (023)). where the class landbi rd is spuriously correlated with Land backgrounds while the class waterbiree rd is spuriously correlated with Water backgrounds.
MultiNLI (Williams al., 2018 The MultiNLI task is a classification task with three classes where the goal is to classify the second sentence in a pair of sentences as entailed by, neutral with, or contradicts. Consistent with the setting large fraction of the cont radicts class contains negation words while the other two only contain few samples with negation words, making the cont radict s class spuriously correlated with negation words and the other two with the lack thereof.
Sagawa et al. (2020a).
EVALUATION.
Current practice in deep learning utilizes Worst- Group Accuracy (WGA) to assess the degree of spurious feature reliance in binary classification tasks. WGA computes the accuracy of test samples that contain the spurious feature associated with the other class during training. While suitable for simple binary classification tasks, WGA becomes insufficient asses the reliance on spurious

---

Published as a conference paper at ICLR 2025
features in settings with multiple classes. This is because WGA cannot differentiate between loss in test accuracy due to spurious correlations, or due to lack of learnability of invariant correlations stemming from limited capacity insufficient training data. In such settings, measure the degree of spurious feature reliance through Spurious Misclassffications i.e., the percentage of samples of one class (c2) containing the spurious feature of another class (c1) that misclassified as (cl) during testing. Lower Worst Group Accuracy indicates heavy reliance on spurious correlations while high Worst Group Accuracy indicates little no reliance on spurious correlations. A high number of Spurious Misclassifications indicates heavy reliance on spurious correlations while a low number Spurious Misclassifications indicate little no reliance on spurious correlations.
5 SPURIOUS CORRELATIONS ARE LEARNED FROM A FEW KEY SAMPLES
In this section, we aim to form a deeper understanding of the relationship between spurious correla- tions and standard training paradigms. Deep neural network training relies on the Empirical Risk Minimization (Vapnik) 1998) principle, where during training, the model attempts to minimize risk or loss on the training data. From recent works, we understand that spurious correlations are learned because the network uses these weakly predictive but simple spurious features to further minimize training risk (Arjovsky et al. 2019; Sagawa al.) 2020a; Geirhos et al., 2020; Kirichenko al.. 2023). In this section, we attempt to understand it this simple statement encapsulates the entire relationship between deep neural network training and spurious correlations.
We understand that spurious correlations are learned be- cause standard training paradigms are prone to accepting any features that help minimize risk or loss during training. However, different samples have different contributions to overall training loss due to differences in difficulty of learn- ing the features used to make correct classification. Does this imply that samples containing harder core/invariant features contribute more to the learning of spurious correla- tions? Or do all samples contribute equally to the learning of spurious correlations, as a network might simply prefer simpler feature over a more complex one?
Spurtous Misclassificattoss
80-
60
<0
To answer this question, we study a variant of the CIFAR- IOS setting described in Sec. In this experiment, we first train ResNet20 on standard CIFAR-10. Early in training, we compute training error for each sample to estimate the difficulty of learning these samples, as is done in Paul et al.] (202T). Such sample-wise training error is computed as ()) - vile. where ((wra)) is the probability distribution given by the network for sample x, w denotes the network parameters early in training, and y is the one hot encoding of the ground truth value. We specify the epoch at which these values are computed in the Appendix. Trivially, the greater the error, the more difficult that sample to learn. Since the standard CIFAR- does not contain significant spurious cues that impact difficulty of learning, we estimate the difficulty of learning the core/invariant feature per sample as the diffieulty of learning that sample. Next, we train ResNet20 two settings: One where we introduce the spurious feature in samples with the lowest training error (samples with easy invariant features) and another where we introduce spurious features samples with the highest training error (samples with hard invariant features). We term these settings as Easiest Core and Hardest Core, respectively. In each setting, we vary the strength of the spurious signal by increasing the amount of area it takes up in each image. Spurious feature S1 takes up the least amount of area and is the most difficult to learn while spurious feature S3 takes up the most amount area and the easiest spurious feature to learn. In other words, a network will have less reliance spurious feature S1 as is difficult to learn and more reliance on S3 as it is easier to learn. S2 in between them. Note that in both experimental settings, introduce spurious features in only 100 samples (2% of class cl, 0.2% of the training set), and thus, we maintain the proportion of samples containing the spurious features in all settings.
Split Difficulty
Figure 3: Spurious feature reliance ex- hibits super-linear growth with increas- ing sample difficulty.

---

Published as a conference paper at ICLR 2025
(1) Pruning Easiest
(2) Pruning Hardest
Unpruned Dataset
Higher Better
Higher is Better
100
100
Worst Group Accuracy (9%)
80
Test Accurscy (%)
80
60
60
40
40
20
20
75
97(%)
50
97(9))
Pruned (96)
Pruned (%)
Figure 4: Excluding only a handful of training samples with spurious features and hard core features mitigates spurious correlations in the CelebA setting. This is indicated by high Worst Group Accuracies. Excluding up to 97% of all training samples with spurious features and easy core features shows no improvements in worst group accuracy.
Samples with simple core features do not contribute to spurious correlations. In the setting where we introduce spurious features in samples with low training loss (Easiest. Core), we find that Spurious Misclassifications tend to zero (Fig. _aa).. This is true even if the strength of the spurious signal is increased. More interestingly, the network learns the spurious features present in these samples in the first couple of epochs of training but overcomes them as training converges. This evidenced by high Spurious Misclassifications in the first half of training followed by low Spurious Misclassifications in the second half of training, as shown in Fig, 2c). In other words, the network learns to ignore weakly predictive but simpler spurious features and prefers strongly predictive but complex core features in those samples that contain relatively easier invariant features.
Samples with hard core features are primary contributors to spurious correlations. In the setting where introduce spurious features in samples with high training loss (Hardest Core), we find that Spurious Misclassification are significantly higher, as shown in Fig. 2b). This is also true for spurious features that are harder learn (S1). Unlike what we observe in the previous setting, the network is unable to overcome the spurious features present in these samples, as we observe that Spurious Misclassification either remains the same or increases the second half of training, as shown in Fig. 2 d). It is important to note that although only 100 of all samples of class el contain the spurious feature (2% of class cl, 0.2% of the training set), almost all samples of class c2 (*x 100% of class c2) are spuriously misclassified during testing (Fig. 2b)). This is in sharp contrast to the setting where we introduce the same spurious features in samples with easy invariant features (Easiest Core), where almost none of the samples of class c2 are spuriously misclassified (Fig. aa).
Spurious feature reliance exhibits super-linear growth with increasing sample difficulty. We extend our original experimental setting introduce the spurious feature in 100 samples ranging from the easiest to the hardest, instead of only considering the Easiest and Hardest 100 samples. Intuitively, scale up the difficulty of the 100-sample split to which observe an increase in spurious misclassifications, as shown in Fig. B Interestingly. we observe that spurious feature reliance exhibits super-linear growth we scale up the difficulty the core features of the 100 samples, implying that those samples with hard core features contribute significantly more spurious feature reliance than those with easy core features.
inject the spurious features,
Excluding a few key samples during training severs spurious correlations. From the experiments above, we learn that samples with harder core features contribute far more spurious feature reliance than samples with easy core features. Consider a setting where spurious features are uniformly distributed across samples containing easy and hard core features. such a setting, what if we simply exclude a handful of the hardest training instances that contain spurious features in them? Does this simple exclusion sever the learned spurious correlations that would be heavily relied on by the network? In Fig. 4 we show that by simply excluding few samples containing the spurious feature with hard invariant features in the CelebA setting studied in Sec. 3 (10% of all samples with spurious features in that class, 1 1% of the total train set), we observe significant improvements in Worst Group Accuracy (WGA: Test accuracy of Female samples with spurious features). On the other hand, excluding up to 97% of the easiest samples containing the spurious feature shows
7

---

Published as a conference paper at ICLR 2025
CEAR-125 (eban. 2
CIAA $15 (Wn-derr Siticiingg
Mazzroadda INterttucke
Ocibb Aericerei ablel
1:0
150
56
80-
2
10-
Suve Rerooe
28-
20
20
(535 (1aritec)
9-andes)
(Sases0
(-arde5)
(a) Training distribution variance based strength of the spurious feature in CIFAR-10S Testhed. Grouped by Quartiles, sorted by difficulty.
(b) Training distribution variance based on strength of spurious fcature in identifiable and unidentifialle settings. Grouped by Quartiles, sorted by difficulty.
Figure 5: Impact of strength spurious signal on sample difficulty.
(1) Pruning Easiest
(2) Pruning Hardest
(3) Unpruned Dataset
Higher Better
Higher is Better
80
100
Worst Group Accuracy (99)
70
80
60
Accurssc (98)
50
40
30
40
20
20
10
10
25
40
50
75 (96)
90
971%)
10
25
40
50
90
97(%)
Figure 6: Excluding small fraction of all hardest samples in the dataset mitigates spurious correla- tions in the CelebA setting. This is indicated by high Worst Group Accuracies.
no improvements WGA. Within the pool of samples containing spurious features, difficulty of learning the invariant feature is similar to the difficulty of learning the sample. This is because spurious features in our settings exhibit very low variance. For instance, in the MultiNLI setting, the spurious feature comprises the same set of negation words across the training data. Additionally, note that on pruning these samples, we do not observe significant drops in overall testing accuracies, implying that these samples do not contribute significantly to generalizability either (Fig. 4
SEVERING SPURIOUS CORRELATIONS WITH DATA PRUNING
Based the observations presented above, we develop novel data pruning method sever learned spurious correlations by identifying and pruning subsets of the training data that contain the few key samples that contribute to spurious feature reliance. To do so, we first understand how the presence of spurious features impacts the difficulty of learning sample in the training set and how it impacts the training distribution when observed through the lens of sample difficulty.
6.1 THE IMPACT OF SPURIOUS FEATURES ON TRAINING DISTRIBUTION
Consider the CIFAR-103 setting described in Sec. We create two versions of this setting: Setting 1: the strength of the spurious signal is significantly greater than the strength of the core, invariant signal: Setting 2: the strength the spurious signal is only marginally greater than the strength of its invariant counterpart. In Setting 2, spurious information is unattainableeunideetffiab settings, we introduce spurious features into samples at random such that both samples with easy and hard core features contain spurious features in them. We observe in Setting almost all of the samples that contain the spurious feature lie in the first half of the data distribution when observed through the lens of sample difficulty. In Setting however, samples containing spurious features
In both
8

---

Published as a conference paper at ICLR 2025
distributed uniformly (Fig. Sap. Note that in both settings, spurious misclassifications significant
(58.0% and 13.76% in Setting and Setting 2, respectively).
We extend these findings to the bench- marks discussed in Sec. 4 In settings where the strength of the spurious signal is significantly greater than the strength of the invariant feature, (Ident i fable setti aeg), samples containing the spu- rious feature occupy large majority of the first half of the training distribution. On the other hand, in the unidentifiable bench- marks, samples containing spurious fea- tures are uniformly distributed across the training distribution when viewed through the lens of sample difficulty (Fig. (Fig.5b). Note that since Hard ImageNet has multiple spu- rious features, we show that this setting is unidentifiable by showing images with and without these spurious features spread uni- formly across the difficulty spectrum in the Appendix.
(1) Pruning Easiest
(2) Pruning Hardest
(3) Unpruned Dataset
136)
Better
Higher is Better
35
100
Spurious Misclassffiaatonn
Test Accurary
66
25
20
20-
15
6.66
10 201%)
6.66
10 201%)
Pruned (9)
Pruned (96
Figure 7: Excluding small fraction of all hardest sam- ples in the dataset mitigates spurious correlations in the Hard ImageNet setting. This is indicated by low Spuri- ous Misclassifications.
6.2 DATA PRUNING IN UNIDENTIFIABLE SETTINGS
From Sec. 6.1 we observe that in settings where the strength of the spurious signal is not significantly greater than the strength of the invariant signal, i.e. in settings where attaining information regarding the sample-wist wise presence of spurious features is impossible, samples containing spurious features uniformly distributed across the training distribution when observed through sample difficulty. In other words, the presence of spurious features does not have a significant impact on the training distribution. We also know that samples containing hard core features that also contain the spurious feature are primary contributors to the learming of spurious correlations. Thus, to mitigate spurious correlations without knowing which samples have spurious features in them, one would only have to prune the hardest samples in the training set, as this subset of the data would contain samples with spurious features that have hard core features.
In Figs. aand we show that by simply pruning a small subset of the hardest samples in the training set, one can overcome spurious correlations. Note that instead of pruning the hardest samples globally, we prune equal proportion of the hardest samples per class to account for differences in difficulty per class. Most importantly, we do not perform any ceeerprraamter-tuning Gulrajani & Lopez- Paz (202T) demonstrate that state-of-the-art techniques fail without typerparameter-tuniig with the help of a validation split that mimics test-time distribution. Our data pruning technique shows promising results without any hyperparameter tuning.
Table 1: We present Worst Group and Mean accuracies. Consistent with literature, Mean accuracy is reported by weighting groups based on their prevalence in the unpruned training dataset. Simple data pruning attains of-tc-o-thhearr performance on standard benchmarks (Identifiable Settings). The group labels column represents the availability of group labels in training and validation sets.
Waterbirds (%)
MultiNLI (%)
Group Labels
Method
Worst%
Mcan
Worst %
Mean%
Train
Val
FRM
74.81 (0.7)
98.10 10 (0.1)
65.9 (0.3)
82.8 (0.1)
x
X
CnC /hhng et al.
2022
88.5 (0.3)
90.9 (0.1)
x
JTT Liu et al.
2021
86.7
93.3
72.6
78.6
x
gDRO
Sagawa
2020a
86.0
93.2
77.7
81.4
/
"t
Kirichenko ct al. 2023
90.2 (0.8)
97.0 (0.3)
71.5 (0.6)
82.5 (0.2)
PDE
Deng el al.
90.3 (0.3)
92.4 (0.8)
Ours
90.93 (0.58)
92.48 (0.72)
75.88 (1.62)
81.07 (0.25)

---

Published as a conference paper at ICLR 2025
6.3 DATA PRUNING IN IDENTIFIABLE SETTINGS
Unlike in unidentifiable settings, in settings where the strength of the spu- rious signal is significantly greater than the strength of the invariant sig- nal, samples with spurious features are not uniformly distributed when sorted by sample difficulty. This is be- cause the presence strong spurious
Waterbirds
MultiNLI
100
100
5OTA
Worst Group Accuracy (%)
90
Worst Group Accuracy
80
80
ERM
70
70-
ERM
information enables the network to have lower training errors for samples with hard core features and spurious features. In such settings, is not fea- sible simply prune the hardest sam- ples of the training data as this will primarily prune samples that do not contain the spurious features (Figs. [5a] & (5). Liu al.[ (202T), y; Zhang et al. (2022) Yang et al. (2024) show that in settings where the strength of the spurious signal is significantly greater than the strength of the invariant sig- nal, it is possible to identify which samples and groups contain spurious features in them and which ones do not. For such identifiable settings, our approach works with group labels as is done in Kirichenko et al.] (2023); Deng et al. (2023) and simply prunes those samples containing the hardest core features within groups containing the spurious feature associated with each class. As shown in Table [] we attain state-of-the-ar performances. Note that in settings with heavy class imbalances, such as Waterbirds, we prune such that the total number of samples in both classes is the same. We also compare our method with two popular techniques that make use of inferred sample-wise spurious information during training, JTT (Liu et al.) (02T) and CnC (Zhang et al. 2022). It is interesting to note that such data pruning attains ofate-of-the-art performances across wide range of pruning sparsities, as observed in Fig. [ This suggests, in addition to all previously observed results, that if one wants to ensure that the model they obtain after training is robust to spurious correlations, they only need to remove few training instances that are hard for the network to understand. This is in contrast to existing methods that mitigate spurious correlations
60
40
40
75(%)
33.33(%))
Prune (%)
Prune (%)
Figure 8: Our data pruning approach is effective across wide range of sparsities. Sparsities reported before class bal- ancing, if applicable. We report SOTA results for Waterbirds and MultiNLI from Kirichenko al. (2023) and (Sagawa et al.) (2020a). respectively. Note that Kirichenko et al. (2023) ob- tain state-of-the-art results on Waterbirds with the help of held-out group balanced dataset (DFR Vol) and thus, exclude it from our comparison in Table
which are more complex and at times, computationally expensive (Sagawa et al., 2020b Ahmed
et al. 2021 Ye et al. 2022 Zhang et al.) 2022) Kirichenko et al. 2023 | M3ayeri al. 2023 Deng
et al. 2023
7 CONCLUSION
Summary. We have shown that, in practice, extracting domain knowledge and information regarding the presence and nature of spurious features is often difficult. This renders all existing techniques that show promise in overcoming spurious correlations as ineffective. We also discover that spurious correlations are formed due to only small fraction of all samples containing the spurious feature and develop novel data pruning technique that overcomes spurious correlations by pruning a small subset of the training data that contains these samples. Finally, we show that such data pruning attains state-of-the-art performance on standard benchmarks where information regarding spurious features easily available.
Outlook. With current practice of training models on increasing large datasets, it has become critical to develop techniques that identify samples that can cause these models to malfunction without requiring human intervention. In the future, we will further develop this work to take into account other failure modes that are commonly exhibited by these deep neural networks.
10

---

Published as a conference paper at ICLR 2025
REFERENCES
Faruk Ahmed, Yoshua Bengio, Harm van Seijen, and Aaron C. Courville. Systematic generalisation with group invariant predictions. In 9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021. OpenReview.net, 2021.
Martín Arjovsky, Léon Bottou, Ishaan Gulrajani, and David Lopez-Paz. Invariant risk minimization. CoRR, abs/1907. 009900079.9... 2019. htttht/p//aaat
wwwgg/aba/99gg/ 02893
Elliot Creager, Jörn- HenriK Jacobsen, and Richard S. Zemel. Environment inference for invariant learning. In Marina Meila and Tong Zhang (eds.), Proceedings of the 38th International Conference on Machine Learning, ICML 2021, 18-24 July 2021, Virtual Event, volume 139 of Proceedings of Machine Learning Research, pp. 2189 2200. PMLR, 2021.
Yihe Deng, Yu Yang. Baharan Mirzasoleiman, and Quanquan Gu. Robust learning with progressive data expansion against spurious correlation. In Alice Oh, Tristan Naumann, Amir Globerson, Kate Saenko, Moritz Hardt, and Sergey Levine (eds.), Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, NeurlPS 2023, New Orleans, LA, USA, December 10 16. 2023, 2023. URL http::/papers wwwp:iwwwssss.ppeeeemme wwtp:cpppppppppeeeeeeeeeeee.eeee fi les /2tl1l/e/aapee//23///////// /aah//////////// 0506ad3d1bccc3 4 ott.a:aceeececcooooooooroaaooaa-
htm
Vitaly Feldman and Chiyuan Zhang. What neural networks memorize and why: Discovering the long tail via influence estimation. In Hugo Larochelle, Mare' Aurelio Ranzato, Raia Hadsell, Maria-Florina Balcan, and Hsuan- Asuan-Tien Lin (eds.), Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020, December 6-12, 2020, virtual, 2020.
Robert Geirhos, Jörn- Moon-Herrik Jacobsen, Claudio Michaelis, Richard Zemel, Wieland Brendel, Matthias Bethge, and Felix A Wichmann. Shortcut learning in deep neural networks. Nature Machine Intelligence, 2(1 11:66))-6:575 2020.
Ishaan Gulrajami and David Lopez-Paz. In search of lost domain generalization. In International Conference on Learning Representations, 2021. URL
forum?
id=1odXeXDoot.
Muyang He, Shuo Yang. Tiejun Huang, and Bo Zhao. Large-scale dataset pruning with dynamic uncertainty. CoRR, abs/2306.05175, 2023, doi: 1048500ARXXIV22360055777 URL https
7/doi oro/1o 48550 2306. 051775
Badr Youbi Idrissi, Martin Arjovsky, Mohammad Pezeshki, and David Lopez-Paz. Simple data balaneing achieves competitive worst-group-accuracy In Bernhard Schölkopf, Caroline Uhler, and Kun Zhang (eds.), Proceedings of the First Conference on Causal Learning and Reasoning, volume 177 of Proceedings of Machine Learning Research, pp. 336-351, PMLR, 11-13 Apr 2022.
URL https: /ttppp:://rrooo htt.:::://ppcceeedngg. www.sssww.........ee....e.meeeee
mlr www.pprssspeeeee paress/v177/idrr/
html
Saachi Jain, Hannah Lawrence, Ankur Moitra, and Aleksander Madry. Distilling model failures as directions in latent space. In The Eleventh International Conference on Learning Representations, 2023.
n wor.mmid=:99pB///2
Polina Kirichenko, Pavel Izmailov, and Andrew Gordon Wilson. Last layer re-training is sufficient for robustness spurious correlations. In The Eleventh International Conference on Learning Representations, ICLR 2023, Kigali, Rwanda, May 1-5, 2023. OpenReview.net, 2023. URL https::/openzeviiet net forum?id=2b6c8A-
Alex Krizhevsky. Learning multiple layers of features from tiny images. 2009.
Evan Zheran Liu, Behzad Haghgoo, Annie S. Chen, Aditi Raghunathan, Pang Wei Koh, Shiori Sagawa, Percy Liang, and Chelsea Finn. Just train twice: Improving group robustness without training group information. In Marina Meila and Tong Zhang (eds.), Proceedings of the 38th International Conference on Machine Learning, ICML 2021, 18-24 July 2021, Virtual Event, volume 139 of Proceedings of Machine Learning Research, pp. 6781-6792. PMLR, 2021.

---

Published as a conference paper at ICLR 2025
Zhuang Liu, Mingjie Sun, Tinghui Zhou, Gao Huang, and Trevor Darrell. Rethinking the value of network pruming. In 7th International Conference on Learning Representations, ICLR 2019, New Orleans, LA, USA, May 6-9, 2019. OpenReview.net, 2019. URL [RLliattpaa///peeeeviis
1 d= J J1335Cmm
Ziwei Liu, Ping Luo, Xiaogang Wang, and Xiaoou Tang. Deep learning face attributes the wild. In 2015 IEEE International Conference on Computer Vision, ICCV 2015, Santiago, Chile, December 7-13, 2015, pp. 3730-3738. IEEE Computer Society, 2015. doi: 10.1109/1CC..2015.2255. URL
https: / /doi..doooo//ooo org/ 10. 1109 /w/./////....... wwt.Cccc..........
Mazda Moayeri, Sahil Singla, and Soheil Feizi Hard imagenet: Segmentations for objects with strong spurious cues. In NeurlPS, 2022.
Mazda Moayeri, Wenxiao Wang, Sahil Singla, and Soheil Feizi. Spuriosity rankings: Sorting data to measure and mitigate biases. In seventh Conference on Neural Information Processing Systems, 2023. URL httpp:://pppptettee www.ppeerrview..eet..eee formf d= JSunnO9QJV
Depen Morwani, jatin batra, Prateek Jain, and Praneeth Netrapalli. Simplicity bias in 1-hidden layer neural networks. In Thirty-sevnnth Conference Neural Information Processing Systems, 2023.
URL htt.hhttss//pppeeviieeee wew.eeewwwwwweeeeeeeeeeeeeeeeeee wtt.r//wwwweeeeeeeeeee.... id=PTvxckOQQE
Vaishnavh Nagarajan, Anders Andreassen, and Behnam Neyshabur. Understanding the failure modes of distr--iittiibution generalization. In 9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021. OpenReview.net, 2021.
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Köpf, Edward Z. Yang. Zachary Vetooo, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. Pytorch: An imperative style, high- performance deep learning library. In Hanna M. Wallach, Hugo Larochelle, Alina Beygelzimer, Florence d' Alché-Buss Emily B. Fox, and Roman Garnett (eds.), Advances in Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Systems 2019, NeurlPS 2019, December 8- 8-14, 2019, Vancouver, BC, Canada, pp. 8024-8035, 2019.
Mansheej Paul, Surya Ganguli, and Gintare Karolina Dziugaite. Deep learning on a data diet: Finding important examples early in training. In Mare' Aurelio Ranzato. Alina Beygelzimer. Yann N. Dauphin, Percy Liang, and Jennifer Wortman Vaughan (eds.), Advances in Neural Information Processing Systems 34: Annual Conference on Neural Information Processing Systems 1021, NeurlPS 2021, December 6-14, 2021, virtual, pp. 20596-20607, 2021.
Mohammad Pezeshki, Diane Bouchacourt, Mark Ibrahim, Nicolas Ballas, Pascal Vincent, and David Lopez-Paz. Discovering environments with XRM. In Forty-firsi International Conference on Machine Learning, 2024. URLptttpa ht..httpss//pppeeevviiee
oppnrevieee net/ / forum? wwwrumiiddpPPtPPFSSS
Shiori Sagawa, Pang Wei Koh, Tatsunori B. Hashimoto, and Percy Liang. Distributionally robust neural networks. In 8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020. OpenReview.net, 2020a.
Shiori Sagawa, Aditi Raghunathan, Pang Wei Koh, and Percy Liang. An investigation of why overparameterizatioo exacerbates spurious correlations. In Proceedings of the 37th International Conference on Machine Learning, ICMI. 2020, 13-18 July 2020, Virtual Event, volume 119 of Proceedings of Machine Learning Research, pp. -356. PMLR, 2020b.
Harshay Shah, Kaustav Tamuly, Aditi Raghunathan, Prateek Jain, and Praneeth Netrapalli. The pitfalls of simplicity bias in neural networks. In Hugo Larochelle, Mare' Aurelio Ranzato, Raia Hadsell, Maria-Florina Balcan, and Hsuan-Tien Lin (eds.), Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020, December 6-12, 2020, virtual, 2020.
Nimit Sharad Sohoni, Jared Dunnmon, Geoffrey Angus, Albert Gu, and Christopher Ré. No subclass left behind: Fine- -rained robustness in coarse grannee classification problems. In Hugo Larochelle, Mare' Aurelio Ranzato, Raia Hadsell, Maria-Florina. Balcan, and Hsuan-Tiee Lin (eds.), Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurlPS 2020, December 6-12, 2020, virtual, 2020.
12

---

Published as a conference paper at ICLR 2025
Ben Sorscher, Robert Geirhos, Shashank Shekhar, Surya Ganguli, and Ari Morcos. Beyond neural scaling laws: beating power law scaling via data pruning. In Sanmi Koyejo, S. Mohamed, A. Agarwal, Danielle Belgrave. K. Cho, and A. Oh (eds.), Advances in Neural Information Processing Systems 35: Annual Conference on Neural Information Processing Systems 2022, NeurlPS 2022, New Orleans, LA, USA, November 28 December 9, 2022, 2022.
Rishabh Tiwari and Pradeep Shenoy. Overcoming simplicity bias deep networks using a feature sieve. In Andreas Krause, Emma Brunskill, Kyunghyun Cho, Barbara Engelhardt, Sivan Sabato, and Jonathan Scarlett (eds.), International Conference on Machine Learning, ICMI. 2023, 23-29 July 2023, Honolulu, Hawaii, USA, volume 202 of Proceedings of Machine Learning Research, pp. 34330, PMLR, 2023.
Mariya Toneva, Alessandro Sordoni, Remi Tachet des Combes, Adam Trischler, Yoshua Bengio, and Geoffrey J. Gordon. An empirical study of example forgetting during deep neural network learning. In 7th International Conference on Learning Representations, ICLR 2019, New Orleans, LA, USA, May 6-9, 2019. OpenReview.net, 2019.
Dimitris Tsipras. Shibani Santurkar, Logan Engstrom, Alexander Turner, and Aleksander Madry. Robustness may be at odds with accuracy. In 7th International Conference on Learning Represen- tations, ICLR 2019, New Orleans, LA, USA, May 6-9, 2019. OpenReview.net, 2019.
Vladimir Vapnik. Statistical learning theory. Wiley, 1998. ISBN 978-0-471-03303-4.
Adina Williams, Nikita Nangia, and Samuel Bowman. A broad-coverage challenge corpus for sentence understanding through inference. In Marilyn Walker, Heng Ji, and Amanda Stent (eds.), Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume I (Long Papers), pp. 1112-1122, New Orleans, Louisiana, June 2018. Association for Computational Linguistics. doi: 10.18653/v 1/N1855511/118-10.11 URL
Yu Yang, Eric Gan, Gintare Karolina Dziugaite, and Baharan Mirzasoleiman. Identifying spurious biases early in training through the lens of simplicity bias. In Sanjoy Dasgupta, Stephan Mandt, and Yingzhen Li (eds.), International Conference on Artificial Intelligence and Statistics, 2-4 May 2024, Palau de Congressos, Valencia, Spain, volume 238 of Proceedings of Machine Learning Re- search, pp. 2953-2961. PMLR, 2024. URL yang24c. html
ngs m press/ saresssvvi pwwess/v38.. v238/
Haotian Ye, James Zou, and Linjun Zhang. Freeze then train: Towards provable representation learning under spurious correlations and feature noise. CoRR, abs/2210.11075, 2022. doi: 10.485500axXiv.22111077..
Michael Zhang, Nimit Sharad Sohoni, Hongyang R. Zhang, Chelsea Finn, and Christopher Ré. Conrect-n-contrast: contrastive approach for improving robustness spurious correlations. In Kamalika Chaudhuri, Stefanic Jegelka, Le Song, Csaba Szepesvári, Gang Niu, and Sivan Sabato (eds.), International Conference Machine Learning, ICML 2022, 17-23 July 2022, Baltimore, Maryland, USA, volume 162 of Proceedings of Machine Learning Research, pp. 26484-26511 26516. PMLR, 2022.
13

---

Published as a conference paper at ICLR 2025
A APPENDIX
A.1 TRAINING DETAILS
We use the ResNet20 implementation from Liu et (2019) that we train for 160 epochs. The network is optimized using SGD with an initial learning rate le-1 and weight decay le-4. The learning rate drops to le-2 and le-3 epochs and 120 respectively. We maintain batch size of 64. Sample difficulty is computed after the 10th epoch.
CIFAR-1OS.
CelebA. We use an ImageNet pre-trained ResNet-50 from PyTorch (Paszke et al., 2019) that we train for 25 epochs. The network is optimized using SGD with a static learning rate le-3 and weight decay Ic-4. We maintain a batch size 64. Sample difficulty is computed after the 10th epoch.
Hard Image-Net. We use ImageNet pre-trained ResNet-50 from Py Torch (Paszke et al. 2019) that we train for 50 epochs. The network is optimized using SGD with static learning rate le-3 and weight decay le-4. We maintain a batch size of 128. Sample difficulty is computed after the 1st epoch.
Waterbirds. We ImageNet pre-trained ResNet-50 from Py Torch (Paszke al. 2019) that train for 100 epochs. The network is optimized using SGD with static learning rate Ic-3 and weight decay 1e-3. We maintain batch size of 128. Sample difficulty is computed after the 1t epoch.
MultiNLI. We use pre-trained BERT model that we train for 20 epochs. The network is optimized using AdamW W using linearly decaying starting learning rate 2e-5. We maintain batch size of 32. Sample difficulty computed after the 5th epoch.
A.2 ADDITIONAL EXPERIMENTAL DETAILS
CIFAR-10S. We follow a similar approach to Nagarajan al. (202I) for adding a spurious line where pixel values for vertical row of pixels in the middle of the first channel are set to the maximum possible value (255) before normalization and before any augmentations.
use the same augmentations generally used for training the original CIFAR- 10 Krizhevsky (2009).
CelebA. In this setting, we maintain 5000 Female Samples without Eyeglasses and 2500 Male sam- ples with Eyeglasses and 2500 Male samples without Eyeglasses. Consistent with implementations in Sagawa et al. 2020a); Liu et al. (2021). we do not use any augmentations.
Hard ImtgeNet. In this setting, maintain 59 Dog Sled samples with minimal spurious features and 100 Ski samples randomly drawn from the dataset. All remaining classes are maintained the same. We use the same augmentations used for training ImageNet.
Waterbirds. We use the original Waterbirds setting commonly used practice (Sagawa et al. 2020a Liu al. [202]] Zhang et al. [022]] Kirichenko al. (023). We the augmentations used in Kinchenkko et al. (2023) when training, which are similar to the augmentations used for training on ImageNet.
MultiNLI.
We use the original MultiNLI setting commonly used in practice (Sagawa et al. 2020a
Liu et al.
202T Kirichenko et al. 2023). Consistent with implementations in Saaaaa et al. (2020a):
Liu et al.
2021 Kirichenko et al. (2023). we do not use any augmentations.
A.3 HARD IMAGENET: VERIFYING UNIDENTIFABILITY
We show that our Hard ImageNet setting an unidentifiable setting by presenting information similar to Fig. 5a and Fig. 5b But since there exist multiple spurious features, namely Snow, People, Trees, and Hi Hills, we simply show that samples that contain these features and those that do not scattered uniformly across the difficulty spectrum (Fig. 9)
14

---

Published as a conference paper at ICLR 2025
Increasing Difficulty
Faslest
Hardest
Figure 9: Training distribution variance based on strength of spurious feature identifiable and unidentifiable settings.
A.4 ADDITIONAL EXPERIMENTS WITH MULTINLI
In this section, reinforce our claims and observations by re-conducting specific vision experiments in the language domain.
Spurious Correlations formed from few key samples. To show this, perform the same experiment Section 4, but instead CIFAR- we use the MultiNLI dataset. First, we remove all samples with negation words from the training data and then compute the sample-wise difficulty scores as we do for CIFAR-1OS in Section We then create two settings: one where we introduce the spurious negation word "never" at the end of the 100 hardest input samples belonging to class (cont r aiiiie ot ct and another where we introduce the spurious negation word "never" at the end of the 100 easiest input samples belonging to class I (cont roditradicto sts).. We do the same to set of test samples belonging to class (neut ral with) and class (entailed by).
Consistent with the standard MultiNLI setting, we measure the degree of spurious feature reliance through Worst Group Accuracy (accuracy of the set of test samples of class 2 or class 3 with the spurious feature).
We observe that WGA is significantly worse when the word "never" occurs in the hardest samples vs. the easiest samples during training.
Introducing the spurious feature in easiest 100 samples: WGA = 55.22%
Introducing the spurious feature hardest 100 samples: WGA 1.04%
Additionally, we note that there are 191,504 training samples in this setting. There are 57,498 samples belonging to the cont cont radi cts class (0. 17% of samples within the class, 0.0522% of all samples in the training set.) also observe that setting with no spurious features during training, Worst Group Accuracy 67.4 67.42%.
cts class. We introduce the spurious feature in only 100 samples of the
This experiment reinforces the claim that samples with hard core features are primary contributors to spurious correlations and that samples with simple core features do not contribute to spurious correlations.
Excluding few key samples during training severs spurious correlations. In the MultiNLI setting, we observe that pruning the samples with hard core features and spurious features attains high worst group accuracy. On the other hand, excluding samples with easy core features and spurious features does not improve worst group accuracy. Note that we do not perform any hyperparameter tuning.
15

---

Published as a conference paper at ICLR 2025
(1) Pruning Easiest
(2) Pruning Hardest
(3) Unpruned Dataset
Higher Better
Higher is Better
100
Worst Worst Group Accuracy (86)
75
Accuracy (96
80
60
70-
65
20
25
33.33(%)
20
25 33.33/%)
(16)
Figure 10: Excluding a fraction all samples with hard core features and spurious features mitigates spurious correlations in the MultiNLI setting. This is indicated by high Worst Group Accuracies.
(1) Spurious Samples
(2) Non-Spurious Samples
MutiiNLI
Sample Percentage (8)
40
30-
20-
10
Q1
Q3
(Easlest)
(Hardest)
Figure 11: Training distribution variance based strength of spurious feature in identifiable and unidentifiable settings. Grouped by Quartiles, sorted by difficulty.
Distribution of the MultiNLI dataset. We show the distribution of the MultiNLI dataset, an extension of Fig. [b] Q1 in this setting contains almost half all samples with spurious features while Q4 only contains 15%. This shows that samples with spurious features are not uniformly distributed when viewed through the lens of sample difficulty. This is in contrast to the CelebA setting (Unidentifiable), in which samples with spurious features are uniformly distributed (Fig. 5b (right)).
16


Reviews:
[{"review_id": "uXTn5BqYXS", "reviewer": "Reviewer_X6Fn", "summary": "This paper aims to mitigate the problem of spurious correlations in deep learning models. Through a sequence of simulation experiments, they authors discover that a small subset of training data containing “hard” core features is primarily responsible for the model learning spurious correlations, even when the spurious signal is weak. The authors then demonstrate through subsequent experiments that pruning this subset of samples effectively severs the link between spurious and core features.", "questions": "My question(s) would simply be can the authors provide theory, models, statistical justification etc. (or fairly robust/general empirics) to address the listed weaknesses?", "limitations": "", "rating": 10, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 4, "strengths": "This work represents an example of my favorite kind of work. A very important and easy-to-understand problem (spurious correlations learned by models that affect their generalizability) and a very intuitive solution, once the authors explain and demonstrate it. I also felt the work was well written, guided the reader through the gaps in the literature, and step by step demonstrated the veracity of their arguments with simple and compelling experiments.", "weaknesses": "Inductive Solution\nThere I assume many types of spurious correlations can be learned by a model. For example, it is will documted that outliers can overinfluence functional estimation, and create spurious corrleations. The proposed solution addresses only a specific type of spurious correlation (i.e., roughly speaking, where a certain class has an over-representation of a given feature that its not truly indicative of the class label), which arises under the particular conditions the authors generate (i.e., \"the spurious feature takes the form of a line running through the center of the images\", \"images of men with glasses\", etc. ), given the distributional properties of the datasets considered, and given deep learning architectures (e.g., mostly ResNet) considered. Moreover, their solution relies heavily on specific empirical observations about how this particular type of spurious correlation manifests and is distributed, given the particular generation process followed. For instance, the authors observe that in their simulations \"in settings where the strength of the spurious signal is not significantly greater than the strength of the invariant signal...samples containing spurious features are uniformly distributed across the training distribution...[and] the presence of spurious features does not have a significant impact on the training distribution...[therefore] samples containing hard core features that also contain the spurious feature are primary contributors to the formation of spurious correlations\". They therefore, conclude that \"to mitigate spurious correlations, one would only have to prune the hardest samples in the training set, as this subset of the data would contain samples with spurious features that have hard core features.” This reasoning appears to be purely inductive and raises questions about the generalizability of the observations and subsequent solution. Specifically, I'd assume that the observations about the distribution of spurious signals and sample difficulty will hold across other types of spurious correlations, which may behave differently under varying dataset characteristics or model architectures. \n\nTheoretical Justification/Generality\nWithout a theoretical foundation supporting the general applicability of the inducitve observations, it remains unclear whether the observations that lead to solution are universal and/or if the prunning method can serve as a general approach to mitigating spurious correlations in models. Therefore, the paper could be significantly improve with a theoretical justifications for the generality of the inductive observation. For example, 1) is the particular type of spurious correlation the authors consider representative of all spurious correlations, 2) does the distributional uniformality of samples containing features with spurious correlation across the training samples generalize across types of spurious correlations, 3) is the pruning of this particular type of subset as a solution to (any type of) spurious correlations, and 4) are these justifications architecture/data dependent. \n\nRelated, but unique, I think structuring the pruning solution as a formal model, statistical hypothesis test, etc. would strengthen the theoretical foundation of the proposed pruning technique, and again perhaps shed light on it's generality. A theoretical or statistical clarity on why the samples containing spurious correlation are distributed uniformaly (e.g., I wonder if this is a consequence of the inverse probability transform). A claim that spurious feature reliance experienceing polynomial growth with increasing sample difficulty, would be more trustworthy if supported by some theoretical formulation or model. \n\nPruning Consequences\nWhile the authors demonstrate that prunning a particular type of subset of data points will reduce spurious correlations, there is no discussion of what are the consequences. I am inclined to beleive that throwing out data is going to have some sort of negative consequence, and therefore it important to know what trade-off is being made. This will allow a better comparison to other methods that do not simply prune data as well as enable practitioners to understand/determine if the cost of pruning data is worth the benefit in removing spurious correlations.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Unq30wBKv1", "reviewer": "Reviewer_eCbW", "summary": "The paper explores how deep neural networks often rely on spurious correlations present in training data, which can lead to performance drop under distributional shifts. The authors highlight a novel setting where spurious signals are weaker, making their identification challenging. They propose a new data pruning technique that selectively removes some training samples that contribute significantly to the formation of spurious correlations. This technique operates without requiring detailed information on the nature or presence of spurious features. The authors demonstrate that their approach achieves state-of-the-art performance on both standard and challenging benchmarks, including scenarios where spurious features are identifiable and unidentifiable.", "questions": "1) How sensitive is the pruning method to changes in hyperparameters?\n2) The robustness of sample difficulty estimation across different model architectures is not clear at the current version of the paper. Would it be possible to add some results or explanations for that?\n3) Table 1 shows that some SOTa methods achieve better results. Could you please give some explanation on that? Also, \n4) could you please open source the code to enhance the reproducibility?\nSuggestions for Improvement:\n1) It would be helpful to include a section that discusses the potential negative impacts or limitations of the method, such as the risk of pruning samples that are informative but rare.\n2)Extending the empirical analysis to more complex and real-world datasets with non-synthetic spurious correlations could further validate the applicability of the method.", "limitations": "", "rating": 5, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "1) The paper addresses a scenario where the strength of the spurious signal is relatively weaker and thus it is difficult to detect any spurious informationhat spurious signals compared to other works  where the strength of the spurious signal is significantly greater\nthan that of the core. \n2) The paper provides a thorough experimental design that spans both identifiable and unidentifiable spurious feature scenarios. The use of multiple datasets (e.g., CIFAR-10S, CelebA, Hard ImageNet, Waterbirds, MultiNLI) showcases the method's robustness.\n3) The paper is well-organized, detailing the rationale behind the proposed method, experimental setup, and results.", "weaknesses": "1) The paper could explore more thoroughly the practicality of applying the proposed pruning method to large datasets. Specifically,even though the proposed method shows promise for datasets of moderate size, an assessment of its computational cost and efficiency on large and real world data would be beneficial.\n2) The approach relies on assessing sample difficulty as a proxy for contribution to spurious correlations. Clarifying the robustness of this metric under different training regimes (e.g., varied architectures or optimization strategies) could strengthen the generalizability of the findings.\n3) Although the paper discusses state-of-the-art methods, further comparative analysis with recently emerging pruning and robust training techniques that do not rely on explicit spurious feature identification would be helpful.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "oNgvQRtGog", "reviewer": "Reviewer_7BQb", "summary": "This paper study the spurious feature associated with the trained model, focusing on how this correlation is formed during the training process and proposing a data pruning approach to mitigate spurious correlation. Specifically, the authors find that spurious correlation is caused by a few samples that are hard to learn and contains spurious features. As a result, this paper proposes to remove those data points from the training dataset to improve model performance.", "questions": "Please find in the weakness section above.", "limitations": "", "rating": 5, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "1. The paper organization is clear and easy to follow.\n2. The problem of identifying and mitigating spurious correlation is critial.\n3. This paper considers a challenging scenario where spurious features have weak signals and therefore are difficult to be detected.", "weaknesses": "**Major Concerns:**\n\n1. Lack of rigorous formulation and solution. Throughout this paper, there is no single equation or definition that clearly states the problem and the proposed solution. Key concepts that heavily mentioned in the paper, such as spurious correlation, core/invariant features, simple/hard features, are not well-defined. For a more readable paper, the authors are encouraged to (1) state a self-contained problem with properly defined concepts (2) write a pesudocode for the proposed data pruning algorithm (3) disclose complete experimental details including but not limited to training/test dataset processing procedure, models, calculation of evaluation metric (which is not well-defined as well), and baseline methods.\n\n2. Lack of clarity in findings. Most figures and tables are not self-explained and neither explained by their titles. As a result, it is confusing how do they support the claims in the main text. This problem is aggravated due to the previous point. For example, I did not understand the message in Figure 2, as concepts like \"easy/hard\" are undefined, and authors do not explain how they perform the experiments, e.g., how to add spurious features into the training data and how to calculate the misclassification rates. The authors should review all their findings and claims and rewrite their evidence to be more supportive and convincing.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "eEIGIJZNJY", "reviewer": "Reviewer_n9k8", "summary": "This paper empirically investigates the phenomenon of spurious correlations being learned by deep learning models.\n\nThe authors are mainly interested in the setting where the spurious signals are \"weak\", in the sense that are not easy to be detected and removed.\n\nIn this setting, the authors propose a data pruning approach to limit the effects of the spurious correlations.\n\nThe method consists in looking at the (few) training points that are more difficult to classify during training. These points are most likely making the model overfit patterns that are not necessarily causal to the right label, and are therefore a major cause to the learning of spurious patterns. This is confirmed experimentally.", "questions": "There should be a typo in line 162 \"as shown in...\"\n\nI am still confused about the setting used to obtain Figure 3. And what do the authors mean by \"polynomial growth\" in this case? What does it mean to \"scale up the difficulty of the core features of the 100 saples\"?\n\nI am also confused by Section 6.3 - What do the authors mean by \"group labels\" in line 457? Can the authors provide a brief and self contained description of the point that wants to be raised in this section (while I believe this is not the central part of the work)?", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "The paper is very well written, it is clear, the approach followed by the authors is logically sound, and I personally enjoyed reading it. While I am not certain that it is true that all previous work adresses cases where the spurious signal is stronger than the core features, the contribution seems indeed novel, and the results are reasonable and well explained.\n\nThe paper is on point. It proposes mainly one single method, and the writing is structured in a way that the reader can digest the phenomenology before being given the final algorithm / approach.", "weaknesses": "It is sometimes not obvious what the authors mean by \"strong\" or \"weak\". While providing precise definitions is beyond the purposes of this work, some examples during the introduction could facilitate the reading. For example, I was confused in the paragraph at line 201, where the strength of the signal is defined both in terms of the geometry of the pattern and it's frequency in the data. These two aspects are fundamentally different, and putting them altogether might not result in the best model to investigate this problem...\n\nThe reason behing I do not give a higher schore is that some of the results are not entirely surprising. The phenomenlogy described by Figure 2 is interesting, but at the same time - to my understanding - in line with what we expect from the influence of individual samples to the final parameters of the model (see, e.g., [1, 2]). Nevertheless, this might be a new perspective in the community of spurious features / correlations, and I therefore recommend acceptance.\n\n[1] https://arxiv.org/pdf/1906.05271\n[2] https://proceedings.neurips.cc/paper_files/paper/2020/file/1e14bfe2714193e7af5abc64ecbd6b46-Paper.pdf", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Spotlight)