EvA: Erasing Spurious Correlations with Activations
Authors: Qiyuan He
Kai Xu
Angela Yao
Venue: ICLR 2025 Poster (2025)
Keywords: spurious correlation
compute efficiency
data efficiency
Primary area: alignment, fairness, safety, privacy, and societal considerations
TL;DR: EvA is a method that effectively handles spurious correlations in pretrained networks by explicitly erasing class-specific spurious connections, improving both data and compute efficiency.

Abstract: Spurious correlations often arise when models associate features strongly correlated with, but not causally related to, the label e.g. an image classifier associates bodies of water with ducks. To mitigate spurious correlations, existing methods focus on learning unbiased representation or incorporating additional information about the correlations during training. This work removes spurious correlations by ``**E**rasing **wi**th **A**ctivations'' (EvA).  EvA learns class-specific spurious indicator on each channel for the fully connected layer of pretrained networks. By erasing spurious connections during re-weighting, EvA achieves state-of-the-art performance across diverse datasets (6.2\% relative gain on BAR and achieves 4.1\% on Waterbirds). For biased datasets without any information about the spurious correlations, EvA can outperform previous methods (4.8\% relative gain on Waterbirds) with 6 orders of magnitude less compute, highlighting its data and computational efficiency.

Published as a conference paper at ICLR 2025
ERASING SPURIOUS CORRELATIONS WITH ACTIVA- TIONS
Qiyuan He, Kai Xu, Angela Yao
National University of Singapore qhe@u. nus sg, kxudu.nus. ayacêcomp. nus. edu. .g
ABSTRACT
Spurious correlations often arise when models associate features strongly correlated with, but not causally related to, the label e.g. an image classifier associates bodies water with ducks. To mitigate spurious correlations, existing methods focus on learning unbiased representation incorporating additional information about the correlations during training. This work removes spurious correlations by "Erasing with Activations" (EvA). EvA learns class-specific spurious indicator on each channel for the fully connected layer of pretrained networks. By crasing spurious connections during re-weighting, EvA achieves state-of-the-an performance across diverse datasets (6.2% relative gain on BAR and achieves 4.1% on Waterbirds). For biased datasets without any information about the spurious correlations, EvA can outperform previous methods (4.8% relative gain on Waterbirds) with 6 orders magnitude less compute, highlighting its data and computational efficiency.
INTRODUCTION
Deep neural networks are susceptible shortcuts and correlations with spurious features features predictive for training but lacking genuine causation. For example, consider an image classifier for cats and dogs; if training images commonly depict dogs (but not cats) on grass, the model may misclassify cat on grass as dog because grass is learned as spurious feature. "Right for the wrong reason" (Geirhos et al., 2020; Ross et al. | 2017), spurious correlations may lead to unintended biases (Zech et al. [018]] Sagawa al. 2019) and catastrophic failures in real-world
environments (Adamson & Smith 2018 Lesort 2022).
Studies spurious correlations and shortcuts (Zhu et al. 2021 Pezeshki al. 2021 Bai et al. 2021bb have shown that deep networks prioritize simple and low wwwfrequency features like back Dackgrounds and textures (Xu et al.) 2019). A straightforward approach adopted by (Sagawa et al. (2019): Idrissi et al. (2022) tries to identify the spurious correlation with "unbiased" data, e.g., dogs and cats with grass background. These methods may be data-wise expensive depending
the amount of unbiased dataset or human annotation on spurious correlation, i.e., spurious feature label. Other works (Arjovsky et al. 2019; Nam al. 22020) Kim al. 2022 Li ci al., 20222 Tiwari & Shenoy 2023) aim to learn invariant representations. These methods may be computationally expensive, as they perform multiple rounds of retraining.
Recent works YE et al., 2023 Kirichenko et al. 2022 Krueger et al. 2021 contend that neural networks, despite having spurious features, also learn "correct" core features; this the case even at the final (linear) layer. Previous methods such as DFR (Kirichenko et al.) 2022) use unbiased dataset to re-learns the last linear layer, freezing the feature extractor. Unfortunately. DFR's s effectiveness decreases when the re-went weightiing dataset small, and even worse when there is additional unbiased dataset or when the additional dataset biased (Ye et al. 2023), underscoring the need for both computational- and data-efficient alternatives.
it possible to identify spuriousness with minimal unbiased data while also preserving computational efficiency? In this work, propose method that is robust to small unbiased dataset, as well as variant that functions effectively without the need for any additional unbiased dataset and spurious feature label during training with only few minutes training single GPU.

---

Published as a conference paper at ICLR 2025
Doy
Spelier
Ixtra Data
Festur Ooment
Ca D.P1
Ciband
Dog Sorure
syurouur Caov Duddcis
Figure 1: Spurious feature detection in EvA. When "unbiased" data is accessible, we measure the spuriousness based on the activation distribution differences between spurious and unbiased data. When only biased data available, we estimate spuriousness by evaluating the contribution of each feature the energy score over the biased dataset.
We focus only on spuriousness at feature (channel) level, specifically before the last linear layer. If unbiased data is available, we compute spurious measure for each feature based on the consistency of penultimate activations between the spurious and unbiased datasets. In the absence of unbiased data, the activations of a model to spurious training set can serve as a reliable indicator of spuriousness. We assess the contribution of each feature the network's prediction confidence. Features with high confidence are likely to be spurious, as the model has over-fitted on the spurious training set; we refer this metric evidence energy. Subsequent this evaluation, the fully connected layer undergoes re- re-weighting based on the computed spurious indicators.
Utilizing consistency and evidence energy as to quantify the extent of spuriousness, the models undergo fine-tuning based their respective spurious indicators. "Erase Spurious Correlation with Activations" (EvA) encompasses EvA-C consistets,,,,. tailored for scenarios with extra training data, and EvA-E (EvA-evidencee energy), tailored for scenarios without extra data. Through channel- based erasure and re- re-weighting on the final linear layer, EvA-C necessitates only 40% of the extra unbiased data previously required and EvA-E can work even if only biased data is provided. For computational efficiency, EvA-E significantly reduces the training duration to a mere 10 minutes, in contrast the 6 days necessitated by previous methodologies. EvA also yield improved performance outcomes across the BAR, CelebA, and Waterbirds datasets (EvA-C achieves 6.2% relative gain on BAR, Ev A-E achieves 4.1% 1% Waterbirds, etc). Our contributions include:
We propose pipeline where the spurious indicators are first computed for each feature via consistency and evidence energy, the spurious correlations are then erased by re-weighting the fully connected layer based on the computed spurious indicators.
We introduce two methods for detecting spurious feature: Consistency and Evidence Energy.
The latter obviating the necessity for extra training data.
Under both settings, training with or without extra data, EvA achieves state-of-the-art performance across various datasets with only minutes of training on a single GPU, compared to six days of training per GPU, Both versions significantly outperform previous methods without unbiased dataset or any annotation about existing spurious correlations.
2 RELATED WORKS
Spurious correlation has been studied from different perspectives, including worst-case generalization and deep feature re-weighting with extra data, unbiased representation learning without extra data. We provides more detailed discussion in Appendix C
Worst case generalization in extra-dalt settings. To address a specific spurious feature, set of works (Sagawa al. 2019 Bai et al. 202 Teney et al.) 2021 explicitly divide the dataset into several feature-labe groups based on the presence or absence of the given spurious feature. And the target to overcome spurious correlation is formalized as minimizing the classification error the worst group (Sagawa et al. 2019). These methods show promising performance but are limited in practicability because of the huge cost of identifying and labeling spurious features in the real world.
2

---

Published as a conference paper at ICLR 2025
Unbiased representation learning. To address the need for spurious feature labels, previous works try to learn unbiased representations from a biased dataset. Most of these approaches (Nam et al. 2020; Kim et al. 2022 Tiwari & Shenoy 2023 Lee et al. 2023 Liu et al., (021)) adopt multi-stage training and introduce additional neural networks and hyperparameters, all which require extensive computational resources for selection via validation data with less spurious correlations. Our method
is more efficient by moving the debiasing module into post-hoc erasure on the linear layer.
Deep feature reweighting. Similar approaches using worst case generalization, methods rooted on DFR (Kirichenko et al. 2022 Ye et al., 2023) are applied in extra-data setting where "unbiased" validation data available during training. DFR (Kirichenko et al. 2022) is efficient by retraining the last linear layer with balanced data to robustify the classifier on overcoming spurious correlation. This supports an important assumption: deep features commonly include both spurious features and core features, motivating our works to erase spurious feature directly. However, different from our work applied in both settings, the performance of DFR decreases significantly when the extra dataset is imbalanced among feature-label groups the size is limited.
METHOD
3.1 PRELIMINARIES
Following (Kirichenko al. 2022 Ye et al. 2023). and spurious components:
model data-label pairs (z, y) based
(Foore: Espu) R y/ B.oore + Ceores
where core component Tcore is drawn from some distribution IP and the spurious component Zsu is correlated to the associated label y for samples from training set Durin but not for testing set D
72 y + Expu E Rdd, ~ D
Beore P E Red,,
Tspu
Espu E Rds, Diest
In Equation [ and 2 the terms Ecore and Espa denote noise associated with the core and spurious components respectively, while the parameters 3 and are normalized coefficients. Furthermore, the spurious component, being dependent on label y, is speciicc......
Consider k-class neural network classifier f ; decomposed into a feature extractor g and classifier h, ie. f = hog. The feature extractor x + Z yields m-dimensional feature Z R.t while the classifier h is typically linear function:
J. where Rd, and R$ f can be
= h()) WZ + b, where W E RR b R.*
We define á as the distribution of 7 over some dataset, with g60+) representing the distribution of the i-th channel of Z. To further characterize the distribution for each class, we use ((ak) to denote the distribution (3)) for class k:, where the input data is restricted class k. f is learned on a training contains spurious correlation for class k. then (sk)) will be spuriously correlated with class k.
3.2 QUANTIFYING SPURIOUS CORRELATIONS.
We aim to quantify the extent of the spurious correlation for feature ((k)) To that end, introduce the Consistency measure and the Evidence Energy measure. The former is computed when unbiased data D Punbiased is available, while the latter can be estimated without unbiased data Dunbiased-
3.2.1 WITH UNBIASED DATA: CONSISTENCY
We define the Consistency between (ii)) and class & as the distance between training and testing distributions of (itk),
(1ik)
(d(c)
(ak) (ike)
Press
dl ((
(tke
(Prain Pumbiisedd
(ak)
3

---

Published as a conference paper at ICLR 2025
Above, d is symmetric function evaluating the distance between two distributions positive real number; we use the Wasserstein distance (Vallender 1974). which is commonly used as distribution metric (Arjovsky et al. 2017: Wang et al. (0211)
To approximate the unknown (les)) we instead compute the distance based on d) (sk)
unbinsed estimated
from additional unbiased data D Dunbiased- In comparison, other approaches use D unbiassed to re-learn
weights (DFR (Kirichenko et al. 2022)) or select hyperparameters (Arjovsky et al. 2019
Nam et al.
2020
Kim et al. 2022 Li et al.
2022: Tiwari & Shenoy 2023).
3.2.2
WITHOUT UNBIASED DATA: EVIDENCE ENERGY
We also define a measure without D Punbiased based on the observation that there is a high probability of spurious correlations for network predictions with high confidence or prediction logits (Tiwari Shenoy 2023 Xu et al.) (019)). We elaborate on this observation from a theoretical perspective in Section 3.4Jand and empirically Section (..2 light of this observation, our aim is to determine the feature contribution to the final logit, based on the energy.
Energy: The energy (LeCun al.
2006 Liu et 2020) for an input x formally defined as:
(xx) T log
where T represents temperature parameter, and fx) denotes the logit associated with class k. Throughout training, the energy of training samples implicitly minimized. As such, the energy
serves as an indicator of the network's prediction confidence (LeCun et al. 2006 Liu et al. 2020); the lower the energy, the more confident the prediction is.
Evidence Energy: We introduce the concept of evidence energy to quantify the correlation between activation ((ikk) and the prediction confidence. To delineate, for a given data point æ from the training set, the evidence energy of is approximated as the linear term of a Taylor expansion:
(ik) (x)
8E(x) BeptER
(((())
Mr WRi
(((k)), where Mr exp( Σ Wii (((K)) + b.).
(6)
E Me
We further define the expectation of the evidence energy training dataset Duain
((i))
EEEDE Inis
(e(it)
e ((ik)
D train
RED
where D denotes training data for class k. The evidence energy is directly correlated with consistency when the network f' is trained on biased data. This observation is grounded in the notion that neural networks prefers learn simple features -the simplicity bias, as established by existing works (Xu et al., 2019; Zhu et al. 2021 Sagawa et al., 2019). Furthermore, spurious features are often simple (Pezeshki et al. 2021: Ye al. 2023 Tiwari & Shenoy] 2023). Subsequently, during inference, spurious correlations leads to elevated contidences and therefore evidence energy scores.
3.3 EvA PIPELINE.
EvA is a post-hoe method that detects spurious feature, crases it by setting the corresponding weights to 0, and then re-weights the remaining core feature.
Detection. Given spurious model f. we define a spurious feature indicator (1k) for feature i on class to identify a set of spurious feature:
(0t
[(ah) 1}
(8)
If Dumbiased is available, we use consistency as an indicator, otherwise, we use the evidence energy from Equation]] 7
(1k)
1, ((k) < dc
and (((k)) = {
1. E (ik)
< 8g
( 0, (ik) > dc
0, E (ik) dg
99)
4

---

Published as a conference paper at ICLR 2025
Both the threshold for consistency &c and evidence energy óg are determined by the erase ratio c, hyperparameter determined in the re-weighting stage.
Reweighting, Re-weighting limits the linear layer's 's dependency on spurious features and prioritizes the core features. After distinguishing the two with the above indicators, we climinate potential spurious features within each class k by setting the corresponding weights to zero before relearning the remaining non-zero weights.
More formally, given the set of spurious evidence for each class 4x. determined by Equation g solve for new weights {W", b. which rely only on core evidence & 4b for each class k. Given a reweighting dataset D,, and frozen feature extractor g, the objective of the linear layer (W*, b*) can be defined as a constrained optimization:
min We,
L(Drig,W*,br
s.t. WAj =0, 0, VΦ E $k. E K).
(10)
Dunbiased is used as D, if it available (EvA-C); note that differs from DFR (Kirichenko et al. 2022) which retrains all the weights of the linear layer from both core and spurious features. Without any D Dunbiased- use Daain instead (Eva-E). Such setting is appealing spurious feature label is required during training. We provide the whole algorithm in Appendix B
extra unbiased dataset or
3.4 THEORETICAL RESULTS
Following the framework used by (Arjovsky et al. 2019 Ye al. 2023). we conceptualize the network with two linear layers and employ data generation mechanism the same as Section[..] for formal theoretical analysis, with further details of these conventional assumptions provided in Appendix A this section, we consider some class l and two features (((k) and (5(bk) For simplicity, we omit the superscript k; and two features can be represented as:
((a) ((s) Feore + " Espu: and (h) goore T care Iapu sspus
(11)
(a) where Score is the coefficient associated with core component Teore for feature (((),, gepic is the (b)
coefficient associated with spurious component T'spa Similarly, feore and gepu are the corresponding
coefficients of feature (b)), We analyze the relationship of consistency and evidence energy with the spurious component Tepu-
Consistency: Feature with low consistency be formally interpreted having greater reliance on spurious components.
Theorem 1. (Informal) Under set of assumption, the consistency of (()) exceeds that of の00 (i.e., ((ak) (bb)) if and only [ E[p |[rppE[
(b)
This theorem substantiates the claim that the spuriousness of a feature can be quantified through con- sistency, as higher consistency indicates reduced reliance, i.e., 19 El (pp I. on spurious component
Espu- The proof and more detailed explanations are given in Appendix A.
By comparing the evidence energy of two features, we demonstrate that spurious features tend to have lower evidence energy, and this tendency becomes more pronounced as the dataset becomes more biased. Let neore! and Tipu denote the variance of Icore and Tepu respectively.
Evidence Energy:
Theorem 2. (Informal) Under a set of assumption, if nepu < noore and W > (b) WaodsppSspp it holds that P (Elak) < B(bk)) = 0.5 + R(Aepus Moore). where R(.,)) is a strictly dnsitive-valuees function.
The condition 7spu Meore mathematically describe that the noise associated with the spurious component is less than that of the core component and making the spurious feature easier to learn (Ye) et al. 2023). Theorem 2 shows that if a feature contributing to the final result relies more &spa T.e. W WppepuZepu: it's more likely to have lower evidence energy. The residual R increases when the noise in the spurious feature is significantly smaller than the noise in the core feature, i.e. the spurious feature is casier for the model to learn. The proof and more details for Theorem 2 are given in Appendix A.2
5

---

Published as a conference paper at ICLR 2025
While the theoretical model is simplified view of neural networks and real-world data, we verify empirically in Section 4.2 that both consistency and evidence energy can identify spurious features.
4 EXPERIMENTS & ANALYSIS
4.1 EXPERIMENTAL. SETUP
Datasets and Evaluation. We verify our proposed framework with the datasets listed in Table I The datasets can be characterized by the conflicting ratio (Tiwari & Shenoy] 2023), or the proportion of samples that counter the spurious correlation within the entire dataset. A detailed description is given in Appendix D For each dataset, we report the mean and standard deviation over ten runs on the average top-1 Accuracy unless otherwise indicated.
CMNIST. ColoT-MNIST (Tiwari & Shenoy 2023) is a synthesized dataset that colors the digits 0 and from MNIST (Deng 2012). In the training set, the 0's are red and the I's green; in testing, the colors are mixed across the two digits.
BAR. The Biased Activity Recognition dataset (Nam et al. 2020). features six human actions spuriously correlated with the background in training but not in testing.
Waterbirds. Waterbirds (Sagawa al. 2019) is two-class image dataset of water- and ground-birds; the class is forrelated with the background (water or ground) in training but not in testing. It evaluated according Accuracy and Worst Acc (Sagawa al. 2019
Ye et al. 2021 2023). which is the accuracy of the worst label-context group.
CelebA Hair. CelebA is a large human face benchmark (Liu et al. 2018). Following settings from (Ye al., 2023 Tiwari Shenoy (023), the task is to predict hair color based on the face image. In the training dataset, >99% instances with blond hair are women so
blond hair and gender are spuriously correlated. CelebA is evaluated additionally based on the Unbiased accuracy over each label-context group and Conflicting accuracy, averaged
over the bias-conflicting samples per class following (Li et al. 2022 Kim et al. 2022).
Tiwari & the ResNet18 (He al. (016) the base model initialized with weights pre-traine trained on ImageNet and the same settings of (Nam et al. 2020
Training and Validation. For fair comparison with previous works (Nam et al. 2020:
Shenoy 2023 Li et al. 2022 Teney et al. 2022).
Tiwari & Shenoy 2023 Li al. 2022 Teney et al., 20022.. To compare with the methods focusing on minimizing the error in the worst label-context group, we also provide the results on ResNe150
with previous work in the Appendix
B.2
We use SGD optimization with fixed learning rate of 0.001. For CMNIST. Waterbirds and CelebA we use the same validation and test dataset as (Sagawa al. 2019 Ye al. 2023 Tiwari Shenoy 2023). BAR has no provided validation set, so we randomly split the lesting dataset into two equal halves across the ten experiments to form a validation set that follows the same testing distribution. To select the erase ratio we retrain the linear layer with different erase ratio candidates and select the one with the highest accuracy on Dunbiased-
Reweighting dataset. Under the setting without extra data, the training dataset is used as a reweight- ing dataset. Under the setting with extra data, the validation dataset is split equally, while one serves as a reweighting dataset, the other serves to select byper-parameter In BAR where the spurious feature is not labeled, the reweighting dataset also biased. In CelebA and Waterbirds, follow DER (Kirichenko et al., 2022) to sample "unbiased" dataset from the validation set.
4.2 ANALYSIS
We analyze our methods according to 1) the correlation between evidence energy and consistency, 2) the effectiveness of feature erasure, 3) data efficiency, and 4) computation efficiency.
Evidence energy & Consistency. We empirically access the correlation between evidence energy and consistency. In Figure 2 we present illustrative example of a feature set predicting the action class Throwing in the BAR dataset, with an crase ratio of 0.1. Inconsistency is computed between the training dataset and the validation dataset. Figure
(a) shows moderate positive correlation

---

Published as a conference paper at ICLR 2025
Dataset
Val ratio
Hparams goal
Train
Conflicting ratio
Val Test
Method
EvA
Target Feature
Color
Color Accuracy
98.74
Digit Accuracy
22.
CMNIST
30%
Accuracy
0%
30%
Digit
51.45
97.32_2s
BAR
16%
Accuracy
0%
100%
ERM
Color
97.78
51 51.6621.8
Waterbirds
25%
Worst Ace
5%
25%
Digit
92.11
56.03
CelebA
12%
Unbiased
0.8%
0.9%
Table 2: Accuracy (%;; mean on CMNIST Color Dataset and Digit Dataset. ERM fails learn the more complex digit feature, whereas EvA achieves 97.78% on the digit dataset and only 51.45% on the color dataset, indicating erasure of the color feature.
Table 1: Summary of dataset details. 'Val ratio'
dicates the proportion of the validation dataset size relative to the training dataset size. CConflcctng ratio' denotes the proportion of images that counter the spu- rious correlation within the entire dataset.
3.0
1.000
90
EvA-C
2.5
80
85
DFR
Incorastncy
2.5
40
1.0
75
0.5
8.00
0.02
0.04
0.06
0.08
%
70 10 15
20
25
30
Negative Evidence Energy
inconsistency
Velliietion size Training size (%)
(a)
(b)
(c)
Figure 2: (a) Correlation between negative consistency and negative evidence energy for class Throwing in the BAR dataset. Each point represents feature Φ for sample. Feature on the right of the blue line is erased by evidence energy and most of spurious feature, i.e. above the green line, is crased well. (b) Percentage of erased feature within a range of inconsistency for class Throwing in the BAR dataset, e.g. 78% feature with inconsistency hetween [1.0, 1.5] is erased. Frasing by evidence energy results erasing most feature with high inconsistency while keeping the core feature. (c) Effect of unbiased dataset size on BAR's 's Accuracy. As the unbiased set diminishes in size, the accuracy of EvA-C surpasses that of DFR by greater margin.
between negative evidence energy and inconsistency, with a Pearson correlation coefficient of 0.704. When EvA-E removes the feature above the erase threshold, we simultaneously eliminate most spurious feature identified by consistency, which is demonstrated in ( This observation underscores the effectiveness of EvA in erasing substantial amount of spurious feature while still maintaining core feature.
We further use Integrated Gradient (Sundararajan et al. 2017) to compute the saliency map of the value of feature with respect to the input, which shows that feature with lower evidence energy and consistency tends to focus spurious feature. We provide more analysis in the Appendix E.3
Effectiveness on Feature Erasure. We verify the ability to control the feature of our EvA framework on the CMNIST Dataset, as proposed by (Tiwari & Shenoy 2023). In the testing dataset where the colors are mixed across the two digits, we report the accuracy of predicting color labels and digit labels. If the model erases the target feature successfully, then it will have nearly 50% accuracy in predicting it but high accuracy when predicting the other feature. Notably, the target feature is determined by the validation dataset. For instance, if the target feature is color, then the label of validation data is also dependent on the color, and vere-versa.
Table shows that EvA easily controls the feature relies on to make predictions. Due to the simplicity bias (Zhu et al. 202T), empirical risk minimization (ERM) still tends to learn color features when the target is digit, failing to overcome the spurious correlation between color and digit. EvA overcomes it by erasing the other feature, showing consistently high accuracy on the target feature.
Size of Unbiased Dataset. We evaluate the impact of the unbiased dataset size on test accuracy and compare with direct training on the CMNIST dataset. The size of the biased training set is fixed at 2048, while varying amounts of unbiased data are supplied to assess the minimal data required for effective debiasing. As shown in Table both EvA-E and EvA-C achieve high performance even with a very small fraction of unbiased dataset (0.7% of the training set). Additional results on the Waterbirds dataset, compared with DFR, presented in Figure (c).
Data Efficiency:
Data Efficiency: Applicability to Biased Datasets without Spurious Feature Labels. We evaluate EvA-E against two sfate-of-the-an approaches that do not use unbiased datasets for tuning, JTT (Liu ett al. 202[) and DivDis (Lee et al. 2023), on Waterbirds using ResNet50. Following DivDis, when the validation set is unbiased, the model is tuned using the Worst Accuracy across groups Otherwise,
7

---

Published as a conference paper at ICLR 2025
Sie unlilee shevet
206
1935
JTT Uaiees
BHDD
Ev E(Oan)
Boau
DiNe
WlAvor
09.3 97.
Dute: Wore
$6.7%
Table 3: Accuracy of different methods across vari- ous sizes of provided unbiased datasets with standard deviations on CMNIST. It's noteworthy that EvA-E and EvA-C can debiased the model well even only using unbiased datasct with very small size (16).
Table 4: Worst Accuracy of methods tuned on
ResNet-50 with different datasct. Compared to previ- ous state-of-the-ar methods, our method EvA-E achieve much better performance even with biased dataset (85.8% against 81.0% and 62.5%).
BAR Top-1 Accuracy
ColobA
Waterbirds
Method
Extra Data
Unbiased
Conllicting
Accuracy
Worst
FLOP
FRM
X
60.51 #4.3
70. 20.2
62.52a0.2
94.10
63.74
10*
LFF (Nam ct al. 2020
X
62 98
84. 24.24
81.24
2 10"
LWBC
Kim et a 2022
68. 15:
88. 1.
87. 87.22
2 10P
Debian
Li al 2022
69.88 12.
86.74_aa
85.33
10P
SIFER Tiwari Shendy
202.3
72.08
89.00.a.v
04.0441
96.1120.4
71.22
10P
EvA-B
73,70 1o..
90.51
88.74
96.95 uu.
81.31
10
DFR
Kirichenko et al.
2022
83.23
90. 90.89_o.4
90.11
91.70.0.7
83.3220.4
10
EvA-C
89.4321.0
91.32_0.2
90.39_a..
92.4820.1
86.70go.5
10
Table 5: Performance (%;; mea on CelebA Waterbirds BAR Test Dataset. The second column indicates
if the method uses the information of validation datasct (V) for training. Our method outperforms other methods under both settings while significantly reducing the computation cost (from 10 to 10²). Here, we use ResNet18 for all methods for fair comparison. Note these may differ from the original paper, which report ResNet50 results; we provide ResNet-50 results on CelebA and Waterbirds in Appendix E.2
it is tuned using the mean accuracy since spurious feature labels are unavailable in biased datasets to align with real world application. As shown in Table 4 our method performs competitively on the unbiased dataset and significantly outperforms DivDis (by 4.8%) and JTT (by 23.3%) on the biased dataset. This demonstrates that EvA-E is more data-efficient. Similarly, EvA-C is also more data-efficient than DFR, which we further show in Appendix that EvA-C is more robust to biased dataset without spurious feature label.
Computation Efficiency, Most methods not relying on specific spurious feature information (Twari & Shenoy 2023 Nam al. 2020 Kim al. 2022) need to introduce hyperparameters to control the feature. The computation expense of hyperparameter selection is commonly ignored in the previous work. However, this is serious to be considered to apply these methods in the real world. Since the selection module in EvA is based on post-hoc retraining linear layer (FLOP ~ 103), we are six orders of magnitude more efficient than other methods that require retraining the whole neural network (FLOP > 10º) (Yu et al. [220 Nam et al. 2200 TTwari & Shenoy, 2023) shown in Table In our experiments, aimed at selecting the best hyperparameters from 90 candidates, EvA-E significantly reduces computation time while achieving higher accuracy within 10 minutes, compared to the 6 days required by SiFER (Tiwari & Shenoy 2023) on a single RTX 3080 GPU. Additional discussions
and detailed FLOP computations are provided in Appendix E.2
4.3 COMPARISON TO STATE-OF-THE-ART
We conducted a comprehensive comparison of our proposed methods, EvA-E and EvA-C, against state-of-the-art approaches in both with extra data and without extra data settings, as summarized in T Our methods have substantial gain whether or not extra information and data are provided. Remarkably, on the BAR dataset, which presents unique challenges due its biased validation dataset, EvA-C outperforms DFR by a substantial margin of 6.2%. We further show the results on ResNet50 in the Appendix E.2 As we show Section 4.2 and Appendix E.2 our methods surpass state-of-the-art methods by 4.8% (EvA-E) and 4.6% (EvA-C) on Waterbirds when no spurious feature label provided with much smaller computation cost for tuning, indicating strong improvement on computation and data efficiency.
4.4 ABLATION STUDIES
Class-wise feature detection. EvA includes a strong consideration that each class has different spurious feature. This is important to guarantee the performance to overcome spurious feature since spurious correlation is dependent on the class. As shown in Figure (b). to each pair of classes, only small portion of erased feature is shared. Concurrently, Figure (a) shows the accuracy of EvA-E
8

---

Published as a conference paper at ICLR 2025
75
1.0
70
865
Accuracy
65
0.6
worrs 1oort-10
80
60
600
55
CX
60
0.2
[vA-E EVA-C
50
FRM
wolclass
wiclass
013
4
5
0.0
70
Erase ratio 180
40
60
80
(a)
(b)
(c)
Figure 3: (a) Accuracy of different settings the BAR dataset. The accuracy of EvA with class-wise erasure stably higher than the without class-wise
(b) Overlapping ratio of erased feature. Illustrates the overlapping ratio of erased feature for each class. Notably, the number of erased feature instances shared between the two classes is relatively low. (c) Impact of the erase ratio on Waterbirds' Worst Acc. With the increment in the erase ratio (c), the accuracy of the worst group for both EvA-E and EvA-C exhibits a trend of gradual improvement followed by steep decline.
on the BAR dataset, where class-wise crasure (w/class (w/class-wise is more powerful than the one without such consideration srooclass-wise))
Impact of erase ratio e, Figure (c) illustrates the effect the hyperparameter erase ratio on the accuracy of our model. For both EvA-E and EvA-C methods, initial increase in the erase ratio results in a gradual improvement in the accuracy for the worst group the Waterbirds dataset. However, this trend reverses into sharp decline once the erase ratio exceeds certain point-600 60% for EvA-C and 50% for EvA-E. This suggests that beyond these thresholds, the core feature is likely being removed, thus hindering the model's accuracy.
Impact of unbiased dataset size. Both EvA-C and DFR reweight based on additional data from the unbiased dataset. As Figure shows, the accuracy of EvA-C and DFR both increase when the unbiased dataset gets larger, though EvA-C consistently outperforms DFR. EvA-C requires less data than DFR have a relatively good improvement. We provide more ablation studies in Appendix, E.4
5 CONCLUSION
To address spurious correlation, we focus on each channel of deep feature, and formally define spurious indicators including consistency if the unbiased data available and evidence energy when there is unbiased dataset. Our findings demonstrate that erasing potential spurious feature and reweighting on core feature can mitigate spurious correlations, irrespective of the availability of additional information.
The proposed method, Erase spurious correlation with Activation (EvA), is distinguished by its remarkable efficiency and effectiveness in addressing spurious correlations in practical scenarios.
Data efficiency. primary advantage of EvA is its superior data efficiency. EvA-C outperforms deep feature reweighting (DFR) methodologies, yielding better results with fewer unbiased dataset or even biased dataset. When no unbiased dataset and spurious feature label is provided, A-A-E offers alternative debias deep learning models. Overall, EvA demands less data to achieve competitive outcomes in mitigating spurious correlations.
Computational efficiency. Our method boasts computational efficiency, requiring only one hyperparameter, the erase ratio, during the erasing phase. Unlike conventional debiasing techniques that typically depend complex network architectures, EvA simplifies the process by relying on alternative linear probing.
In summary, the EvA methodology heralds potent solution for combating spurious correlations, with its efficiency and adaptability to real world. By utilizing indicators including consistency and evidence energy, it identifies and erases spurious feature, thereby elevating the bias. Our contributions pave the way for future investigation into activation- based spurious feature detection and erasure, offering fresh perspective overcome the spurious correlation problem.
Acknowledgments This research was funded by the NUS Artificial Intelligence Institute (NAII) seed grant number NAII-SF-2024-005

---

Published as a conference paper at ICLR 2025
REFERENCES
Adewole Adamson and Avery Smith. Machine learning and health care disparities in dermatology. JAMA dermatology. 154411211,,- 2018.
Martin Arjovsky, Soumith Chintala, and Léon Bottou. Wasserstein gan, 2017.
Martin Arjovsky, Léon Bottou, Ishaan Gulrajani, and David Lopez-Paz. Invariant risk minimization. arXiv preprini arXiv:1907.0288 02799,2893.. 2019.
Sanjeev Arora, Nadav Cohen, and Elad Hazan. On the optimization of deep networks: Implicit acceleration by overparameterization. In International Conference on Machine Learning, pp- 244-253. PMLR, 2018.
Haoyue Bai, Rui Sun, Lanqing Hong, Fengwei Zhou, Nanyang Ye, Han-Jia Ye, S-H Gary Chan, and Zhenguo Li. Decaug: and semantic augmentation. In Proceedings of the AAAI Conference on Artificial Intelligence, pp. 6705-6713. 13, 2021a.
generalization via decomposed feature representation
Haoyue Bai, Fengwei Zhou, Lanqing Hong. Nanyang Ye, S-H Gary Chan, and Zhenguo Li. Nas-ood: Neural architecture search for out-of-distribution generalization. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 8320-8329, 2021b.
Li Deng. The mnist database of handwritten digit images for machine learning research. IEEE Signal Processing Magazine, (9(6):141118 2012.
Robert Geirhos, Moon-Heriik Jacobsen, Claudio Michaelis, Richard Zemel, Wieland Brendel, Matthias Bethge, and Felix A Wichmann. Shortcut learning in deep neural networks. Nature Machine Intelligence, 2(1
2020.
Kaiming Hc. Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 770-778, 2016.
Badr Youbi Idrissi, Martin Arjovsky. Mohammad Pezeshki, and David Lopez-Paz. Simple data balaneing achieves competitive corst-group-aceuracy In Conference on Causal Learning and Reasoning, pp. 336-351. PMLR, 2022.
Naycong Kim, Schyun Hwang, Sungsoo Ahn, Jaesik Park, and Suha Kwak. Learning debiased classifier with biased committce, Advances in Neural Information Processing Systems, 35:18403- 18415, 2022.
Polina Kirichenko, Pavel Izmailov, and Andrew Gordon Wilson, Last layer re-training sufficient for robustness to spurious correlations. arXiv preprint arXiv:2204.02933 2022.
David Krueger, Ethan Caballero, Joern-Henrik Jacobsen, Amy Zhang, Jonathan Binas, Dinghuai Zhang, Remi Le Priol, and Aaron Courville. -ut-of-distt generalization via risk extrapola- tion (rex). In International Conference on Machine Learning, pp. 5815-5818 58266. PMLR, 2021.
Ananya Kumar, Aditi Raghunathan, Robbie Jones, Tengyu Ma, and Percy Liang. Fine-tuning can distort pretrained features and underperform out- out-of-distribution arXiv preprint 22Xiv:2202.10054, 2022.
Tyler LaBonte, Vidya Muthukumar, and Abhishek Kumar. Towards last-layer retraining for group robustness with fewer annotations. Advances in Neural Information Processing Systems, 36, 2024.
Yann LeCun, Sumit Chopra, Raia Hadsell, M Ranzato, and Fujie Huang. tutorial on energy-based learning. Predicting structured data, 1(0), 2006.
Yoonho Lee, Huaxiu Yao, and Chelsea Finn. Diversify and disambiguate: Learning from underspeci- fied data, 2023. URL httt:shttss///rxxit wttp:/wwwwrr/////////0/2 / 03418
Timothée Lesort. Continual feature selection: Spurious features in continual learning. arXiv preprint arXiv:2203.0m0 000012, 2022.
10

---

Published as a conference paper at ICLR 2025
Gaotang Li, Jiarui Liu, and Wei Hu, Bias amplification enhances minority group performance. arXiv preprint arXiv:2309.0677 2023.
Zhiheng Li, Anthony Hoogs, and Chenliang Xu. Discover and mitigate unknown biases with debiasing alternate networks. In European Conference on Computer Vision, pp. 270-288. Springer, 2022.
Evan Z Liu, Behzad Haghgoo, Annie S Chen, Aditi Raghunathan, Pang Wei Koh, Shiori Sagawa, Percy Liang, and Chelsea Finn. Just train twice: Improving group robustness without training group information. In International Conference on Machine Learning, pp. 6781-6792. PMLR, 2021.
Weitang Liu, Xiaoyun Wang, John Owens, and Yixuan Li. Energy-based out-of-distribution detection. Advances in neural information processing systems, 33:21464-21475, 2020.
Ziwei Liu, Ping Luo, Xiaogang Wang. and Xiaoou Tang. Large-scale celebfaces attributes (celeba) dataset. Retrieved August, 15(201 11: 2018.
Junhyun Nam, Hyuntak Cha, Sungsoo Ahn, Jacho Lee, and Jinwoo Shin. Learning from failure: De-biasing classifier from biased classifier. Advances in Neural Information Processing Systems, 2020.
Mohammad Pezeshki, Oumar Kaba, Yoshua Bengio, Aaron C Courville, Doina Precup, and Guil- laume Lajoie. Gradient starvation: A learning proclivity in neural networks. Advances in Neural Information Processing Systems, 14:2 2021.
Shikai Qiu, Andres Potapezynski, Pavel Izmailov, and Andrew Gordon Wilson. Simple and fast group robustness by automatic feature reweighting. In International Conference Machine Learning. pp. 28448-8844888 PMLR, 2023.
Andrew Slavin Ross, Michael Hughes, and Finale Doshi-Velez Right for the right reasons: Training differentiable models by constraining their explanations. arXiv preprint
2017.
Shiori Sagawa, Pang Wei Koh, Tatsunori Hashimoto, and Percy Liang. Distributionally robust neural networks for group shifts: On the importance of regularization for worst-case generalization. arXiv preprint arXiv: 19l:::1117.131 2019.
Antoine Salmona, Julie Delon, and Agnès Desolneux. Gromov-wasserstein distances between gaussian distributions. arXiv preprint arXiv:2104. 07970, 2021.
Mukund Sundararajan, Ankur Taly, and Qiqi Yan. Axiomatic attribution for deep networks, 2017.
Asuka Takatsu. Wasserstein geometry of gaussian measures. 2011.
Damien Teney, Ehsan Abbasnejad, and Anton van den Hengel. Unshuffling data for improved generalization in visual question answering. In Proceedings of the IEEE/CVF international conference on computer vision, pp. 1417-1427, 2021.
Damien Teney, Ehsan Abbasnejad, Simon Lucey, and Anton Van den Hengel. Evading the simplicity bias: Training a diverse set of models discovers solutions with superior ood generalization. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 16761- 16772, 2022.
Rishabh Tiwari and Pradeep Shenoy. Overcoming simplicity bias in deep networks using a feature sieve. In JCML, 2023.
SS Vallender. Calculation of the wasserstein distance between probability distributions on the line. Theory of Probability & Its Applications, 18(4):784-786, 1974.
Yinan Wang, Wenbo Sun, Jionghua Jin, Zhenyu Kong, Xiaowei Yue, et al. Wood: Wasserstein-based out-of-distribution detection. arXiv preprint arXiv:2112.06384, 2021.
Zhi-Qin John Xu, Yaoyu Zhang, Tao Luo, Yanyang Xiao, and Zheng Ma. Frequency principle: Fourier analysis sheds light deep neural networks. arXiv preprint arXiv: 1901.06523 2019.
11

---

Published as a conference paper at ICLR 2025
Haotian Ye, Chuanlong Xie, Tianle Cai, Ruichen Li, Zhenguo Li, and Liwei Wang. Towards theoretical framework distrr-idit generalization. Advances in Neural Information Processing Systems, 34:23519-222 23-331, 2021.
Haotian Ye, James Zou, and Linjun Zhang. Freeze then train: Towards provable representation learning under spurious correlations and feature noise. In A/STATS, 2023.
Yaodong Yu, Kwan Ho Ryan Chan, Chong You, Chaobing Song, and Yi Ma. Learning diverse and discriminative representations via the principle of maximal coding rate reduction. Advances in Neural Information Processing Systems, 33:9422-9434 2020.
John R Zech, Marcus A Badgeley, Manway Liu, Anthony B Costa, Joseph J Tittno, and Eric Karl Oermann. Variable generalization performance of a deep learning model to detect pneumonia in chest radiographs: cross seotsonectional study. PLoS medicine, (5(11)=e102668) 2018.
Wei Zhu, Haitian Zheng, Haofu Liao, Weijian Li, and Jiebo Luo. Learning bias-invariant repre- sentation by cross-sample mutual information minimization. In Proceedings of the IEEE/CVF International Conference Computer Vision, pp. 15002- 15012, 2021.
12

---

Published as a conference paper at ICLR 2025
In the supplementary material, present
Section
A
Theoretical analysis of evidence energy and consistency in Section 3.4
Section
Algorithm of EvA with pseudo code.
Section
More discussion on the related work and the main difference with our methods.
Section
D
More detailed description of dataset utilized in the experiments.
Section Supplementary experiments results with 1) More details on the results of BAR dataset; 2) Experiments on ResNet50 mentioned in Section 4.1 further demonstrating computation efficiency and data efficiency of our methods; 3) Additional analysis on detected spurious feature including activation distribution and saliency map; 4) Additional ablation study with Lasso Regression and other possible indicators.
Note that all the notations and abbreviations here are consistent with the main manuscript.
A THEORETICAL ANALYSIS
In this section, we conduct a formal analysis of the characteristics pertaining to consistency and evidence energy, under reasonable assumptions. Our approach aligns with the frameworks established in Ye et al. (2023); Arjovsky et al. (2019). In this setting, the data tuple (æ, y) is generated via the following mechanism:
Tcore P. E Rdix1,
y = BIeore + Ecore:
(T y Expu E RRdxx1,
Drran
(12)
T. Tepu
Espu Rdxx1
Drest
x = (Xcores Tspu) E Bdxx
Here, Teore represents the core component, while Espa denotes the spurious component, which exhibits different distributions in Dimin and Diest The term coore signifies the noise associated with the core component, possessing a variance of noores and Capa represents the noise linked to the spurious component, with variance of ppp Both noise terms have mean of zero. The parameters 39 and are normalized coefficients, each with unit lg norm.
In our study, conceptualize the network using two key components: a feature extractor g and linear probing mechanism h. Simplifying our approach, we treat g as a linear layer. The activation function gi is then defined as linear transformation g applied to the input vector æ = (Ioore: Tepu). This simplification is aligned with prevalent theoretical models employed in the analysis of non- convex optimization challenges in deep learning, as elaborated in references Arora et al. (2018); Kumar al. (2022); Ye al.) (2023). Additionally, we base our analysis on the assumption that the training (2 (Dirinn) and testing (Disa) datasets are infinitely large and that training is conducted with an infinitesimally small learning rate. This perspective enables us to distinctly represent the model in terms of core and spurious features.
(x) = ( Toore + I Eaya:
(13)
where denotes the feature index, with a total count of m feature. Derived from the training dataset D the overall model can be expressed as:
② VeoveWeore + Uspu
(14)
Vcore
Wki Jeures
(15)
Uspa
Wei
(i) 9apu
(16)
i=1
13

---

Published as a conference paper at ICLR 2025
A.1 ANALYSIS ON CONSISTENCY
In our analysis, we demonstrate that feature with low consistency can be formally interpreted as having a greater reliance on spurious features, assuming the random variables follow a Gaussian distribution. To substantiate this claim, we reference established conclusions regarding the Wasserstein distance between two Gaussian distributions as detailed in Takatsu (201T); Salmona et al. (202T).
Lemma 1: Consider two independent random variables 21 ~ ((41 E and 22 (u2, Σ2). each following Gaussian distribution. The 2-Wasserstein distance between them is given by:
d' | μ2 Tr(Ei + Σ2 - -( E
(17)
Building upon this lemma, we present the following theorem: Theorem 1: Given data distribution:
((ooores Tspu)s y) Derains
((Ecoves I sppu)?? y) Dieses
Teore ~ N (Hoores E Decre),
écore ~ [0, Meoreds
Expu ~ N(0, Nepu)s
and considering two pieces of feature Φ and sh:
ola) (x) (())
Teore + (a) g Sspa:
(b)) (I) = -eore + Iepu Espu:
(b)
The consistency of (á(a) exceeds that of (1b) (i.e., C(ak) C(bk)) and only if (a) [[xspu]]
<
Igepu E[xspu]]-
Proof: Considering the assumption of infinite data points within the datasets Durin and Drests these represent continuous joint distributions of y). Given that Zspu N (0,, ). we deduce:
7" y + Capu with Cxpu.
Tspu (N²²² ys Nepu),
d spu
N N(0, Nepu).
Since Teore is independent of Z Zapus we can further deduce:
gla) Dirain)
Nggla) Mcore NigboraMcore
glall 70 y:
glol ²22 + good napu)
(la) (x Dress) Nigiod Mcore:
((a) 252
Iapa Pspa)-
Combining this with Lemma establish that the consistency of feature is:
(akk) - 1o1p) >7 TTl. (bk) -9g 7" yy-
(b) Notice that TFy E(xrppu. it's obvious that C(ok) -lybوn E [ppul and (bbk) = -1g E [3
Therefore, follows that ((ak) > (bbk) and only if 1g E [ppallll < Igsp E [Tepplll-
This theorem with an assumed model substantiates the claim that the spuriousness of feature be quantified through consistency, as higher consistency indicates a reduced reliance on spurious features and provides insights on the more complicated scenario.
14

---

Published as a conference paper at ICLR 2025
A.2 ANALYSIS ON EVIDENCE ENERGY
In this section, we delve into the analysis of evidence energy as a metric for estimating the spuriousness of specific feature, comparing the activation energies of two pieces of feature. Initially, let's consider the energy score of a sample derived from the Negative Log-Likelihood (NLL) Loss during training. The NLL loss, which minimized in this process, is defined follows:
Cull B(z, v) ~ p" (-log -log ef/a3/T
(18)
Σj=1 cfi(1)/T)
Given that the energy function is defined as E(x,y) v) -fy(z), we can rewrite the NLL loss as:
Cnll = E((x, y) ~ p" E(x,y) v) + log Σe e
E(nJi//T
(19)
j=1
The concept of evidence energy for (xx) is derived from the activation's 's contribution the negative free energy, expressed as:
K
E(x) = - llog
R(2,A)/T
(20)
k=1
and this can be approximated using Taylor Expansion:
F(a,,)/TT Wk
E (ik)
(z)
(21)
E F(a,e)TT e-
To elucidate what activation energy reveals, let us consider two pieces of feature, represented as follows:
)la) (x) = (a)
Tcore + (a) Espu:
(22)
(b) の (x) (b) Feore + Iepu Sspa
(23)
The comparison of their relative energies can be quantified by:
e E(z,k))
[(ak)
E (bb)
E(e,e/T (Wka (z))(x) WKb T0 (x))
(24)
De E
e F(Z,N//T
Ee=1 e Ec=ee- e-F(ar,) (Wka (goore + gggh #ppl - WEO (goore + g.ih Tspu)
(25)
e E(z,A)/T Ee E((,,)) ( Whaglede Roore + Whaalph I Ww Zcove + W Who
(26)
This formulation allows us to quantitatively assess how different feature leverage core versus spurious features in their activation energy. Specifically, our objective is prove the following theorem.
Theorem 2: Under the same condition given in Theorem 1, if 7spa < Moooe and W Wkagspiaa Watdepassppe holds that P (B(ak) E(()) 0.5 + R(]]spu: Moore)) where R(ysps Mcore) >> 0.
This theorem is particularly relevant in scenarios where the noise associated with the spurious feature is less than that of the core feature, i.e., the training dataset is biased. This indicates that spurious correlations are more easily learned. To establish the foundation for proving Theorem 2, we first introduce Lemma 2.
Lemma 2: Consider model D* === = Veore + 0 Tspu trained with empirical risk minimization, perfectly matching an infinite number data points in Durin- Nepu Mlore then both the expectation and variance of the spurious component's 's contribution exceed those of the core component. Formally:
B Tppp
E2 ( Tcoree
(27)
Var Tspu
Var (V Woone
(28)
15

---

Published as a conference paper at ICLR 2025
Proof of Lemma 2: Utilizing the conclusion from Ye et al. 2023), have:
1s 3, Papu + Veore
Neore
(ooore Uppu)
(29)
Nope
+ Moore
Considering = Booor + ccore and spu
yn + epu in Der, we derive:
neore
Uppu Tspu
(y YEspu),
Nepu Veore
Nepn
VeoveWeore
(y Becore)
Nepa
Meore
Given that [[Espa] become:
0 and [[cooee]
O, and considering ED [w] Bpoores the expectations
Teore
Ep D ( Copul
3yoores
Nspu Whore
Napu
Exa [ooee
Bcoore
Napa + Thore
Moreover, with Var[cspu] = napu: Var[ccore] neorer we find:
Varp,, Uipa Topul
(1/ppu + moore) ) (Ppp + " )
Ncore
Nspu
Varp,, I Weore Feorel
Weore + (Toore
(napp + 1ioore )
These derivations underpin the conclusion that when Nayu < Yoores the model is more inclined to rely on the spurious feature, as evidenced by IE [Capul in Lemma 2.
> E² [ccorrl and Var[Capul Var[Ccore] as posited
In proceeding with the proof Theorem based on the intermediate steps and outcomes provided in Lemma 2, aim evaluate the probability that feature exhibits higher activation energy when it is more dependent on spurious component compared to core component.
Proof of Theorem 2: In the following, denote
core Ka HeoreToone
(a)
Apa Wka "
(b) = Web Reore
spu Whs I
(b)
Notice that sian(E(ak) (a)
E((k))) sign( coore
(a)
(a)
(Ccore + c(pu)) Based on the condition
(6)
Capu
Wta Iop WAb Dspu-Tapu. we can obtain Capu
(a)
Capu- Our focus is on a specific scenario
characterized by a positive real number I, for which we examine the probability:
P(c[o) + clg)
coore c(b)
(b)
c(a)
cpp 1)
(30)
This probability can be rewritten as:
P(c (u) < t)
(31)
Using the computation results induced from Lemma 2:
Ccore N Menre Speove,
Dapu (Noore + &Ecore 37
(32)
Nipn + 12ree
(12 + more )
16

---

Published as a conference paper at ICLR 2025
Considering the independence of Ceore and ceore given by Teores their difference also follows Gaussian distribution:
(b)
N(0, 2 Nspu (Tooe BEoore 3t) (1ápu + Neore )
(b)
(a) core
(33)
core
Let Z. be the standard normal variable, where Z ~ N(., 1), Then P(ccore coore t) can be translated to:
t 12 (nfore + Eeore 3r) (N5pp + M
PIZ
(34)
v2.
Recall that our target is:
(() +
c(a) spa
core
+
(.b) spa
cpp
Spp
(35)
To obtain this, we integrate from 0 1. Because ceore is independent Capu with known parameter, we can decompose the probability in our objective as follows:
Piceore
.a) < 1) P(c
Capu
l)dl
1=0
P(Z < V2. ( Yeore (Naps B 8
Pz
)dl
TTpp (Roree B.orr 5²) (Mapu t
1=0
V2.
(ppu + Veure
We extract the residual expression as:
t (o + oone (Mipu moore 1
R(Aspus Noove)
P(Z <
1=
V2.
Pz(
Pz(
de
V2. Copu (11g + oo 97) (Mipu + )
V2.
Yeore (n2 + &eoor B2)
( t
Therefore, the original expression can be induced
Plclo)e
de <t) P(cle)
app = t)dt
10
P(Z
)P
dt + R(Nspuy Meore)
1=
V2.
Meore (n + (P tooore
V2
Meore (1pp Dc 8² (Di t
0.5 + R(Nspus 1core)
0.5
In the third line above, we use the condition that Meooe < Nspa which is true when the provided training dataset is biased. Theorem 2 indicates that if a feature relies more heavily on spurious feature, it is more likely exhibit lower evidence energy.
B ALGORITHM
EvA is two-stage methodology, comprising detection and reweighting phases. In the experimental implementation, assume the deep feature dimension is m. We crase [em] feature for each class. The measure of spuriousness is conducted using consistency in open-box setting and evidence energy
17

---

Published as a conference paper at ICLR 2025
in closed-bo boosed-bx setting. Following this, the Topk function is employed to select [em] pieces of feature with the highest spuriousness, as delineated in Algorithm
During the reweighting stage, the weight corresponding to spurious feature is 'frozen' and assigned a value of 0 in the linear layer. The final step involves selecting the reweighted linear layer based various erase ratio candidates, evaluating performance using the Eval function. The choice of evaluation metric in Eval depends on the specific hyperparameter goals outlined Section 3.1 The entire algorithm is comprehensively illustrated in Algorithm
Algorithm I: Detect (b, Dimin e) (4, D Dunin exx.)
Input:
training dataset Duiin with K classes, extra dataset Dex (optional); feature set 4 = {Ø(() i {1, m.. linear probing hg. where: 8 = ((W,,bb) Rm R k E (1,... K}}
HParams :erase ratio c:
Output:
spurious indicator P {pk E {0,1)m} k (1, K})
for k = I... do
initialize Pk (0, 0), P 0: for i m. do
+ E(ik) (((s)
end
Pk (Pkji.... Pkjiem) <- Topk(nn, Lemm);;
for i = l. do
8
Pks + 1 (pkr E Pp)
end
P - PU {pk):
10
11 end
Return P
Algorithm Reweight
Input:
training dataset Dusin validation dataset D, with K classes; feature set + = {م(() i i [....)]].
linear probing hgg where: 0 = [(W,, by) Rm x R k E K}
HParams eerase ratio candidates & = (r)
Output:
re-weighted linear probing hi-
Initialize o* =0 0
for ( 1, ...., T do
[PI. ........ PK] < Detect (p, Dusin Et);
for 1,.... K do
for i = 1, m do
Pt: then
Freeze(W
end
end
Wt P4 WR:
11
end
12
0, < arg ming Loss Dusin he)
e, < Eval(p. Dunbiused, hgg );
14
if vt > 8" then
V" = Ur 4* 8,
16
end
17 end
Return 8"
18

---

Published as a conference paper at ICLR 2025
C COMPARING METHODS
We provide additional details of comparing methods in this work based on two different categories: methods without training with extra data and methods require extra data for training. Then we discusses the main difference between our works and these comparing methods.
C.I WITHOUT EXTRA DATA AND GROUP LABEL
LIF. Nam et al. (020) proposes a method to reduce bias in neural networks without requiring explicit labels assumptions about the type of bias. They introduce dual-network debiasing strategy: the first network is trained to amplify its biases, while the second network is trained using samples misclassified by the first, encouraging it to learn beyond the biases. This approach effectively mitigates bias in various datasets and often outperforms traditional methods that need more supervision. However, it trade-off computation efficiency for data-efficiency, i.e., reduces the reliance on group label but requires extensive computation on hyper- parameter selection over two neural networks together.
JTT. Liu et al. (2021) also proposes two-stage method to improve worst-group accuracy without extensive group annotations. It first trains standard ERM model and then re-trains by upweighting misclassified examples, thereby focusing on groups that ERM struggles with. JTT closes 75% of the performance gap between ERM and group DRO across various tasks with spurious correlations, while only needing minimal group annotations for hyperparameter tuning. However, it still requires retraining the based neural network to tune several hyperparameter.
LWBC. Kim et al. (2022) introduces method to train debiased classifiers without using labels for spurious attributes. uses a committee of classifiers to identify and assign higher weights to data that do not exhibit spurious correlations. The committe,, which is intentionally biased and diverse, helps detect bias-confficting samples by agreeing on their prediction difficulty. It learns alongside the main classifier, gradually becoming less biased and emphasizing harder examples over time. However, it also suffers from computation efficiency with training a commitee of classifiers.
Debian. Li et al. (2022) also addresses biases in deep image classifiers without relying labels for protected attributes. DebiAN adopts similar idea with LfF (Nam et al.) 2020) and the framework consists of networks: a Discoverer that identifies unknown biases and Classifier that unlearns them through alternate training. DebiAN effectively mitigates multiple biases simultaneously where most previous methods focus on single type of bias. But it also requires extensive computation to tune the hyperparameter.
DivDis. Lee al. (2023) is another two-stages debiasing method without requiring label. DivDis first learns diverse set of hypotheses using unlabeled test data, and then selects the most robust one with minimal extra data. is shown to effectively identify hypotheses that rely on robust features. The computation cost can be reduced by training a singal model with different heads but still suffering from retraining the whole model during tuning stage.
Tiwari & Shenoy (2023) focus on general simplicity bias rather than only spurious correlation and proposes feature sieve. During training iteration, they make assumption on simple features mostly can be extracted from lower layer and add additional penalty to make it harder to classify based on feature in the shallow layer. The method is effective in debiasing but lack computation efficiency in tuning a large number of hyperparameters such designs of classifier for adding the panalty and the layer to append such classifier.
SiFER.
C.2 WITH EXTRA DATA AND GROUP LABEL
DFR. Kirichenko et al. (2022) significant improves the efficiency from previous methods by applying last linear layer re-training to robustify the whole classifier. The method is simple but effective and the work provides insights that the model learns spurious feature as well as core feature during training but just biased to the spurious feature. And this bias can be suppressed by only reweighting the last linear layer by a less unbiased dataset. However, our analysis shows that DFR may be worse than the method without extra data in terms of data efficiency where there and Ye et al. (2023) shows that DFR can only be effective when the extra training data is less unbiased, i.c., the noise exists in correlation with core feature and label should be less than the correlation with spurious feature. The
19

---

Published as a conference paper at ICLR 2025
requirement for utilizing data for training makes it relies more less biased data than those only use it for tuning hyperparameter.
C.3 DIFFERENCES BETWEEN OUR WORK AND PREVIOUS METHODS
We notice that almost all existing methods without group label two-stage where the first stage includes training and collecting information related to potential spurious correlation and removing explicitly indirectly in the second stage.
Commonly, the computation cost in the first stage is necessary to get the base model and debias on it. However, the computation cost the second stage is usually very large to tune the hyperparameter by retraining another neural network or whole neural network, which makes these methods data-efficient but less computation-efficien
On the contrary, DFR (Kirichenko et al.
2022 provides compute-efficien method by only reweighting linear layer in the second stage using unbiased dataset. However, its performance is only guaranteed when tuning with Worst Accuracy but not robust when there is no group label, weaken its data- efficiency.
EvA exploits the advantage from both sides. We observe that the deep spurious feature can be identified without group label but the activation distribution over training dataset (EvA-E) or validation
dataset (EvA-C). This makes EvA as computation efficient as DFR (Kirichenko et al. 2022 and even
more data-efficient than previous methods (Nam et al. 2020 Liu et al.) 2021 Kim al. 2022 Lee
et al. 2023 Tiwari& & Shenoy, 2023) not relying on group label as shown in Table 4
Recently, several works, including AFR (Qiu et al., 2023). BAM (Li et al. 2023), and SELF (LaBonte) et al. 2024). have also explored reweighting the last linear layer while relying on fewer no additional annotations. However, these methods primarily focus on the data level, identifying failure cases during the first training stage and addressing model bias by leveraging these detected cases. In contrast, our approach focuses on debiasing the channel-lever characteristics of the features. A promising direction for future research is to combine feature-level and data-level methods to further enhance reweighting strategies for feature representation.
D DATASET DETAILS
In this section, we provide detailed group information and the specific size of each group in the datasets used for analysis and performance comparison. Each group is defined by human-identified feature and a label from the dataset, represented as (feature, label).
CMNIST. The ColOT-MNIST dataset Tiwari & Shenoy 2023) consists of four groups: (Red, Zero), (Red, One), (Green, Zero), and (Green, One). In the training dataset, the size of each group is (1024, O, O, 1024), respectively. In the validation and test datasets, the size of each group is (128, 128, 128, 128), making the datasets perfectly unbiased with respect to the defined feature. Consequently, models trained using ERM the biased training dataset struggle distinguish the digit label from the color attribute.
BAR. The BAR dataset Nam et al. (0200) standard benchmark for debiasing tasks. Although it does not provide explicit human-definee spurious features, it ensures that the background in the training images for each class is different from the background in the test images for the same class. The dataset contains images from six human action classes: "Climbing," "Diving," "Fishing," "Racing," "Throwing," and "Vaulting." It can be represented as 12 groups, with the first corresponding to the training dataset and the remaining 6 to the test dataset. The sizes of the groups the training dataset are (326, 520, 163, 336, 317, 279, 0, O, O, O, O, 0), while in the test dataset, the sizes (0, O, 0, 0, 0, 0, 105, 159, 42, 132, 85, 131), respectively. This dataset is challenging due to the complex spurious correlations and class imbalances. As shown in Table the performance of debiasing methods classes with fewer samples, such as "Fishing" and "Throwing, is often worse.
Waterbirds. The Waterbirds dataset Sagawa et al. (2019) is a benchmark for studying spurious correlations. It is synthesized from real-world data and includes four groups: (land, landbird), (land, waterbird), (water, landbird), and (water, waterbird). The sizes each group in the training dataset (3498, 18 184, 56, 1057), and in the validation dataset, the sizes (467, 466, 133, 133), respectively.
20

---

Published as a conference paper at ICLR 2025
Method
Climbing
Diving
Fishing
Racing
Throwing
Vaulting
Average
LIF
79.39 4..79
34.59 2.
75.39 3.66
1.90
33.72 67.92+4.19
a1.75
62.98 22278 84.70
DFR
92.08 -.188
93.55
70.39
92.97
92.33
EvA-C
90.47
24.7812.60
77.41
97.19
84.43
92.72
89,501..60
Table 6: Top-1 Accuracy (with standard deviations) of LfF (Nam et al., 2020) and DFR (Kirichenko et al. 2022) with EvA-C (Ours) across different classes over BAR dataset. Asterisks (*) indicate the best performance. Our method achieves much better results compared to previous methods.
Base Model
Waterbirds
CelebA
FLOP for debiasing
Debiasing Method
Worst Acc
Mean Acc
Worst Ace
Mcan Ace
ResNet50 + JTT
86.7% %
93.3% %
81.0%
88.0%
x 10" cnms
ResNet50 + DivDis
85.6%
55.0%
x 10 cnms
ResNet50 EvA-E
86.6%
92.5%
82.7%
88.7%
8 10a -emms
Table 7: Comparison of EvA-E (Ours) with two ofate-of-the-art methods, JTT Liu et al. (202T) and
DivDis Lee et al. (2023) without using extra data for training. The performance here is all tuned with
Worst Acc, i.c., the tuning dataset is with group label. The best performance is highlighted in bold.
Our method achieves competitive worst-case and mean accuracy with six orders of magnitude less computational cost.
The minority groups in the training dataset are (water, landbird) and (land, waterbird). Models trained with ERM on the entire training dataset tend to classify landbirds based land backgrounds and waterbirds based on water backgrounds.
CelebA Hair. The CelebA Hair dataset (Liu al. 2018 another standard benchmark for studying spurious correlations and has a relatively larger size. It is a real-world dataset containing four groups: (non-blond, woman), (non-blond, man), (blond, woman). and (blond, man). The sizes of each group in the training dataset are (71629, 66874, 22880, 1387), and in the validation dataset, the sizes are (8535, 8276, 2874, 182), respectively. The minority group in the training dataset is (blond, man). Models trained with ERM on the entire training dataset tend to incorrectly classify individuals with blond hair as women.
E ADDITIONAL EXPERIMENTAL RESULTS
E. PERFORMANCE OVER EACH CLASSES ON BAR DATASET
We present additional experimental results on the BAR dataset with ResNet18, focusing on the mean and standard deviation of accuracy for each class. Our method, EvA-C, compared against LfF (Nam et al. 2020) and DFR (Kirichenko et al. 2022), shown in Table EvA-C consistently outperforms the previous methods across most classes, demonstrating superior performance. Notably, it achieves significant improvements in the "Fishing" and "Throwing" classes, where the previous methods exhibited biases towards other classes.
E.2 COMPARISON WITH STATE-OF-THE-AA METHODS ON RESNET-50 AND VIT
We extend our experiments analysis on the backbone ResNet-50 to compare our methods with three state-of-the-art methods inleuding JTT (Liu et al.] 2021), DivDis (Lee et al., 2023) and DFR (Kirichenko et al., (022) to further demonstrate the computation and data efficiency of our methods while remaining competitive performance on both ideal and practical settings.
Comparison between EvA-E and Methods without Extra Data for Training: We compare the Worst Accuracy of EvA-E against JTT and DivDis on both unbiased and biased datasets, as shown in Table 4 in the main text. Our results demonstrate significant improvement in accuracy when tuning data is biased, highlighting the practical applicability of our method real-world scenarios. Following previous studies (Kirichenko et al. 2022 Sagawa et al. 2019), also present results using the standard comparison method, where Worst Accuracy is measured with known group labels, shown in Ta
In addition, we report the FLOPs in Table to quantify the computational cost of debiasing the model, where denotes the constant computation overhead for updating the gradient of each parameter,
21

---

Published as a conference paper at ICLR 2025
Conflicting Ratio
50%
40%
30%
20%
ResNet50 EvA-C
92.1
92.5
91.2
87.8
ResNet50 + DFR
90.7
89.0
86.6
83.2
Table 8: Comparison of the Worst Acc between EvA-C and DFR under different conflicting ratios on the Waterbirds Dataset. To make fair comparison where group label is not accessed in the real world, this result is obtained by tuning on Mean Acc rather than Worst Acc therefore the results is different from 92.9 in the original paper (Kirichenko et al., 2022) tuned with Worst Acc. Under the original setting using a "unbiased" dataset with a 50% conflicting ratio, our method still performs better than DFR. As the conflicting ratio decreases, i.e., the validation dataset more imbalanced, EvA-C performs better than DFR.
represents the number of epochs, is the dataset size, and s is the size of the hyperparameter search space. JTT requires retraining the entire model with a reweighted loss to select the appropriate weight for the error set, and DivDis also involves retraining the entire network to determine the weight of the mutual information loss during the diversification stage. In contrast, EvA-E only updates the linear layer for debiasing. For a ResNet50 model, the FLOPs for retraining the entire network are approximately 4 x 10", whereas updating the linear layer requires only about x 10 FLOPs. Therefore, the computational cost of EvA-E is significantly lower than that of previous debiasing methods.
Considering the results in Table 4 our method not only significantly improves computational efficiency and data efficiency but also achieves competitive accuracy.
Comparison between EvA-C and DFR with Extra Data for training: Apart from the setting where extra data is not provided for training, we also extend the results on ResNet-50 on the settings where extra data is available, i.e., comparing EvA-C with DFR. To make fair comparison where group label is not accessed in the real world, this result is obtained by tuning on average accuracy, ie.. Mean Ace rather than worst accuracy, i.e., Worst Acc even when the conflicting ratio is 50% where it's unbiased. As demonstrated in Table In the original setting, which has 50% conflctingg ratio, EvA-C outperforms Deep Feature Reweighting (DFR).
We further investigate how the performance of our methods improves with an increase in the conflicting ratio within the reweighting dataset. To manipulate the conflicting ratio, we selectively add non-conflicingg samples and remove conflicting samples into the tuning dataset. For instance, in the Waterbirds dataset, which encompasses four groups (water, water-birds). (water, land-birds), (land, water-birds), and (land, land-birds)- we adjust the conflicting ratio by adding samples from the (water, water-birds) and (land, land-birds) groups and eliminating samples from (water, land-birds) and (land, water-birds) to keep the size of tuning dataset as the same.
The data presented in Table g illustrate that our method significantly outperforms DFR as the conflicting ratio decreases. This result underscores the data efficiency of EvA-C, where EvA-C requires substantially fewer labeled data points to achieve relatively good results. This is in contrast to DFR, which is less robust to "biased" reweighting dataset.
Comparison between EvA-E and DFR on VIT and ImageNet Background Challenge: We additionally evaluated our method on ViT-B using ImageNet-9 [a], which involves background spurious correlations in real-world datasets. Our method improves the baseline model's accuracy from 87.9% to 88.7%, requiring no additional data or labels and only 10 minutes for hyperparameter tuning while DFR is harmful when performing on original dataset and only achieves 87.
E.3 ADDITIONAL ANALYSIS ON DETECTED SPURIOUS FEATURE
Activation distribution of detected spurious feature: We conduct a comparative analysis of the activation distribution between potential spurious feature, identified by evidence energy, and the remaining feature in the BAR dataset. This comparison visually represented in Figure Notably, the activation distribution disparity between the training and validation datasets is significantly more pronounced for the detected spurious feature than for the remaining feature. This further echos Theorem 2 that if the feature exhibits higher evidence energy, it's more likely to have higher consistency. And it's consistent with the empirical analysis on Section 4.2 Evidence Energy & Consistency.
22

---

Published as a conference paper at ICLR 2025
zhring
throwinn
pae vaultng
ai-00t Bra
Vill aieiooo
04
03
M4
111
acousic
04-
04
A2Peidon
actertion
Figure 4: Activation distribution of potential spurious feature (the first row), identified by evidence energy, compared to the remaining feature (the second row). The potential spurious feature exhibits more marked difference in activation distribution between the training and validation datasets than the remaining feature, which is categorized as core feature.
Eeore 0.0029
Ceore 0.192
E Espa 0.0019 Capss 0.339
(a) Input image (b) Saliency map of core feature (c) Saliency map of spurious feature
Figure 5: Saliency map of the activation on two feature annotated with their evidence energy (E) and consistency (C) for class "climbing" with respect to the input image (a) from BAR dataset (Nam et al. 2020). For action recognition task, background is considered as spurious feature while the human body is considered as core feature. (b) The feature with higher consistency and evidence energy, which is identified as core feature, focus more on the human body; while (c) the other feature
detected as spurious feature by EvA with lower evidence energy and consistency, is attributed to the background.
Saliency map of detected spurious feature: We further analyze the feature detected by both EvA-E and EvA-C by visualizing their corresponding saliency maps. We use the BAR dataset for this analysis, as it represents a real-world scenario where the distinction between core and spurious features is clear. the BAR dataset, which is designed for action recognition, the background considered spurious feature, while the human body region is regarded the core feature. For instance, the training set, the action "climbing" is predominantly associated with normal mountain background rather than snow-covered one.
To indicate which input component used for a specific feature, we use Integrated Gradient (Sun- dararajan et al. 2017) to compute the saliency map of the value of feature with respect to the input as Figure 5 shows. The core feature identified by both EvA-E and EvA-C assigns higher importance on the human body area while the detected spurious feature assigns higher importance the background, which is also aligned with human intuition.
E.4 ADDITIONAL ABLATION STUDIES
EvA v.s. Retrain Linear Layer with Lasso Regression: Notice that the implementation of DFR implicitly considering tuning the weight decay which is the strength of LI Norm. Therefore, the comparison between EvA and Lasso Regression last linear layer is directly reflected when comparing EvA and DFR.
Other spuriousness indicator: In closed- box setting, where additional data including spurious feature information is unavailable, we extend our ablation study to include evidence energy alongside other indicators like strength and instability.
23

---

Published as a conference paper at ICLR 2025
Class
climbing
diving
fishing
racing
throwing
pole vaulting
Evidence Energy
0.46
0.34
0.73
0.69
0.70
0,46
Strength
0.44
0.29
0.62
0.64
0.63
0.42
Instability
0.17
0.20
0.37
0.27
0.37
0.16
Table 9: Pearson correlation between three indices- their consistency on the BAR dataset. It is observed that evidence energy has a higher correlation with consistency compared the other two indices.
instability, and evidence energy- and
Climbing
diving
Sching
accing
throding
polc vaulting
7.5
20
200
1/5
1.5
1.3
1.0
02
C5
0.5
05-
00
0.0
0.0
Istaablty
Tnttat
Inaaeity
issabilly
Steegth
Srangh
Stargh
Seorgh
Strrgth
Smangh
25
35
3.5
25-
C11 283
2.33
5.31
0.0 0251
055
020 022
3.11 082 2.1 Noottive
10o (111 0.00 GUE QUS
Segatie
Figure 6: This figure illustrates the correlation between three indices-instabilins strength, and negative evidence energy-amn and inconsistency across six classes in the BAR dataset. In the subplots, each point represents data point from a specific class. The rows, ordered from top to bottom, correspond to instability, strength, and negative evidence energy, respectively. The analysis reveals that both negative evidence energy and strength exhibit positive correlation with inconsistency, while instability does not display similar correlation trend.
Concretely, the mean of feature's activation distribution is termed its strength, while the variance is defined as instability. For given feature g(f), class kt and dataset D, strength is formulated as:
S(g k, D) ExEDA 16 (x)
(x)
(36)
D&
RED
Similarly, instability is defined
1(600) ki D) Exe Ey ((d (() (,,0,,kDD D))
(x) (( k. D))²
(37)
ZEDz
where D₁ = (x () E D, = k} represents the subset of data with the ground truth label k.
In comparison to strength and instability, evidence energy demonstrates stronger correlation with consistency, as illustrated in Table9 and F This observation aligns with our findings from the BAR dataset. Experimentation with variations of EvA, including EvA-S (erasure based on strength) and EvA-I (erasure based on instability). yielded accuracies 71.81% and 64.81% respectively. These results are lower than the 73.70% accuracy achieved by EvA-E. Interestingly, strength is relatively good as well for detecting spurious feature. This may be because that the strength indicates the level of reliance on certain input component as well.
24


Reviews:
[{"review_id": "PQoTP8cwTm", "reviewer": "Reviewer_Jy78", "summary": "This paper presents an approach to mitigate biases learned through spurious correlations by erasing dimensions in the embedding space that are more likely to be associated with spurious features. The authors propose two metrics - namely consistency and evidence energy, for cases when additional unbiased data respectively are and are not available for training, which are used to identify spurious dimensions in the embedding space. Experiments reveal significant improvements over state-of-the-art bias mitigation approaches mainly in the absence of unbiased training data, but also to some extent when it is available.", "questions": "Please see the Weaknesses section.", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The authors propose a post-hoc method for developing robustness to spurious correlations, which does not involve any additional training / heavy computational workload, which is a big positive in my opinion. The method operates by identifying specific dimensions in the embedding space that satisfy certain threshold criteria for some forms of spuriosity metrics proposed by the authors, dropping them from the data representations, and solving a logistic regression from this updated embedding space to the label space. None of these operations seem to be computationally expensive.\n\n2. The authors propose ways to measure the degree of spuriosity of a specific dimension, when additional unbiased samples both are and are not available. This is a useful contribution which may also be applicable in other scenarios beyond bias mitigation.\n\n3. The experimental results are strong. Performance improvements over state-of-the-art are quite significant on standard benchmarks, with minimal extra computational overhead.", "weaknesses": "1. The calculation of both consistency (eq. 4) and evidence energy (eq 7) are done independently for each feature dimension $i$. However, it is well known that input features do not necessarily map neatly onto independent dimensions in the feature space [a, b]. Given this, the feature erasure in eq. 9 should technically not be achieving what it claims to achieve. By turning full dimensions on/off, it could be - (i) partially turning off some core features that could be in superposition with a spurious feature; and (ii) not fully turning off spurious features, since it may be in superposition with some core feature dimension which is not turned off.\n\n2. In computing $C_{(ik)}$ in eq. 4, there would be significant disparity in the distribution of the features stemming from sampling bias, since the number of unbiased samples is typically << the number of biased samples in practical datasets, which itself could skew the value of $d$, not providing a faithful measure of consistency. How do the authors account for this imbalance when applying the distance metric $d$?\n\n3. Since the idea of EvA is essentially to remove dimensions from the embedding space that correspond to spurious features, an ablation for what constitutes the best approach for selecting the subspace to drop is necessary. EvA employs thresholding on consistency and evidence energy to select spurious subspaces. However, whether consistency and evidence energy are really the best metric to look at when it comes to performing this selection needs to be either theoretically or empirically established. For this, one empirical baseline could be comparison with random feature drop out in the embedding space. I would suggest the authors to explore this in further detail and consider developing such baselines to establish the optimality of their specific feature erasure mechanisms.\n\n4. Table 5 shows that the gains from EvA are more significant in the absence of extra unbiased training data. The authors should provide a discussion on why their method might have a stronger advantage over SOTA when unbiased training data is not present.\n\n5. When extra training data is not available, how does one perform the hyperparameter search for $\\epsilon$? Additionally, when training data is available, although a single pass of EvA is computationally inexpensive, the hyperparameter sweep over $\\epsilon$ may be time consuming. Discussing these, especially, the former case of selecting $\\epsilon$ in the absence of an unbiased train subset is necessary.\n\n[a] Elhage, et al., \"Toy Models of Superposition\", Transformer Circuits Thread, 2022. \\\n[b] Radhakrishnan et al., \"Mechanism for feature learning in neural networks and backpropagation-free machine learning models\", Science 2024.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Z2ppCic1aG", "reviewer": "Reviewer_Feuq", "summary": "The paper introduces a method to detect and mitigate spurious correlations learned by an image classifier. They present two measures to identify channels in the penultimate layer of the model which correspond to such spurious correlations, one of them requiring an additional unbiased dataset (“consistency”) while the other can be computed on the the training data (“evidence energy”). After identifying a set of spurious channels, the corresponding weights in the last layer are set to zero and the remaining weights (of the last layer) are retrained.", "questions": "In Table 5, how many runs were performed to compute mean and std, and was the same erase ratio used in all of the EvA runs?\n\nSection 4.4 (and Fig. 3 c) discuss the effect of the erase ratio on the worst accuracy. How does this hyperparameter influence the mean accuracy?\n\nIn Table 7, mean acc and worst acc are reported for CelebA but unbiased/conflicting acc in Table 5. The results on Waterbirds suggest that your method results in a better worst acc but worse mean acc for the ResNet-50 than for the ResNet-18: Does this pattern hold on CelebA?\n\nGiven the computational efficiency and no requirement for unbiased data or annotations of the spurious correlations, it should be quite feasible to evaluate EvA-E on ImageNet scale spurious correlations benchmarks (e.g. [1],[2]). In these more realistic settings, the distinction between spurious and core features become more complex. Can you provide empirical evidence or arguments that your evidence energy measure can still be used for detection and mitigation in such settings?\n\n\n[1] Singla et al., Salient ImageNet: How to discover spurious features in Deep Learning?\n\n[2] Neuhaus et al., Spurious Features Everywhere -- Large-Scale Detection of Harmful Spurious Features in ImageNet", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "- computational cost are lower compared to similar methods that do not require unbiased data\n- their mitigation outperforms other methods on several common spurious correlation benchmarks\n- for the variant including unbiased data, the introduced method requires less samples for the same accuracy\n- the procedure seems to be well motivated including theoretical analysis", "weaknesses": "- the experiments establishing the correlation between their two measures are based only on one class of one of the datasets\n- experiments are only based on small toy settings with strong spurious correlations\n- the method is strongly inspired by OOD methods, but apart from a brief mention in the conclusion, this is not discussed in the main paper \n\nminor:  fontsize in Fig. 1, Tab. 3, Tab. 4 are too small, the definition of “energy” actually corresponds to the “free energy” which might be confusing", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "USavzyLp1T", "reviewer": "Reviewer_rgWi", "summary": "Existing methods on mitigating spurious correlations typically requires unbiased data and multiple rounds of retraining. The paper proposes a computational- and data-efficient pipeline that detects spurious features before the last linear layer. When there is no unbiased datasets, the contribution of each feature to the network’s prediction\nconfidence is exploited for detection.  If the unbiased data is available, the consistency of penultimate activations between spurious and unbiased datasets is used for detecting spurious features.  Through channel-based erasure and re-weighting on the final linear layer, reliance on spurious correlations can be effectively mitigated.", "questions": "- Why spurious features often lead to high confidence predictions?\n- Can the proposed method used for other model architectures beyond convolutional neural networks?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "- Two practical methods, namely EvA-C and EvA-E, are proposed for detecting and mitigating spurious features in the penultimate activations, depending on the existence of an unbiased dataset. The two proposed methods are both efficient and effective in mitigating spurious correlations.\n\n- The two metrics originate from the spurious feature detection methods can be used to measure the spuriousness of features from the penultimate layer of a model. These metrics are also useful to validate the effectiveness of spurious correlation mitigation methods.", "weaknesses": "- Contrary to the claim in Line 93, the performance of the proposed method is not the state-of-the-art in comparison with some methods [1,2]. For example, with ResNet50 as the backbone, EvA-E has a worst-group accuracy of 86.6% (Table 7), while [2] achieves 89.1%.\n\n- The unbiased dataset $\\mathcal{D}\\_{\\text{unbiased}}$ is first introduced in Line 156; however, it is unclear what $\\mathcal{D}\\_{\\text{unbiased}}$ can be considered as \"unbiased\". Based on the descriptions in Line 230 and Line 312, $\\mathcal{D}\\_{\\text{unbiased}}$ is selected from the validation set. In some datasets, such as CelebA, the validation set also has gender bias. If $\\mathcal{D}\\_{\\text{unbiased}}$ is defined as a set of group-balanced data where each class of samples distribute equally across different spurious features, then group annotations are required to obtain $\\mathcal{D}\\_{\\text{unbiased}}$, contrary to the claim in Line 50-51. Further clarification on this point would be beneficial.\n\n- In Eq. (4), why the unknown test distribution $\\phi_{\\text{test}}^{ik}$ can be approximated via $\\phi_{\\text{unbiased}}^{ik}$? In some datasets such as CelebA, the test set is also biased. It would be helpful to further clarify this point.\n\n[1] LaBonte et al., Towards last-layer retraining for group robustness with fewer annotations, NIPS, 2023.\\\n[2] Li et al., Bias Amplification Enhances Minority Group Performance, TMLR, 2024.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "7QwyZ0RuHC", "reviewer": "Reviewer_iH3N", "summary": "This paper proposes EVA, a method for addressing spurious correlations in datasets. It identifies spurious features using a consistency measure and the evidence energy measure in scenarios with and without unbiased datasets, respectively. Theoretical analyses are provided regarding the relationship between these two measures and a feature’s spuriousness. Experiments are conducted to demonstrate the method's effectiveness.", "questions": "See the questions raised in the Weaknesses section.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "- The method is well-motivated, and I appreciate its simplicity (in terms of both understanding and computation).\n- Providing theoretical analysis adds a more principled foundation to the two measures.\n- There are experiments showing that the erased features are indeed the spurious ones on CMNIST, and the correlation between evidence energy and consistency, which further supports that the method works as expected.", "weaknesses": "- Since the original DFR paper uses ResNet50 as the architecture, it would be a good sanity check to compare results in that same setting. The authors provided results for ResNet50 in Appendix E.2, but they mention that, to mimic real-world scenarios, they tuned hyperparameters based on Mean Accuracy instead of Worst Accuracy. This might be debatable, as the setting where DFR is applicable assumes an unbiased validation set. Additionally, I wonder if the setup is consistent between ResNet18 in the main paper and ResNet50 in the appendix for the Waterbirds dataset. Could you clarify whether hyperparameters for ResNet18 were tuned based on Mean Accuracy or Worst Accuracy?\n- Overall, since most existing papers evaluate their methods on ResNet50 while this paper primarily uses ResNet18 (this makes the numbers not comparable to those reported in other papers; in general, the numbers are much lower for all methods than those in the literature potentially due to the architecture), and some baselines (e.g., JTT, SSA [1], AFR [2], and [3]) are not included in the main table, Table 5, it is hard to compare the proposed method to the state of the art and to assess whether it truly stands out among existing techniques. For example, I wonder if the proposed method can outperform JTT on the Waterbirds dataset when ResNet50 is used, which has a reported worst-group accuracy of 86.7%, a relatively high value compared to the numbers in this paper obtained on ResNet18.\n- Perhaps a detailed discussion and comparison with [1] is necessary, given that it also achieves the same goal: lightweight reweighting of the last layer without group labels.\n\n\n[1] Qiu, Shikai, et al. \"Simple and fast group robustness by automatic feature reweighting.\" International Conference on Machine Learning. PMLR, 2023.\n\n[2] Nam, Junhyun, et al. \"Spread spurious attribute: Improving worst-group accuracy with spurious attribute estimation.\" arXiv preprint arXiv:2204.02070 (2022).\n\n[3] Liu, Sheng, et al. \"Avoiding spurious correlations via logit correction.\" arXiv preprint arXiv:2212.01433 (2022).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)