NeuronTune: Towards Self-Guided Spurious Bias Mitigation
Authors: Guangtao Zheng
Wenqian Ye
Aidong Zhang
Venue: ICML 2025 poster (2025)
Keywords: spurious correlation
robustness
bias mitigation
Primary area: deep_learning->robustness
TL;DR: We introduce NeuronTune, which mitigates spurious bias in neural networks by adjusting biased neurons without needing external annotations.

Abstract: Deep neural networks often develop spurious bias, reliance on correlations between non-essential features and classes for predictions. For example, a model may identify objects based on frequently co-occurring backgrounds rather than intrinsic features, resulting in degraded performance on data lacking these correlations. Existing mitigation approaches typically depend on external annotations of spurious correlations, which may be difficult to obtain and are not relevant to the spurious bias in a model. In this paper, we take a step towards self-guided mitigation of spurious bias by proposing NeuronTune, a post hoc method that directly intervenes in a model's internal decision process. Our method probes in a model's latent embedding space to identify and regulate neurons that lead to spurious prediction behaviors. We theoretically justify our approach and show that it brings the model closer to an unbiased one. Unlike previous methods, NeuronTune operates without requiring spurious correlation annotations, making it a practical and effective tool for improving model robustness. Experiments across different architectures and data modalities demonstrate that our method significantly mitigates spurious bias in a self-guided way.

Nune:nTune: Towards Self-Guided Spurious Bias Mitigation
Guangtao Zheng'
Wenqian Ye Aidong Zhang'
of waterbird (Sagawa al., 2019), as the background of Deep neural networks often develop spurious bias, reliance on correlations between non-essential fea- tures and classes for predictions. For example, a model may identify objects based on frequently co-oceurring backgrounds rather than intrinsic features, resulting in degraded performance on data lacking these correlations. Existing mitiga- tion approaches typically depend on external an- notations of spurious correlations, which may be difficult to obtain and are not relevant to the spu- rious bias in a model. In this paper, we take a step towards self-guided mitigation of spurious bias by proposing NeuronTune, a post hoc method that directly intervenes in a model's internal de- cision process. Our method probes in model'ss s latent embedding space to identify and regulate neurons that lead to spurious prediction behav- iors. We theoretically justify our approach and show that it brings the model closer to an unbi- ased one. Unlike previous methods, NeuronTunn operates without requiring spurious correlation an- notations, making it a practical and effective tool for improving model robustness. Experiments across different architectures and data modalities demonstrate that our method significantly miti- gates spurious bias in a self-guided way.
Abstract
the water is spurious attribute. In contrast, core attributes, such as bird feathers, causally determine class. A model with spurious bias may achieve a high prediction accuracy (Beery et al., 2018: 8: Geirhos et al., 2019; 2020; Xiao et al., 2021; Zheng et al., 2024a) even without core attributes, such as identifying an object only by its frequently co-occurring background (Geirhos al., 2020). However, the model may perform poorly on the data lacking the learned spurious correlations, which poses great challenge to robust model generalization.
Existing methods (Sagawa et al., 2019; Kirichenko et al., 2023; Deng et al., 2024) that mitigate spurious bias are mostly at the sample level, using a curated set of samples with annotations of spurious correlations called group labels to retrain biased model. A group label (class, spurious attribute) annotates sample with spurious attribute and its class label, representing spurious correlation. However, group labels are difficult to acquire and often require costly human-guidee annotations. To circumvent this, group label estimation (Nam et al., 2022) and various sample reweight- ing mechanisms (Nam et al., 2020; Liu et al., 2021; Kim et al., 2022; Qiu et al., 2023; LaBonte et al., 2024) adopted using the idea that spurious bias can be identified through the misclassification of bias-conflicting samples.
Despite significant progress in spurious bias mitigation, ex- isting sample-level methods that rely on group labels or sam- ple reweighting offer limited and indirect control over how spurious bias is addressed. On the one hand, group labels are data annotations that are external to model and may not accurately reflect the specific spurious bias developed in the model. On the other hand, sample reweighting does not directly target the internal mechanisms that give rise to spurious bias. This highlights the need for self-guided approach that directly intervenes in a model's decision process, providing more targeted and model-relevant signals for mitigating spurious bias than sample-level approaches.
1. Introduction
Deep neural networks trained using empirical risk minimiza- tion (ERM) often develop spurious bias: a tendency to rely on spurious correlations for predictions. A spurious correla- tion refers to non-causal relationship between a class and an attribute that not essential for defining the class, com- monly referred as a spurious attribute (Ye et al., 2024). For example, the class of waterbird and the background of the water can form spurious correlations in the predictions
To this end, we focus on developing self-guided methods that directly analyze the internal prediction mechanism of a model to identify components of the model that are af- fected by spurious bias and then mitigate their influence to final predictions. We take a step towards this goal by proposing a novel method termed Neuron Tune,,Tuuee which sys-
'Department of Computer Science, University of Virginia, Charlottesvillle VA, USA. Correspondence to: Guangtao Zheng virginiiiiinn...du>....
Proceedings of the 4nnd International Conference Machine Learning. Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).

---

NeuronTune: Towards Self-Guided Spurious Bias Mitigation
tematically reduces spurious bias in deep neural networks. NeuronTunne first probes in the latent embedding space of trained model to identify dimensions (neurons) of sam- ple embeddings affected by spurious bias, termed biased dimensions-thow those where spurious attributes predominantly contribute to prediction errors (Bykov et al., 2023; Singla & Feizi, 2021). Those dimensions can be identified when high activation magnitudes are strongly associated with in- correct predictions, indicating that features represented by those dimensions are not truly predictive of target classes. Importantly, rather than attempting to explicitly distinguish dimensions representing spurious and core attributes, an inherently challenging task given the complex entanglement of features in deep networks, NeuronTune instead identifies biased dimensions and suppresses the contributions of the these dimensions to final predictions. This intervention en- courages the model to discover robust decision rules and mitigates spurious bias in the model.
neck (Tartaglione et al., 2021) the distributional distance between bias-aligned samples (Barbano et al., 2023), also proved to be effective in spurious bias mitigation. The concept of neural collapse has also been exploited recently for spurious bias mitigation (Wang et al., 2024). However, this setting requires to know what spurious bias needs to be mitigated a priori and only focuses on mitigating the specified spurious bias.
Semi-Supervised Spurious Bias Mitigation. This setting aims to mitigate spurious bias without extensive spurious correlation annotations. A small portion of group labels in a held-out set are required for achieving optimal per- formance. One line of works is to use data augmentation (Zhang et al., 2018; Han et al., 2022; Wu et al., 2023; Yao et al., 2022). Some methods propose to infer group labels via misclassified samples (Liu et al., 2021), clustering hid- den embeddings (Zhang et al., 2022), or training a group label estimator (Nam et al., 2022). Creager et al. (2021) adopts invariant learning with inferred group labels. Other approaches include using biased models (Bahng et al., 2020), poisoning attack (Zhang et al., 2024), and mining intermedi- ale attribute samples (Zhang et al., 2023). Recently, LaBonte et al. (2024) proposes to only use small set of samples with group labels selected by the early-stop disagreement criterion. Relevant to our method are the last layer retraining (Kirichenko et al., 2023; Qiu et al., 2023) methods which only retrain the last layer of a model. In addition to re- training the last layer to minimize computational overhead, our method intervenes the internal decision process of a targeted model for more relevant and targeted spurious bias mitigation than existing methods.
Compared with the existing sample-level methods for spu- rious bias mitigation, NeuronTune provides direct inter- vention at the neuron level, allowing for more precise and targeted control over the mitigation of spurious bias during model tuning. Unlike approaches that rely on sample-level annotations such group labels, NeuronTune enables the model to self-debias without externl supervision. This makes it applicable in standard ERM training settings, where no additional annotations beyond class labels are available. As result, NeuronTune serves as a practical and effective post hoc tool for mitigating spurious bias.
We theoretically demonstrate that neuron activations cou- pled with their final prediction outcomes provide self- identifying information on whether the neurons are affected by spurious bias. Our theoretical findings further suggest a practical metric for identifying biased dimensions and proves that NeuronTune can bring a model closer to the unbiased one, Experiments on vision and text datasets with different model architectures confirm the effectiveness of our method.
U Unsupervised Spurious Bias Mitigation. The goal in this setting is to train a robust model without using any group labels. Typically, we would expect relatively lower per- formance for methods working in this setting than in the above two settings as no information regarding the spuri- ous correlations in test data is provided. Prior works use vision-language models to extract group labels (Zheng et al., 2024c;b), upweights training samples that are misclassified by a bias-amplified model (Li et al., 2024), or regularizes model retraining with detected prediction shortcuts in the latent space of a model (Zheng et al., 2025). A recent work (He et al., 2025) uses the observation that features with high confidence are likely to be spurious and mitigates spurious bias by erasing the corresponding activations. Our method considers the activation patterns of both correctly and incorrectly predicted samples of the same class to iden- tify and mitigate spurious features.
2. Related Work
Depending on the availability of external supervision, we summarize prior spurious bias mitigation methods into su- pervised, semi-supervised, and unsupervised categories.
Supervised Spurious Bias Mitigation. In this setting, cer- tain spurious correlations in data are given in the form of group labels. With group labels in the training data, balanc- ing the size of the groups (Cui et al., 2019; He & Garcia, 2009), upweighting groups that do not have specified spuri- ous correlations (Byrd & Lipton, 2019), or optimizing the worst-group objective (Sagawa al., 2019) can be effective. Regularization strategies, such as using information bottle-
3. Problem Setting
We consider a standard classification problem. The training set Dunin {(x,)) E X, y E }) typically contains
2

---

NeuronTunee Towards Self-Guided Spurious Bias Mitigation
data groups Di with Dasin = Dg Dy. sample in the input space X, is the corresponding label in the finite label space J' g ;= (y, a) denotes the group label defined by the combination of a class label y and a spurious attribute a E A. where A denotes all spurious attributes in Duain: and G denotes all possible group labels. Sample-label pairs in the group g have the same class label y and the same spurious attribute a.
gi where x denotes
Bern(p) is Bernoulli random variable, and p is close 1, indicating that Xapa is mostly predictive of y but not always. In (1) and (2), B e R Pixl and E IR Pax1 are coefficients with unit L norm, and Score E IR and Epu E D2x1 represent the variations in the core and spurious components, respectively. We set Ecore and each element in Espu as zero-mean Gaussian random variables with the variances Moune and nepu respectively. We set Veooe > Vspu to facilitate learning spurious correlations (Sagawa et al., 2019).
where (2a 1) E [-1,+1],
Our Scenario. We consider unsupervised spurious bias miligation, where no group labels are available, resembling a standard ERM training. A commonly used performance metric is the worst-group accuracy (WGA), which is the accuracy on the worst performing data group in the test set Diau i.e., WGA = minges (c, ) where Die denotes group of data in Disst with Diest = Ug 5 and f denotes trained model. Typically, data in Duain is imbalanced across groups, and the model I' tends to favor certain data groups, resulting in a low WGA. Improving WGA without knowing group labels during training is challenging.
Prediction Model. We adopt a linear regression model with two linear layers (Ye et al., 2023) defined as [(x b" Wx, where W E MR x denotes the embedding matrix simulating a feature extractor, b E M denotes the last layer, and M is the number of embedding dimensions. The model [(x) can be further expressed as follows,
f(x) Σ by (x Weore, + X Wapu,s)
(3)
X Meore + xpu Uspu:
4. Methodology
where Weore, E R Pixx W Wapu,i R w. [wT "o w spu E
We propose NeuronTane, self-guided method for mitigat- ing spurious bias without requiring group labels. Neuron- Tune identifies neurons (dimensions) affected by spurious bias in a model's 's latent space and tunes the model while sup- pressing the identified neurons. In Section 1, we present an analytical framework that outlines the design principles and theoretical properties of NeuronTune. Section 4.2 in- troduces practical implementation for mitigating spurious bias in real-world settings.
RIID is the i-th row of W, Meore
Σ ii Wore,,, and Uspa = EM bs The training objective is ey (W, b) 2 (x.)) -1l)
To better understand our data and prediction mod- cls, consider that a in Eq. (2) controls subpopulations in data, e.g., when a 1, it may represent a group of water- birds on water, and when a it may represent a group of waterbirds on land. The probability p controls the severity of imbalance in subpopulations. When is close to one, the data is severely imbalanced in subpopulations. After train- ing with ERM, the model minimizes the training loss, i.e., maximizes the average-case accuracy, but obtains a large nonzero weight on the spurious feature (Lemma in Ap- pendix) and is away from the optimal model (Corollary in Appendix). For example, the model may focus on correctly classifying waterbirds water, the expense of its ability to recognize waterbirds on land.
Remark:
4.1. NeuronTune: An Analytical Framework
At the core of Neuron'Tune is the identification of neurons that are affected by spurious bias. We establish analytical framework to (1) elucidate the principle of neuron selection, (2) derive a selection metric that follows the principle of neu- ron selection, and (3) reveal the mechanism of Neuron'l Tune in mitigating spurious bias.
1. DATA AND PREDICTION MODELS
4.1.2. PRINCIPLE OF NEURON SELECTION
Data Model. We design data generation process that facilitates learning spurious correlations. Following the setting in (Arjovsky et al., 2019; Ye et al., 2023), we model a sample-label pair (x, y) in Durin as:
CeuronTune aims to identify neurons that reflect spurious bias. Proposition 4.1 specifies the principle of NeuronTunee in terms of what neurons are to be identified and suppressed during model tuning.
x = Xeore ① Xapu E Dx1, y = 3" 3"xxoor
Xcore + Ecore
(L)
Proposition 4.1 (Principle of NeuronTune). Given the model f(x) = ba bTx trained with the data specified in (1) and (2), it captures spurious correlations when T" W Wapu. < 0, E (l....,M, The principle of Neuron- Time is to suppress neurons containing negative o W
where X.ore E R Dix1 1 is the core component, @ denotes the vector concatenation operator, and the spurious component Xspu E D2x1 with Dj + D2- = D is associated with the label y with the following relation:
X = (2a- 1)yy + Espu; a ~ Bern(p),
If 1 w ii 2 0, the model handles the spurious component
3

---

Neuron Tune: Towards Self-Guided Spurious Bias Mitigation
(4) Extract Latent Emboddings
(b) Identity Biased Neurons
(c) Retrain with Selected Neurons
Cassiler hog
Festure Extractor eti
Classifter heg
Feature Extractor Cd
Lanent Embredeingg
Drone
Men
For each neuron from 1 to M:
Suppress
Daee
Select Biased Dimensions
8/ Manis Maxe
V1 V2 V3 V2
Figure 1. Practical implementation of NcuronTune. (a) Extract latent embeddings Vi,..., Vx and prediction outcomes (bluc for correct and red for incorrect predictions) from an ERM-trained model using the identification data Dise (b) Identify biased neurons (dimensions) utilizing the statistics Mis and Mox derived from neuron activations for correct (bluc) and incorrect (red) predictions from Eq. (5). (c) Retrain the last prediction layer on Drome while keeping the feature extractor frozen and suppressing identified biased dimensions.
conrectly. Specifically, when a = 1, the spurious component Xpp positively correlates with the core component Xcore and contributes to the output, whereas when a = 0, its correlation with Xgor breaks with a negative one and has a negative contribution the output. The relations reverse when 7" W 0, i.e., the model utilizes Xup even when the correlation breaks. demonstrating a strong reliance the spurious component instead of the core component. The proof is in Appendix A.2.4.
Let Mmis = Me) and Meor = ())() Intuitively, a high Mmis indicates that high activations at the i-th di- mension contribute to misclassification when predicting the class y. A low Mearr implies that the i-th dimension has little effect in correctly predicting the class y. Thus, large dif- ference between Miss and Marr i.e., large 6% indicates that the i-th dimension represents features that are irrelevant to the class y. In other words, with a high likelihood, the dimension is affected by spurious bias. In contrast, a neg- ative 5 highlights the relevance of the i-th dimension for predictions as most correctly predicted samples have high activation values in this dimension, and most incorrectly predicted samples have low activation values.
4.1.3. METRIC FOR NEURON SELECTION
Guided by the principle of NeuronTune in Proposition 4.1, the following theorem gives practical metric to select neurons that are affected by spurious bias.
Remark: Proposition 4.1 and Theorem 4.2 state that when spurious correlation breaks, neurons that continue to pos- itively contribute to mispredictions will be selected. For example, in the case of waterbird with water and land back- grounds, neurons that cause misclassification on images of waterbird appearing on land will be identified.
Theorem 4.2 (Metric for Neuron Selection). Given the model fxx = Wx, we cast it to a classification model by training it to regress y E {-p,,,) v.} (μ > 0) on x bused on the data model specified in (1) and (2), where p. B|ø" xwor]- The metric 8y defined in the following can identify neurons affected by spurious bias when 8y 0:
4.1.4. MECHANISM OF NEURONTUNE
8 Med(V Med()"),
Neuron Tune mitigates spurious bias by retraining the last layer while suppressing (zeroing out) the identified neurons. The following theorem shows that this improves model ro- bustness and explains how it achieves this.
where V. and D are the sets of activation values for mis- classified and correctly predicted samples with the label y from the i-th neuron, respectively; an activation value is defined as T Weore, + spu W W, and Med(.) returns the median of an input set of values.
(Neuron Tune Mitigates Spurious Bias). Consider the model f*( x "" trained on the biased training data with p >> 0.5, where u =IueT ux Un- der the mild assumption that 3J Weon = = W.p.i. i Vi
Theorem 4.3
We show in Appendix A.2.5 that the theorem establishes the approximation 6y ~ -2py""Wssu.i Wspu, which confirms that neurons selected by the metric defined above follow the principle in Proposition 4.1. Adopting medians in the metric makes the metric robust to outlier values.
1,...,A M, then applying NeuronTume to fx produces a model that closer to the unbiased one.

---

NeuronTunee Towards Self-Guided Spurious Bias Mitigation
The assumption 3² Weore. of W Wspa,s Vi = 1, 1,...:. M generally holds for biased model, the model has learned to associate spurious attributes with core attributes. The proof is in Appendix A.2.6. Denote the NeuronTune solu- tion by u and pp- Our finding reveals that retraining the last layer does not alter the weight on the spurious com- ponent, i.e., u = uu: which is the optimal solution achievable by last-layer retraining methods (see Lemma 3 in Appendix). However, it does adjust u to be closer to the optimal weight on the core component, B.. Overall, Neu- ron'Tune brings the model parameters closer to the optimal, unbiased solution compared to the parameters of the origi- nal biased model. Therefore, Neuron is guaranteed to outperform the ERM-trained model. Further discussion on the connection last-layer retraining methods is provided in Appendix A.3.
Identification Criterion. As shown in Fig. 1(b), for each embedding dimension i, we separate V into two sets V and V representing values at the i-th embedding dimen- sion from V contributing respectively to correct and in- correct predictions, i.e., = [v]]](v, 1) E V*), and DM = {v]i]](v 0) E V*J, = I,...,,M, M,, y E V, where v denotes the i-th dimension of v. We propose a spuri- ousness score 8 to measure the spuriousness of the i-th dimension when predicting the class y. Following the in- sight from Theorem 4.2, we define oy¹ as follows:
8 = Mmis -Meur:
(5)
where Mmis (ed and Mooo Med
Theorem 4.2 assumes that each dimension of input embed- dings consists of a linear combination of spurious and core components. While it generally holds that each dimension represents a mixture of spurious and core components, in real-world scenarios, the combination is typically nonlin- ear. To account for this, we introduce À a threshold and identify dimensions using the following criterion:
Our findings suggest that our approach makes a slight trade-off in average-case accuracy to achieve im- proved worst-group accuracy. For example, our method may slightly reduce the model's ability to classify waterbird on water due to a relative decrease in reliance on the water feature, while significantly enhancing its ability to classify waterbird on land.
Remark:
S = {ild"' > A, Vi = I,..., M, E y}.
(6)
We set À to 0 by default, as it works well in practice.
4.2. Neuron'Tune: Practical Implementation
In the following, we refer to a dimension as a biased di- mension when 87 >> A and unbiased dimension otherwise. A biased (unbiased) dimension does not imply that the di- mension exclusively represents spurious (core) attributes. In practice, an unbiased dimension exhibits high activation values for target classes, whereas biased dimension shows high activation values for undesired classes. Visualizations of several identified biased and unbiased dimensions on real-world datasets are provided in Appendix A.9.
For real-world spurious bias mitigation, we con- sider a well-trained ERM model fo where O arg ming ((x. (fe (x), y), and l denotes the cross- entropy loss function. The model fe co, ho, consists of a feature extractor Ea, X' - I followed by a linear classifier heg : R M + R! where M is the number of di- mensions of latent embeddings obtained from EØ,, o denotes the function composition operator, and f 0₁ U 02
We include the dimensions identified for all the classes into the set S since an identified biased dimension for one class cannot serve a core contributor predicting some other class in a well-defined classification task. For example, con- sider that the dimension representing "blue color" is biased for the "rectangle" class while being unbiased for the "blue color" class. This happens when we have a blue rectangle as the input, which makes the classification ambiguous.
NeuronTune aligns best with our theoretical analysis when implemented as a last-layer retraining method where the feature extractor co, is fixed and the last layer is linear and tunable. Fig. 1 gives an overview of Neuron'Tune which mainly includes identifying affected neurons (Section 4.2.1) and model tuning with identified neurons (Section 4.2.2).
4.2. 1. IDENTIFYING AFFECTED NEURONS
As shown in Fig. 1(a), use set of identification data Didc. which typically contains a set of diverse features not seen by the model, to identify dimensions (neurons) affected by spurious bias in the model's latent space. We first extract latent embeddings and prediction outcomes for samples of class y in Diae. i.e.,
4.2.2. MODEL TUNING WITH IDENTIFIED NEURONS
As illustrated in Fig. 1(c), we tune the last prediction layer while suppressing the signals from the identified biased dimensions. In this way, we explicitly intervene the internal decision process of the model to discover robust decision rules beyond using spurious correlations.
y = {(v. = co. (x), v(x, v) E Dise).
(4)
Learning Objective. Concretely, given a model tuning dataset Drune- we optimize the following objective,
where o= = 1 { farg max (x) = v). v E RM is an M- dimensional latent embedding of x, and o is the correspond- ing prediction outcome with being an indicator function.
0 = arg min O2 B- (a.N)E
E
E
((he,(V),)) v).
(7)
5

---

Neuron Tune: Towards Self-Guided Spurious Bias Mitigation
where B contains class-balanced sample-label pairs from Dume, addressing that the classifier may favor certain classes during model tuning, and V is the latent embedding after zeroing-out activations on the biased dimensions in S. Un- less otherwise stated, we use Irain as Tunee
(a) Original Model
Model Selection. Without group labels, is challenging select robust models (Liu al., 2021; Yang al., 2023). We address this by designing novel model selection metric, termed spuriousness fitness score (SFit), which is the sum of magnitudes of spuriousness scores across dimensions and classes, i.c., SFit = Em=t Σyey Abs( where Abs(-) returns the absolute value of given input. The score holisti- cally summarizes whether biased and unbiased dimensions in the model are distinguishable. A low SFit indicates that the model tends memorize samples. Empirically, find that high SFit effectively selects robust model.
During Training: Acc: 95.495, WGA: 66.2%
During Testing:
Acc: 70.756, WGA: 67.856
(b) Identify Spurious Dimensions
-05
65
15
20
Dimension
Dimension
Neuron is highly efficient as it only requires tuning the last layer of the model. We use (6) and (7) to iteratively perform the biased dimension detection and model tuning while using SFit for model selection.
8/ Manis
MMea Mcd( Mcd(V,)
(c) After Neuron Tuning
5. Experiments
5.1. Datasets
We tested NeuronTune on four image datasets and two text datasets, each with different types of spurious attributes. (1) Waterbirds (Sagawa et al., 2019) is an image dataset for recognizing waterbird and landbird. cally by combining images of the two bird types from the CUB dataset (Welinder et al., 2010) and the backgrounds, water and land, from the Places dataset (Zhou al., 2017). large-scale image dataset of celebrity faces. The task is to identify hair color, non- blond or blond, with male or female as the spurious attribute. (3) ImageNet-9 (Xiao et al., 2021) is subset of ImageNet (Deng et al., 2009) containing nine super-classes. It com- prises images with different background and foreground signals and be used assess how much models rely on image backgrounds.
During Training: Acc: 90. 795, WGA: 87.6%
During Testing: 90.6%6,
generated syntheti-
Figure 2. Synthetic experiment. (a) Training and test data distri- butions along with decision boundaries (b) Value distributions of the correctly (bluc) and incorrectly (red) predicted samples at the first (left) and second (right) dimensions of input embeddings, with the second dimension identified as biased dimension. (c) NeuronTune improves WGA. Data groups +1,a a 0): orange dots;
trained model.
(2) CelebA (Liu al., 2015)
(y +1, 1): red dots; (y 1, = 0): blue dots; (yr 11,a = 1): green dots.
(4) ImageNet-A (Hendrycks al., 2021) is a dataset of real-world images, adversarially cu- rated to test the limits of classifiers such as ResNet-50. We used this dataset to test the robustness of a classifier after training on ImageNet-9. (5) MultiNLI (Williams al., 2018) text classification dataset with three classes: neu- tral, contradiction, and entailment, representing the natural language inference relationship between premise and hypothesis. The spurious attribute is the presence of nega- tion. (6) CivilComments (Borkan et al.. 2019) is a binary text classification dataset aimed at predicting whether an internet comment contains toxic language. The spurious attribute involves references to eight demographic identities. The dataset uses standard splits provided by the WILDS
benchmark (Koh et al., 2021).
5.2. Experimental Setup
Training Details. We first trained ERM models on each of the datasets. We used ResNet-50 and ResNet-18 (He et al., 2016) models pretrained on ImageNet for experi- ments on the Waterbirds and CelebA datasets, and on the ImageNet-9 and ImageNet-A datasets. respectively. For text datasets, used the BERT model (Kenton & Toutanova, 2019) pretrained on Book Corpus and English Wikipedia data. We followed the settings in Izmailov et al. (2022) for ERM training, with the best models selected based on the average validation accuracy. For our Neuron Tune train- ing, unless otherwise stated, we used the validation data
6

---

NeuronTune: Towards Self-Guided Spurious Bias Mitigation
Table 1. Comparison of worst-group accuracy (%), average accuracy (%), and accuracy gap (%) on the image datasets. denotes using
fraction validation data for model tuning. The best result in each group of methods is in boldface.
Group annotations
Waterbirds
CelebA
Algorithm
Train
Val
WGA (1)
Acc. (1)
Acc. Gap (1)
WGA (t)
Acc. (1)
Acc. Gap (4)
ERM (Vapnik. 1999)
72.6
97.3
24.7
47.2
95.6
48.4
JTT (Liu et al., 2021)
No
Yes
86.7
93.3
6.6
81.1
88.0
6.9
SELFI (LaBonte al., 2024)
No
Yes
93.0
94.0+1.7
1.0
83.9
91.7=004
7.8
CNC (Zhang et al., 2022)
No
Yes
88.5
90.940.1
2.4
88.8
89,9
1.1
BAM (Li al., 2024)
No
Yes
89.2+..33
91.4+0.4
2.2
83.5
88.0
4.5
AFR (Qiu al., 2023)
No
Yes
90.4+1111
94.2+1.2
3.8
82.0+0.5
91.3=0.3
9.3
DFR (Kirichenko et al., 2023)
No
Yes
92.4
94.940.3
2.5
87.0
92.620.5
5.6
BPA (Seo et al., 2022)
No
No
71.4
82.5
GEORGE (Sohoni et al., 2020)
No
No
76.2
95.7
19.5
52.4
94.8
42.4
BAM (Li et al., 2024)
No
No
89. 89.1
91.440.3
2.3
80.1
88.4-2.3
8.3
Neuron
No
No
92.2+..3
94,4100.
2.2
83.1
92.0
8.9
NeuronTunee
No
No
10.9
94.5
87.3
90.3
3.0
as Diee and the training data as Drune- We took the abso- lute values of neuron activations before the identification process, ensuring that high activation magnitudes reflect strong contributions to predictions. We ran the training under five different random seeds and reported average accuracies along with standard deviations. We provide full training details Appendix A.8. Code is available at https::/// pettps://githh /tttttttttttt/tt/// com/ gt wwweaa/eennnnnnuunnnn
regression model (v =
v- +b)}) with w [w, b]. The model predicts +1 when de (v) > 0.5 and -1 otherwise. We trained dw on the generated training data and tested it on the corresponding test data. Details of the data generation are provided in Appendix A.1.
Fig. 2 illustrates spurious bias and how Neuron mit- igates it. First, we observe that the decision boundary of the trained model tends to separate the majority groups of training samples. This leads to high average accuracy but small WGA the training set (Fig. 2(a), left) and poor performance on the test set (Fig. 2(a). right). Then, Fig, 2(b) demonstrates the value distributions of the first (core) and second (spurious) dimensions of the input samples with y = -1. NeuronTune identified the second dimension as a biased dimension, which indeed represents spurious attributes. Next, Fig. 2(c) shows that NeuronTune signif- icantly improves WGA both the training and test sets by suppressing the contributions from biased dimensions. Finally, independent of how Neuron works, there exists tradeoff between average accuracy and WGA due to com- plexity of input samples, as demonstrated in the left parts of Figs. 2(a) and 2(c).
Evaluation Metries. To evaluate the robustness spurious bias, we adopt the widely accepted robustness metric, worst- group accuracy (WGA), that gives the lower-bound perfor- mance of a classifier on the test set with various dataset biases. We also focus on the accuracy gap between the stan- dard average accuracy and the worst-group accuracy as a measure of a classifier's reliance on spurious correlations. A high worst-group accuracy and low accuracy gap indicate that the classifier is robust spurious correlations and can fairly predict samples from different groups.
5.3. Synthetic Experiment
We considered input v == [uS, of, o*] R that has three dimensions: core dimension with the component 2 E R, a spurious dimension with the spurious component v E I. and a noise dimension with the noise component v. We generated training and test sets with sample-label pairs (v, y). where E {-1,+11 +1}. The core component in v noisy version of the label y both sets. The spurious component in the training set is noisy version of the spurious attribute
5.4. Comparison with Existing Approaches
We evaluated NeuronTune on both image and text datasets to showcase its effectiveness and versatility in handling dif- ferent data modalities and model architectures. Our primary comparisons were with methods specifically designed for unsupervised spurious bias mitigation, where no group la- bels are available for bias mitigation. To provide additional context, we also included methods for semi-supervised spu- rious bias mitigation, which leverage group labels in the validation set to select robust models.
0 in 95% (5% for 1) of samples with -1 and in 5% (95% for = 1) of samples with y +1. The noise component is an independent zero- mean Gaussian variable. In the test set, for each label, we reduced the 95% group to 10%, effectively reversing the majority and minority group roles. We adopted a logistic
Results in the lower parts of Tables and 2 were obtained
7

---

NeuronTune: Towards Self-Guided Spurious Bias Mitigation
Table 2. Comparison of worst-group accuracy (%), average accuracy (%) and accuracy gap (%) on the text datasets. fraction validation data for model tuning. The best result in each group methods is in boldface.
denotes using
Group annotations
MultiNLI
Civi Comments
Algorithm
Train
Val
WGA (1)
Acc. (f)
Acc. Gap (4)
WGA (t)
Acc. (1)
Acc. Gap (4)
ERM (Vapnik. 1999)
67.9
82.4
14.5
57.4
92.6
35.2
JT (Liu et al., 2021)
No
Yes
72.6
78.6
6.0
69.3
91.1
21.8
SELFI (LaBonte al., 2024)
No
Yes
70.7
81.2++.7
10.5
79.1
87.7=0.6
8.6
CNC (Zhang et al., 2022)
No
Yes
68.9+2.1
81.7=0.5
12.8
BAM (Li al., 2024)
No
Yes
71.2+1..6
79.6
8.4
79.3
88.3
9.0
AFR (Qiu al., 2023)
No
73.4
81.440.2
68.7.2.6
89.8
21.1
DFR (Kirichenko et al., 2023)
No
Yes
70.8
81.740.2
10.9
81.8
87.5
5.7
BAM (Li et al., 2024)
No
No
70.8
80.3+
9.5
79.3
88.3
9.0
NeuronTune
No
No
72.1
81.1
9.0
82.4
89.2-0011
6.8
Nuur
No
No
22.5
80.3
7.8
82.7
89.420.2
6.7
Table Average accuracy (%) and accuracy gap (%) comparison the ImageNett-9 and ImageNet-A datasets. All methods used ResNet-18 18 as the backbone. The best results are in boldface.
Method
ImageNet-9
ImageNet-A
Acc. Gap (L)
ERM (Vapnik, 1999)
90.8
24.91111
65.9
StylisedIN (Geirhos et al., 2019)
88.4
24.6+114
63.8
RUBi (Cadene et al., 2019)
90.5.0.3
27.7+2.1
62.8
ReBias (Bahng al., 2020)
91.9+1.7
29.6+166
62.3
LfF (Nam et al., 2020)
86.0
24.6
61.4
CaaM (Wang et al., 2021)
95.7
32.8
62.9
SSL+ERM (Kim al., 2022)
94.2+0.1
34.2+0.5
60.0
LWBC (Kim et al., 2022)
94.0+0.2
36.0+0.5
58.0
NeuronTune
93.7+0.1
37.3
56.4
in the unsupervised spurious bias mitigation setting. In this setting, our method achieves the highest worst-group accu- racies and smallest accuracy gaps across the datasets, high- lighting its effectiveness in enhancing models' robustness spurious bias and balancing performance across different data groups. Results in the upper parts of Tables 1 and 2 were from methods in the semi-supervised spurious bias mit- igation setting. Methods in this setting benefit from group labels for selecting robust models. Despite this advantage, NeuronTune demonstrates strong self-debiasing capabilities, competing favorably with methods such as AFR and DFR that rely on group labels. When a half of the validation set was used in training, Neuron'Ture achieved better WGAs and accuracy gaps on three out of four datasets than DFR and SELF that exploited the same set of data for training.
achieved when AFR models were selected without group labels (Appendix A.4).
We further used the ImageNet-9 (Kim et al., 2022; Bahng et al., 2020) and ImageNet-A (Hendrycks et al., 2021) datasets evaluate NeuronTune's robustness distribu- tion shifts, which are challenging to depict in group la- bels. We first trained an ERM model from scratch using ImageNet-9 and then fine-tuned its last layer with Neuron- Tune. In Table 3, Neuron'Tune achieves the best accuracy on the challenging ImageNet-A dataset, which known for its natural adversarial examples. While this improvement comes with slight trade-off in in-distribution accuracy ImageNet-9, Neuron maintains the smallest accuracy gap between the two datasets, making it robust method for out-of-distribun generalization.
Notably, compared with sample-level last-layer retraining methods, such as AFR, NeuronTune manipulates the neu- rons within a model, providing more targeted control how spurious bias is mitigated. Hence. NeuronTune theory achieve better robustness spurious bias (Appendix A.3). In general, NeuronTune compares favorably with AFR in terms of WGA and accuracy gap, with larger gains
Finally, in Tables 1, 2, and 3, we observe a common trade- off between average accuracy and WGA that exists across many spurious bias mitigation methods. For NeuronTune. this trade-off primarily occurs when samples sharing the same spurious attribute but belonging to different classes are difficult to separate in the latent space, as illustrated in
8

---

NeuronTune: Towards Self-Guided Spurious Bias Mitigation
Table Comparison of worst-group accuracy (%) between different choices of Duue and Dime well neuron neuron-based based tuning (NT) on the
four datasets. The best results are in boldface.
Dide
D Dune
NT
Waterbirds
CelebA
MultiNLI CivilComments
D Draan
Dwain
Yes
78.0+2.3
58,5+1.2
42.0+10.5
80.0+10.5
Veal
D minn
Yes
92.240.3
83.1
42.1
82.4
Dval
D Dwain
No
82.7
53,9+0.0
63.4+0.7
81.5
Dval2
/
Yes
92.5100.
87.3
72.5
82.7 1:0.4
Table Analysis of the impact of partial suppression (masking value > 0) and full suppression (masking value 0) the performance of NeuronTune', evaluated on the CelebA dataset.
Masking value
0
0.2
0.4
0.6
0.8
1.0
WGA (T)
87.3
71.5
72.2+1.2
72.9+1.5
11.5
73.0+1.2
Acc. (t)
90.3
93.810.2
93.8
93.810.2
93.810.2
93.910.2
Fig. 2. While improving sample embeddings could help alle- viate this issue, it often demands substantial computational resources. In contrast, NeuronTune as post hoc method, efficiently mitigates spurious bias by tuning only the last layer with low computational complexity (Appendix A.5) while still achieving favorable balance between WGA and overall performance.
to further performance improvements on datasets such as CelebA and MultiNLI, demonstrating the advantage of us- ing separate and independent datasets for bias identification and model tuning. Identifying the optimal choice for Diae and Drume remains avenue for future research.
Finally, we analyze different strategies for handling the iden- tified biased dimensions, as shown in Table 5. Our default approach, described in Section 4.2.2, fully suppresses the activations on the biased dimensions by multiplying the ac- tivations with masking value of zero. To explore the effect of partial suppression, we varied the masking value from 0.2 to 1.0, where 1.0 corresponds to suppression. As shown in Table 5, on the CelebA dataset, only the full suppression strategy (masking value 0) led to an improvement in WGA.
5.5. Ablation Studies
In Table 4, we compare NeuronTune' 's performance between different choices of the identification dataset Dide and the model tuning dataset Drune- Additionally, we demonstrate the effectiveness of neuron-based tuning on the identified biased dimensions (denoted as NT).
This highlights that while partial suppression may reduce the loss average accuracy, its impact on spurious bias is similar to no suppression at all. With nonzero masking values, models can still adjust their weights using biased activations, resulting in persistent spurious bias.
When using Pide Drsinn we observe relatively low per- formance across datasets. After switching to held-out validation data Dvat. observe significant performance improvements. This highlights the advantage of using new and independent dataset to identify biased dimensions, as models may have already memorized patterns in Durinn By default, NeuronTune adopts Dvat as Diae- is important to note that using Pval identify biased dimensions is anal- ogous using it for model selection. Hence, D is not directly used for updating model weights.
6. Conclusion
We proposed a self-guided spurious bias mitigation method that directly intervenes the prediction mechanisms within model without using group labels. Our method exploits distinct patterns of neuron activations in a model's latent space to identify biased dimensions and suppresses signals from these dimensions while tuning the remaining model. We theoretically validated our neuron identification method and proved that our method can bring a model closer to an unbiased one than its ERM counterpart. Experiments vali- dated our theoretical findings and showed that our method lightweight post hoc bias mitigation method that can work across different data modalities and model architectures. Future work may explore different choices of identification and model tuning data to enhance spurious bias mitigation.
Next, we disabled NT during model tuning (NT=No), which effectively reduces NeuronTund to class-balanced model tuning. We observe consistent performance degradation across the four datasets, which validates the effectiveness of NT across datasets.
Moreover, inspired by the success of DFR (Kirichenko et al., 2023), which uses half of the validation data for model uniing we divided Deai into two equal halves: one half (denoted as Dra/2) was used as Diae. while the other half served as Drune- Unlike DFR, our method does not rely on group labels in the validation data. This strategy leads

---

NeuronTune: Towards Self-Guided Spurious Bias Mitigation
Acknowledgments
neural networks. Transactions on Machine Learning Research, 2023.
This work supported in part by the US National Sci- ence Foundation under grants 2217071, 2213700, 2106913, 2008208. Any opinions, findings, and conclusions or rec- ommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation.
Byrd, J. and Lipton, Z. What is the effect of importance weighting in deep learning? In International Conference on Machine Learning, pp. 872-881. PMLR, 2019.
Cadene, R., Dancette, C., Cord, M., Parikh, D., al. Rubi: Reducing unimodal biases for visual question answering. In Advances in Neural Information Processing Systems, volume 32, 2019.
Impact Statement
This paper presents work whose goal is advance the field of Machine Learning by improving model robustness through self-guided spurious bias mitigation. Our method Neuron reduces reliance on spurious correlations with- out requiring additional annotations, thereby enhancing gen- eralization across diverse data distributions. This has im- portant implications for trustworthiness and reliability in AI systems, particularly in high-stakes applications where bi- ased model predictions can lead to unintended consequences. While our work contributes positively by mitigating spuri- ous correlations, we acknowledge that no method is entirely free from potential biases introduced during training or de- ployment. We encourage further research on assessing and addressing residual biases in real-world settings. However, we do not identify any specific ethical concerns that require special attention beyond standard considerations in trustwor- thy machine learning.
Creager, E., Jacobsen, J.-H., and Zemel, R. Environment inference for invariant learning. In International Con- ference on Machine Learning, pp. 2089218922000 2200. PMLR, 2021.
Cui, Y., Jia, M., Lin, T.-Y., Song, Y., and Belongie, S. Class- balanced loss based on effective number of samples. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 9268 9277,,92777777 2019.
Deng, J., Dong, W., Socher, R., L.-J,, Li, K., and Fei-Fei, I.. Imagenet: A large-scale hierarchical image database. In Proceedings of the IEEE/CVF Conference on Com- puter Vision and Puttern Recognition, pp. 248-255, 2009. doi: 10.1109/CVPR.2209.5200
Deng, Y., Yang, Y., Mirzasoleiman, B., and Gu, Q. Robust learning with progressive data expansion against spurious correlation. In Advances in Neural Information Process- ing Systems, volume 36, 2024.
References
Geirhos, R., Rubisch, P., Michaelis, C.,, Bethge, M., Wich- mann, F. A., and Brendel, W. Imagenet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustness. In International Conference on Learning Representations, 2019.
Arjovsky, M., Bottou, L., Guirajani, I., and Lopez- Paz, D. Invariant risk minimization. arXiv preprint 1oXXvv1907.02893 2019.
Bahng, H., Chun, S., Yun, S., Choo, J., and Oh, J. Learn- ing de- di-bid representations with biased representations. In International Conference on Machine Learning, pp. 528-539, PMLR, 2020.
Geirhos, R., Jacobsen, J.-H., Michaelis, Zemel, R., Bren- del, W., Bethge, M., and Wichmann, F. A. Shortcut learn- ing in deep neural networks. Nature Machine Intelligence, 2(1 1):66) 673, 2020.
Barbano, C. A., Dufumier, B., Tartaglione, E., Grangetto, M., and Gori, P. Unbiased supervised contrastive learning. In International Conference on Learning Representations, 2023.
Han, Z., Liang, Z., Yang, F., Liu, L., Li, L., Bian, Y., Zhao, P., Wu, B., Zhang, C., and Yao, J. Umix: Im- proving importance weighting for subpopulation shift via uncertainty-awe aware mixup. In Advances in Neural Informa- tion Processing Systems, volume 35, pp. 2022.
Beery, S., Van Horn, G., and Perona, P. Recognition in terra incognita. In European Conference Computer Vision. pp. 456-473, 2018.
Borkan, D., Dixon, L., Sorensen, J., Thain, N., and Vasser- man, L. Nuanced metrics for measuring unintended bias
He, H. and Garcia, F. A. Learning from imbalanced data. IEEE Transactions on knowledge and data engineering, 21(9):16331228, 2009.
with real data for text classification. In Companion pro- ceedings of the 2019 world wide web conference, pp. 491-500, 2019.
He, K., Zhang, X., Ren, S., and Sun, J. Deep residual learn- ing for image recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770-778, 2016
Bykov, K., Deb, M., Grinwald, D.. Muller, K. R., and Höhne, M. M. Dora: Exploring outlier representations in deep
10

---

NeuronTune: Towards Self-Guided Spurious Bias Mitigation
He, Q. Xu, K., and Yao, A. Eva: Erasing spurious corre- lations with activations. In The Thirteenth International Conference Learning Representations, 2025.
Nam, J., Cha, H., Ahn, S., Lee, J., and Shin, J. Learning from failure: De-biasing classifier from biased classifier. In Advances in Neural Information Processing Systems, volume 33, pp. 20673-20684, 2020.
Heinze, G., Wallisch, C., and Dunkler, D. Variable selection- review and recommendations for the practicing statisti- cian. Biometrical journal, 60(3):431-449) 2018.
Nam, J., Kim, J., Lee, J., and Shin, Spread spurious attribute: Improving worst-group accuracy with spuri- ous attribute estimation. In International Conference on Learning Representations, 2022.
Hendryeks, D., Zhao, K., Basart, S., Steinhardt, J., and Song, Natural adversarial examples. In Proceedings of the IEEE/CVF Conference on Computer Vision and Paltern Recognition, pp. 15262-15271, 2021.
Qiu, S., Potapczynski, A., Izmailov, P., and Wilson, A. G. Simple and fast group robustness by automatic feature reweighting, In International Conference Machine Learning, pp.
Izmailov, P., Kirichenko, P., Gruver, N., and Wilson, A. G. On feature learning in the presence of spurious corre- lations. In Advances in Neural Information Processing Systems, volume 35, pp. 38516-38532, 2022.
PMLR, 2023.
Sagawa, S., Koh, P. W., Hashimoto, T. B., and Liang, P. Distributionally robust neural networks. In International Conference on Learning Representations, 2019.
Kenton, J. D. M.-W. C. and Toutanova, L. K. Bert: Pre- training of deep bidirectional transformers for language understanding. Proceedings of NAACI-HILT, pp. 4171- 4186, 2019.
Sagawa, S., Raghunatham, A., Koh, P. W., and Liang, P. An investigation of why overparamettrization exacerbates spurious correlations. In International Conference on Machine Learning, pp. 8346 1446-8356. PMLR, 2020.
Kim, N., Hwang, S., Ahn, S., Park, J., and Kwak, S. Learn ing debiased classifier with biased committee. In Ad- vances in Neurul Information Processing Systems, vol- ume 35, pp. 18403-18415, 2022.
Seo, S., Lee, J.-Y., and Han, B. Unsupervised learning of debiased representations with pseudo-attributes. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 16742-16751, 2022.
Kirichenko, P., Izmailov, P., and Wilson, A. G. Last layer re-training is sufficient for robustness to spurious corre- lations. International Conference on Learning Repre- sentations, 2023.
Singla, S. and Feizi, S. Salient imagenet: How to discover spurious features in deep learning? International Conference on Learning Representations, 2021.
Koh, P. W., Sagawa, S., Marklund, H., Xie, S. M., Zhang, M., Balsubramani, A., Hu, W., Yasunaga, M., Phillips, R. L., Gao, I., et al. Wilds: A benchmark of in-the- wild distribution shifts. International Conference on Machine Learning, pp. 5637-5664. PMLR, 2021.
Sohoni, N., Dunnmon, J., Angus, G., Gu, A., and Ré, C. No subclass left behind: Fine-grained robustness in coarse- grained classification problems. In Advances in Neural Information Processing Systems, volume 33, pp. 19339 19352, 2020.
LaBonte, T., Muthukumar, V., and Kumar, A. Towards last-layer retraining for group robustness with fewer an- notations. Advances Neural Information Processing Systems, volume 36, 2024.
Tartaglione, E.,, Barbano, C. A., and Grangetto, M. End: Entangling and disentangling deep representations for bias correction. In Proceedings of the IEEE/CVF confer- ence on Computer Vision and Pattern Recognition, pp. 13508- 2021.
Li, G., Liu, J., and Hu, W. Bias amplification enhances minority group performance. Transactions on Machine Learning Research, 2024.
Tiwari, R. and Shenoy, Overcoming simplicity bias in deep networks using feature sieve. In International Con- ference on Machine Learning, pp. 34330-3444 -3343.. PMLR, 2023.
Liu, E. 7.,, Haghgoo, B., Chen, A. S., Raghunathan, A., Koh, P. W., Sagawa, S., Liang, P., and Finn, C. Just train twice: Improving group robustness without training group information. International Conference on Machine Learning, pp. 6781-6792. PMLR, 2021.
Vapnik, N. An overview of statistical learning theory. IEFE Transactions on Neural Networks, 10(5):988-99) 1999.
Liu, Z., Luo, P., Wang, X., and Tang, X. Deep learning face attributes in the wild. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 3730- 3738, 2015.
Wang, T., Zhou, C., Sun, Q. and Zhang, H. Causal attention for unbiased visual recognition. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 3091-3100, 2021.

---

Neuron Tune: Towards Self-Guided Spurious Bias Mitigation
Wang, Y., Sun, J., Wang. C., Zhang, M., and Yang, M. Nav- igate beyond shorteuts: Debiased learning through the lens of neural collapse. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 12322-12331, 2024.
Zhang, Y., Wang, Z., Hu, R., Duan, X., ZHENG, Y., Huai, B., Han, J., and Sang. J. Poisoning for debiasing: Fair recognition via eliminating bias uncovered in data poi- soning, In ACM Multimedia, 2024.
Zhang, Y.K.,, Wang, Q--.,, Zhan, D.-C., and Ye, H.- J. Learning debiased representations via conditional at- tribute interpolation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 7599-7608, 2023.
Welinder, P., Branson, S., Mita, T., Wah, C., Schroff, F., Belongie, S., and Perona, P. Caltech-UCSD Birds 200. Technical Report CNS-TR-2010-001, California Institute of Technology, 2010.
Williams, A., Nangia, N., and Bowman, S. A broad- coverage challenge corpus for sentence understanding through inference. In Proceedings of the 2018 Confer- ence of the North American Chapter of the Association for Computational Linguistics: Human Language Tech- nologies, Volume 1 (Long Papers), pp. 1112-1122, Asso- ciation for Computational Linguistics, 2018.
Zheng, G., Ye, W., and Zhang, A. Benchmarking spurious bias in few-shot image classifiers. In European Confer- ence on Computer Vision, pp. 346-364, Springer, 2024a.
Zheng, G., Ye, W., and Zhang, A. Learning robust classi- fiers with self-guided spurious correlation mitigation. In Proceedings of the Thirty-Third International Joint Con- ference on Artificial Intelligence, pp. 5599-5607, 2024b.
Wu, S., Yuksekgonul, M., Zhang, L.,, and Zou, J. Discover and cure: Concept-aware mitigation of spurious correla- tion. In International Conference on Machine Learning, pp. 37765-37777 PMLR, 2023.
Zheng, G., Ye, W., and Zhang, A. Spuriousness-awaat meta- learning for learning robust classifiers. In Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pp. 4524 2024c.
Xiao, K. Y., Engstrom, L., Ilyas, A., and Madry. A. Noise or signal: The role of image backgrounds in object recog- nition. In International Conference on Learning Repre- sentations, 2021.
Zheng, G., Ye, W., and Zhang, A. Shortcutprobe: Prob- ing prediction shortcuts for learning robust models. In Proceedings of the Thirty-Fourth International Joint Con- ference on Artificial Intelligence, 2025.
Yang. Y., Zhang, H., Katabi, D., and Ghassemi, M. Change is hard: a closer look at subpopulation shift. In Inter- national Conference on Machine Learning, pp. 39584- 39622, 2023.
Zhou, B., Lapedriza, A., Khosla, A., Oliva, A., and Tor- ralba, A. Places: A 10 million image database for scene recognition. IEEE Transactions Pattern Analysis and Machine Intelligence, 40(6); 1454-14444,, 2017. 2017.
Yao, H. Wang. Y., Li, S., Zhang, L., Liang, W., Zou, J., and Finn, C. Improving outto--distribution robustness via selective augmentation. In International Conference on Machine Learning, pp. 25407-25437. PMLR, 2022.
Ye, H., Zou, J., and Zhang, L. Freeze then train: Towards provable representation learning under spurious corre- lations and feature noise. In International Conference Artificial Intelligence and Statistics, pp. 8968-8990. PMLR. 2023.
Ye, W., Zheng, G., Cao, X., Ma, Y., and Zhang, A. Spuri- ous correlations in machine learning: A survey. ICML Workshop on Data-Centric Machine Learning Research, 2024.
Zhang, H., Cisse, M., Dauphin, Y. N., and Lopez-Paz, D. mixup: Beyond empirical risk minimization. In Interna- tional Conference on Learning Representationss 2018.
Zhang, M., Sohoni, N. S., Zhang, H. R., Finn, C., and Re, C. Correct-N-Contaac A contrastive approach for improv- ing robustness to spurious correlations. In Proceedings of the 39th International Conference on Machine Learning, volume 162. pp. 26484 26516. 16. PMLR, 2022.
12

---

NeuronTune: Towards Self-Guided Spurious Bias Mitigation
A. Appendix
The appendix is organized as follows:
Section A.I: Details of the Synthetic Experiment
Section A.2: Theoretical Analysis
Section A.2.1: Preliminary
Section A.2.2: Proof of Lemma
Section A.2.3: Proof of Corollary
Section A.2.4: Proof of Proposition 4.1
Section A.2.5: Proof of Theorem 4.2
Section A.2.6: Proof of Theorem 4.3
Section A.2.7: Proof of Lemma
Section A.2.8: Proof of Lemma
Section A.3:
Connection to Last-Layer Retraining Methods
Section A.4:
Comparison between Models Selected with Worst-Class Accuracy
Section A.5:
Complexity Analysis
Section A.6:
Advantages over Variable Selection Methods
Section A.7:
Dataset Details
Section A.8: Training Details
Section A.9:
Visualizations of Biased and Unbiased Dimensions
A.1. Details the Synthetie Experiment
Data Model. Without loss of generality, we considered an input v R to simulate latent embedding before the last prediction layer, which consists of three dimensions: core dimension with the core component 2 E R. spurious dimension with the spurious component vf R, and noise dimension with the noise component vf. We considered dataset Dyy {[v.,, 3 of N sample-label pairs, where y4 {-1,+11, of yı + ne. and vf and ne are zero mean Gaussian noises with variances o? and o² respectively. When y; = -1,, 0++ with the probability a and eg = 1 n, with the probability 1 a; when = -1, 2² = 1 n, with the probability and e² = 0 n, with the probability 1 - where n, is an independent zero-mean Gaussian noise with the variance o². To facilitate developing the spurious bias of using the correlation between ue and y/ for predictions, we generated a training set D with easy-to-learn spurious attributes by setting o² of and o. labels. To demonstrate, with the same of parameters except a 0.1. Now, spurious correlations between uf and y, only predictive of small portion of the test samples. Fig. 2 shows four data groups along with their respective proportions in each class.
(Sagawa et al., 2020). Thus, the correlations between of and ys are predictive of set a 0.6, o² 0.1, o² 0.1, 0.95, and N 5000. We generated test set Dzyn
Classification Model. We considered logistic regression model dw (v) = 1/(11 exp(-(w b)}), where w [w, 6). The model predicts +1 when dw (v) > 0.5 and -1 otherwise. We trained Φv on syn and tested on syo
Spurious Bias. observed a high average accuracy of 95.4% but a WGA 66.2% (Fig. 2(a) in main paper) on the training data. The results show that the model heavily relies on the correlations that exist in the majority of samples and exhibits strong spurious bias. As expected, the performance the test data is significantly lower (Fig. 2(a), right). The decision boundary (Fig. 2(a). black lines) learned from the training data does not generalize to the test data.
Mitigation Strategy. Without group labels, it is challenging to identify and mitigate spurious bias in the model. We tackled this challenge by first finding that the distributions of values of an input dimension, together with the prediction outcomes for a certain class, provide discriminative information regarding the spuriousness the dimension. (1) When the values for misclassified samples at the dimension are high, while values for the correctly predicted samples are low, this indicates that
13

---

NeuronTune: Towards Self-Guided Spurious Bias Mitigation
the absence of the dimension input does not significantly affect the correctness of predictions, while the presence of the dimension input does not generalize certain groups of data. Therefore, the dimension tends to be biased dimension. The plots in Fig. 2(b) illustrate the value distributions of the first and second dimensions of input embeddings when y/4 = -1. (2) In contrast, if the absence of the dimension input results in misclassification, then the dimension tends to represent core attribute. The left plot of Fig. 2(b) represents the first dimension of input embeddings when y = -1. Next, we retrained the model while suppressing the second and third dimensions. As a result, the retrained model has learned to balance its performance on both the training and test data with a significant increase in WGA on the test data (Fig. 2(c)).
A.2. Theoretical Analysis
A.2.1. PRELIMINARY
For the ease of readability, we restate the data model specified by (1) and (2) in the following
X Xoore B Xpa
E RR Dx1
X + Ccor::
(8)
and
Xpp (2a 1) Esuu Berm(p),
(9)
where (2a- - 1) E (11,+1}, a Bern(p) is a Bernoulli random variable, p close to 1, cooe a zero-mean Gaussian random variable with the variance Veure. and each element spu follows a zero-mean Gaussian distribution with the variance Vmm--
We set neore >> 1u to facilitate the learning of spurious attributes. The model f(x) b b"W in Section 4.1 can be further expressed as follows,
1 Σ bi (oor Weure, + spu Wspu,s) x
x Meure + x spa Uspu
(10)
where w E IRIxD is the i-th row of W, w = [w su w spu.i. with W E Dix1 and Wspu. E P2x1, Mgore
EM Wewe and Ussp = Em by Spp, The loss function which we use to optimize W and b is
(w(W b) =
2 (...))DD...ll v11..
(11)
With the above definitions, the following lemma gives the optimal coefficients n and u based on the training data.
A.2.2. PROOF OF LEMMA
Lemma 1. Given a training dataset Drain with p defined in (9) satisfying 1 12p >> 0.5, the optimized weights in the form of Woore and Upu are
2 Mo + Mip B
u
Uoore
(12)
Hoore
+ 1ppm
and
(2p 1) 7
Dppu
(13)
Merre Tipu
respectively. When p = 0.5, the training data unbiased and we obtain an unbiased classifier with weights u B and
spu 0.
Proof. Note that f(x) = bT Wxx
xJ = T Ugooe + x xp Uspu. then we have
b)
Bl| x T'
core Heore
+ Spp Uspu - -112
(14)
2
BIll X Meore
+ [(2a - 1) + Espu] T
Usuu v112
(15)
2
1
Eli T Meare [1- 11-(20) (2aa -1)?"" "uspp 1yl12 +
2 Uapo
(16)
2
1
1 1pp Uuspa 12.
(pEi + (1-p) B2)
(17)
14

---

Neuron Tune: Towards Self-Guided Spurious Bias Mitigation
where E1 | Meone (1 7² Uppu)yll when and Eg ||x Meore (1 o Uspu) when 0. We first calculate the lower bound for Fi as follows
E = E| X zone Meore (1 7 Uspu (B" Xcore + Ecore)ll2
(18)
B X Moore (1 - 7" Uspu 3 Xcore + (1 7 7" Usp @core)
(19)
E/ Core Meore
(1 - 3" Uspa)) 3r Xcore 122 + Moore (1 7² Uspu )
(20)
2 ncnre (1 7 Uspu)²
(21)
Similarly, have
E2 = E| x
core Mcore (1 + 7" Uppu "Bp (3
Xoore + Ecore)
(22)
E| X Moore (1 + 7 Uspu)) 3 Xoore 12 + Mecro (1 7 Uguu)
T
(23)
2 Neore + 7² Uppu)*
(24)
Then, plug in (21) and (24) into (17), we obtain the following
eo (W,
precore (1 - --" "qgu)2
uguu + (1 - p) (1 + y" "spp)) + npu Uspa
(1-p)
(25)
(moooc - >-" ummn)) )² + (1 - p) Yooro + 7 umm)) + Mau || 11
(26)
2 (miicce (1 - -"" Uqu) + p)r (1 + 7" upm))² npu y U
1
(27)
where (26) uses the fact that y has a unit norm, and the inequality (27) exploits the 7 Uspu we have {(z) preure (1 - 2)² + (1 - p)rie (1 + +2) nau 2² Let
inequality. Let
0({s)
= 0, we obtain
0z
(2p - 1)
z² x uu
Newe + 1/pu
Given upp we can obtain the optimal u'ure for minimizing E3 in (20) as u'une = (1 --1--)) 33) similarly, we can obtain the optimal Mooee for minimizing E2 in (23) as Usore (1 =" ) Via proof by contradiction, only u'are U is the solution for Usere. Since p >> 0.5, E1 contributes to the majority error of (20). Thus, u (1 2n G, i.e.
(2 2p) mene Vapn G n + R7.
Uoore (1- - =²
A.2.3. PROOF OF COROLLARY
Lemma 1 gives the optimal model weights under a given training dataset Drain with the parameter controlling the strength of spurious correlations. Lemma 1 generalizes the result in (Ye et al., 2023) where = 1. Importantly, we obtain the following corollary for unbiased models:
Corollary 1. The unbiased model f(x) = uuxx X Ucooe + x7 spu Uspu is achieved when Ugore U and y"
TTuupu Uspu 0.
Proof. Plug 7" Ucoree = into (20) and (23), then we observe that Usore minimizes errors from both the majority (a = 1) and minority (a = 0) groups of data.
If we could obtain a set unbiased training data with = 0.5, then we obtain an unbiased model with _ = 0 and U = B. However, in practice, it is challenging to obtain a set of unbiased training data, i.c., it is challenging to control the value of p.
15

---

NeuronTune: Towards Self-Guided Spurious Bias Mitigation
A.2.4. PROOF OF PROPOSITION 4.1
Proposition 4.1 (Principle of NeuronTune), Given the model f(x) = b" Wx trained with data generated under the data model specified in (8) and (9), captures spurious correlations when "/ Wap,s 0, {M)...,Af.. The principle of NeuronTune is to suppress neurons containing negative 7 Sspu.k
Proof. Consider the i-th neuron e; (i I,...,,. .) before the last layer. We first expand it based on our data model specified by (8) and (9) follows:
x T core
W 'f + Xsp W "pu,
(28)
W i + [(2a 1)yy + Esqul 1 Wspu,i
(29)
x core W Weore, + (2a - 1)13" Xcore + Ecore] 7 W spuu, +
o. spo spu..
(30)
cxe i (2a- 1).3² Xcore? W squ,, + Erem;
(31)
where Srem E W Wspa, + 2 Wspa. In (31), if 72 Wspu,i O, the model handles the spurious component correctly, Specifically, when 1, the spurious component positively correlates with the component and contributes to the output, whereas when contribution to the output. In contrast, if 7² Wapu.s < 0 and a = 1, then the model still utilizes the spurious component even the correlation breaks, demonstrating a strong reliance on the spurious component instead of the core component. Therefore, the principle of selective activation is to find neurons containing negative 7 W W spu, so that suppressing them improves model generalization.
O, its correlation with the core component breaks with a negative one and has negative
A.2.5. PROOF OF THEOREM 4.2
The following theorem validates our neuron selection method.
Theorem 4.2 (Metric for Neuron Selection). Given the model f(x) = b"Wx, we cast it classification model by training it to regress y E {-H, /hr (μ >> 0) based the data model specified in (8) and (9), where μ E|3"xxcme! X The metric 8 defined in the following can identify neurons with spurious correlations when 8 0:
8 MMd((e)))))) (Med(V))
where D and Vy are the sets of activation values for misclassified and correctly predicted samples with the label y from the i-th neuron, respectively; an activation value is defined x Weore, + X W spu,ii and Med(-) returns the median of an input set of values.
Proof. We start by obtaining the set of correctly predicted samples D,, and the set of incorrectly predicted samples Dy as D,, = (x) 2 0, (x, y) E Dide) } and D., = [xlf(x) < 0, (x, y) E Diae), where Día the set of identification data. Then, we have D (ea /x E Dy), and Di = {ex|x E Dyy, where is the i-th neuron activation defined in (31). Expanding following (31), we obtain
6, = x Weore,s + (2a- - 1).8 7 Wspu,s + Egem-
Note that xT W i and Erem exist for all the samples, regardless of the ultimate prediction results, and e; follows a Gaussian distribution given a. Then, among all the correctly predicted samples with the label y, according the Lemma 2, have Med( E E[xT Weore, + jey² Wspu,s Similarly, among all the incorrectly predicted samples with the label y, have (1.) E X Weore.i] - wn/ Wapa.i. Then, the difference between the two is
8 ~ - 2 T Wspa.
When 6 > 0, we have 7 ssp.,s < O. According Proposition 4.1, using 8. > indeed selects neurons that have strong reliance on spurious components.
16

---

MeuronTune: Towards Self-Guided Spurious Bias Mitigation
A.2.6. PROOF OF THEOREM 4.3
Theorem 4.3 Tueeron'Tune Mitigates Spurious Bias). Consider the model (x) x2 trained on the biased training data with p > 0.5, with Yoor and upn defined in (12) and (13), respectively. Under the mild assumption that 3 W Woore, 22 7 W Ws Vi ,,...,M, M, then applying NeuronTune to f*(x) (x) produces a model that is closer to the unbiased one.
Proof. Consider (x) the base model. We aim to prove that the retrained model obtained with NeuronTune is closer to the unbiased model defined in Corollary than the base model in the parameter space.
First, the assumption that 3d We 2 7 Wspu,s: Vi = ,....,M .M generally holds for a biased model as the model has learned to associate spurious attributes with the core attributes.
Then, we denote the retrained parameters obtained with NeuronTune uLvre and Uppu. We start with calculating u Focusing on (27) and following the derivation in Lemma 1, we obtain ulpu = =iizz, b,Wapu,s = upp where I+ denotes the of neuron indexes satisfying >" W W O. Note that NeuronTune is a last-layer retraining method; thus we only optimize b, here and w a is the same as in (x). Left multiplying uspu with y², we have
Y² u pu
=
Σ IEI
Wspu,
(32)
(2p 1)More
>
neure + 7spu
Note that ""' spp., > O, Vi E I+ because of NeuronTune. Hence, we have by > E I4. Moreover, we observe that uppa is the same as u long I. is non-empty. This shows that NeuronTune is not able to optimize parameters related to the spurious components in the input data.
According to the Corollary 1, the unbiased model is achieved when p = 0.5 and Ugore = 3. The Fuclidean distance between and the biased solution Ucore = (1 - 3 is |||| || - 3 = z². Based on (32), we estimate the distance between our
NeuronTure solution ufore and 3 as follows
| B112 113 (u B3]12
(33)
-11,3 u. 1
(34)
|| Σ
Woore, 1 ||2
(35)
W Wspu, 1112
(36)
||=
1/|
(37)
where (34) uses the fact that TT and (35) uses the condition 3d Weore, RJ 7 W.pu.i, Vi 1, 1,..., M. Note that 2" is achieved on the training data with >>> 0.5 and Marc > Vapur hence we have 2 1 and ||u Bll2 xe 0. In other words, NeuronTune can bring model parameters closer to the optimal and unbiased solution than the parameters of the biased model.
A.2.7. PROOF OF LEMMA
Lemma (Majority Samples among Different Predictions). Given the model f(x) bT trained on y
{- (n > 0) with μ = E Xcore), and the conditions that p > 3/4 and m' 1z we have the following claims:
Among the set of all correctly predicted samples with the label y, more than half of them are generated with a = 1;
Among the set of all incorrectly predicted samples with the label y, more than half of them are generated with
17

---

NeuronTune: Towards Self-Guided Spurious Bias Mitigation
Proof. With the two regression targets, -n and μs, the optimal decision boundary is O. Without loss of generality, consider y = js. Then, the set of correctly predicted samples Dy is
by = fxlf(x) 2 0, (x.y) E Diee:
and the set of incorrectly predicted samples Dy is
Dy [x]s(x) (x. E Dide).
The probability of sample with the label y that is correctly predicted is
P(x Dylv) P(a 1)P((())) 0 + P(a P(a= 0la 0,
pP((f(x) 2 0 +1 2 0/a 0,
Similarly, the probability of sample with the label y that is incorrectly predicted is
P(x Dylu) = pP((()) < 0/a 1, + p))((()())))))))))))))))) (1) Ola 0,y)
To calculate P(f(x)= 2 0la 1, we expand fxx follows:
T f(x) x Mcore + x uip
= T Blo - -2") + (B" Xsore + Ecore) + E Uuuu u
B1 - =*) xoor By upp + 7" u coree + e Upu
= 3 + Ecore + E spu Uspa-
The output of [(x) follows a Gaussian distribution, with the mean pi [[J(()] p. and the variance of Var(xx2 3) + Neure (z*)² + (2*)² Therefore, we have
P(f(x) 2 0/a 1, P(x E Dyla 1,y) Φ(
0 - 11
/
(38)
01
01
P(f(x) < 0/a 1, P(x Dyla 1,,) Φ6
11
$
p
(39)
01
01
Similarly, to calculate P(f(x) 2 0la = 0, y). we expand f(x) as follows:
f(x) = T 3(1 - x²) Byy upp y uu core + T
Epu Usu
200 22²) =² Ecore
spu upp
The output of f(x) follows Gaussian distribution, with the mean μo E[f(x)] 1(1 -2²), and the variance of = (1 22 V + n (2*)² nam (2*)2. As result, have
$b (1 22- )11
(x 2 0/a = 0,9) P(x E D2yaa la 0,y) +
(40)
00
Oo
P(f(x) 0/a = 0, P(x E Dyla 0,y) = =1 Φ
Hg
Φ(
-(1
22 ){
(41)
Then, we obtain the probabilities for correctly and incorrectly predicted samples with the label y, i.e.,
P(x E (D,e), - p 1 01 (1 p) (1- (1-2z")
(42)
dp
and
(1 2z )p
P(x E Dylv) = p σ1
+ (1 @
(43)
00
18

---

NeuronTunee Tune: Towards Self-Guided Spurious Bias Mitigation
Next, we seek determine whether the majority of samples in the correctly (incorrectly) predicted set D. (D₂) is generated with 1. To achieve this, in the set of correctly predicted samples, use the Bayesian theorem based on (42), L.e.
P(a 1/x E Dy P(x E Dyla = 1,
P(x E D. v)
pT(H1/11)
(44)
(D(1//1)) + (1 p) - 2z 22/o0)
and
P(a 0|x Dy,u)
P(a 1/x Dy:
(1- p) - 22 3200)/000))
(45)
pD(1//11) (1 ())((11 -22- u1/0)//0))
Similarly, in the set of incorrectly predicted samples, we have
P(a = 1|x E Dy,,) P(x Dyla 1,y)
P(x Dyly)
pD(-11/01)
(46)
p (3))0(((111 2z" 14/00)
and
P(a = 0|x E Dy. P(a 1 E Dy,y)
(1 (1-p)((-(1 22" 22 )p/00)
(47)
pD(-u/0) + (1 p) ((1((1111)) 2=" 11/00)
Under the assumption that p > 3/4 and noore >> Vgg we have 1 2-" = ((3 1-)ppp.r + Map ( m < 0. Hence,
(- 22*)p/ pp/o)) 1/2 and P(a 1|x E Py. v) 1/2: in other words, among the set of all correctly predicted samples with the label y, more than half of them generated with =
Moreover, under the assumption that (--1/o)) R 0, i.e., predictions of the model have a high signal-to-noise ratio, then P(a = 0|x Dy. v) > 1/2, i... among the set of all incorrectly predicted samples with the label y, more than half of them are generated with a = 0. This assumption is generally true, as of = Var (x + )2 + 7 (2*) is typically very small when z* approaches zero given > 3/4 and n >>>>>>> 1
A.2.8. PROOF OF LEMMA 3
Lemma 3. Consider the model fxx x with u = [Ucores Mapu), the optimal solution for Usp. denoted as U.g can be achieved by last-layer retraining on the retraining data with Pie and is calculated as
(2pre
1)
u spu
7.
(48)
noore + Tipu
Proof. First, have f(x) = Wx. For last- retraining, b is optimized. Following the derivation in Lemma 1, we similarly obtain the inequality in (27) with p Pre. i.c..
1 ( ( (1 7"' Usgu) 2 (1 Pre Manee (1 + n" uupu)) Vapa 11 11pul13) {(b) 2
(49)
Note that the terms the right side of the inequality independent of any manipulation of the retraining data, such as reweighting. Then, taking the derivative of the sum of these terms with respect to b, we obtain the following equation
7 W spu b (2pre 1) name
(50)
7p
where Uspu W W.pub. b. Since >TT-= = the terms on the right hand side of (49). Pre equals to p for the training data, then u u defined in (13).
then we have Uspu = Uppu- We finally verify that u indeed minimizes the sum of
19

---

NeuronTunge Towards Self-Guided Spurious Bias Mitigation
A.3. Connection to Last-Layer Retraining Methods
Although our method shares similar setting to last-layer retraining methods, such as AFR (Qiu et al., 2023) and DFR (Kirichenko et al., 2023), our method is fundamentally different from these methods in how spurious bias is mitigated. Take AFR for an example. It, in essence, is sample-level method and adjusts the weights of the last layer indirectly via retraining on samples with loss-related weights. Our method directly forces the weights identified as affected by spurious bias to zero, while adjusting the remaining weights with retraining.
The advantage of NeuronTune can be explained more formally in our analytic framework. First, considering the training loss in (17), we can express it as the sum of the following terms for brevity,
1 er (W b) 2 pPElus i (ueore: Uspu ) + 2 - pp)pppBBpp 2h (ucorey Mspu)] + 2 U' (Umg)):
1
(51)
where p is the data generation parameter and is constant, and 1/, 22.2 and Vi are defined as
w Ucores Mspa)
EE| [e||xxaree Hanre (1 7 Uqu) 3 Xawel 1
2/2 Ucore, Uspu) EE| EE|x x Mecore
(1 + 72 Usgu 3 X 11:
and
4/ (1ppp) pr (1 7 Up ) + (1 pooc (1 + >" Usuu) )2 + nppully u
respectively. Based on Lemma 3, for last-layer retraining methods in general, the optimal solution for Uspa is up given that the retraining data follows the same distribution as the training data.
AFR changes the distribution within the first two expectation terms 4/1 (Ueere, Uspu) and 2/2 (eere Uspu) and jointly updates Ucore and ugu. while there is optimality guarantee for Upu (1/s (4)g(U)))) is not considered in AFR). By contrast, according to Theorem 4.3, NeuronTune first ensures that Uspa is optimal, then it moves Ugore close the the unbiased solution.
A.4. Comparison between Models Selected with Worst-Class Accuracy
We compared approach with AFR (Qiu al., 2023) and JTT (Liu al., 2021) to demonstrate the challenges of the unsupervised setting for semi-supervised methods. These methods were tuned using worst-class accuracy (Yang et al., 2023) on the validation set instead of WGA. As shown in Table 6, our method exhibits larger performance gains over AFR and JTT compared to their results presented in Tables and 2.
Table 6. WGA comparison when models selected by the worst-class accuracy on the validation set.
Method
CelebA Waterbirds
JTT
84.2
52.3
AFR
89.0
68.7
NeuronTune
91.8_0.8
83.0
A.5. Complexity Analysis
We analyze the computational complexity of our method, NeuronTune, alongside representative reweighting-bast based methods, including AFR (Qiu et al., 2023). DFR (Kirichenko et al., 2023), and JTT (Liu et al., 2021). Let the number of identification samples be Nide, the number of retraining samples be N the total number of training samples be N, the number of latent dimensions be M, and the number of training epochs be E. Additionally, denote the time required for inference as Tiw, for last-layer retraining as Mi and for optimizing the entire model as Topt- The computational complexities of these methods are summarized in Table 7.
Among the methods, JTT has the highest computational complexity since Topt >> Tii requiring full model optimization. DFR is much faster due to its reliance on last-layer retraining. though it requires group annotations. AFR extends DFR by additionally precomputing sample losses, increasing its computational cost slightly. NeuronTune, while requiring more time than AFR to identify biased dimensions across all M embedding dimensions, remains computationally efficient. This is because Tiw, the time required for forward inference, is typically very small. As a result, Neuron offers an effective balance between computational efficiency and robust spurious bias mitigation.
20

---

NeuronTune: Towards Self-Guided Spurious Bias Mitigation
Table Computation complexity comparison hetween NuuronTune and reweighting methods.
Method
Time complexity
JTT (Liu et al., 2022)
( ETpp)
AFR (Qiu al., 2023)
((Nide Mdde Tww + E. Emi)
DFR (Kirichenko al., 2023)
O Eru)
NeuronTune
O(B(Nia: + Bmi)
A.6. Advantages over Variable Selection Methods
Although the identification of biased dimensions in (6) may resemble traditional variable selection methods (Heinze et al., 2018), our approach extends beyond simply selecting subset of variables that optimally explain the target variable. Instead, it specifically addresses spurious bias-an an issue often neglected in traditional variable selection.
Traditional variable selection methods, such as LI regularization, do not differentiate between variables representing spurious attributes and those representing core attributes. Since spurious attributes are often predictive of target labels in the training data and are easier for models to learn (Tiwari & Shenoy, 2023; Ye et al., 2023), these methods may mistakenly prioritize spurious attributes, thereby amplifying spurious bias. contrast, our method explicitly targets dimensions affected by spurious bias and rebalances the model's reliance on features to reduce dependence on spurious information.
Furthermore, unlike many variable selection methods that require explicit supervision (e.g., labels or statistical relationships) to mitigate spurious bias, NeuronTune operates in an unsupervised setting where group labels indicative of spurious attributes are unavailable. By leveraging misclassification signals to estimate spuriousness scores, our method is better suited for scenarios where group annotations are costly or infeasible, offering a practical and scalable solution to challenges of spurious bias mitigation.
A.7. Dataset Details
Table 8 gives the details of the two image and two text datasets used in the experiments. Additionally, the ImageNet-9 dataset (Xiao et al., 2021) has 54600 and 2100 training and validation images, respectively. The ImageNet-A (Hendrycks et al., 2021) dataset has 1087 images for evaluation.
A.8. Training Details
Table 9 and Table 10 give the hyperparameter settings for ERM and NeuronTune training, respectively.
A.9. Visualizations of Unbiased and Biased Dimensions
We provide visualizations of the neuron activation value distributions for the identified unbiased and biased dimensions in Figs. 3 to 6. The biased and unbiased dimensions selected for visualizations are obtained by first sorting the dimensions based on their spuriousness scores and then selecting three biased dimensions that have the largest scores and three unbiased dimensions that have the smallest scores. Note that dimension does not exclusively represent a core or spurious attribute; it typically represents a mixture of them.
On the CelebA dataset, as shown in Fig. 3, samples that highly activate the unbiased dimensions have both males and females; thus, the unbiased dimensions do not appear to have gender bias. For samples that highly activate the identified biased dimensions, all of them are females, demonstrating a strong reliance on the gender information. In Fig. 4, samples that highly activate the identified biased dimensions (right side of Fig. 4) tend to have slightly darker hair colors or backgrounds, as compared with samples that highly activate the identified unbiased dimensions (left side of Fig. 4). With the aid of the heatmaps, we observe that these biased dimensions mostly represent a person's face, which is irrelevant to target classes.
On the Waterbirds dataset, as shown in Fig. 5, for the landbird class, the identified unbiased dimensions mainly represent certain features of a bird and land backgrounds. For the identified biased dimensions, they mainly represent water backgrounds, which are irrelevant to the landbird class hased on the training data. For the waterbird class, as shown in Fig. 6, the identified unbiased dimensions mostly represent certain features of a bird and water backgrounds, while the identified biased dimensions mainly represent land backgrounds
21

---

Nune:nTune: Towards Self-Guided Spurious Bias Mitigation
Table 8. Numbers of samples in different groups and different splits of the four datasets.
Class
Spurious attribute
Train
Val
Test
Waterbirds
landbird
land
3498
467
2225
landbird
water
184
466
2225
waterbird
land
56
133
642
waterbird
water
1057
133
642
CelebA
non-blond non-blond
female
71629
8535
9767
male
66874
8276
7535
blond
female
22880
2874
2480
blond
male
1387
182
180
MultiNLI
contradiction contradiction
negation
57498
22814
34597
negation
11158
4634
6655
entailment
no negation
67376
26949
40496
entailment
negation no negation
1521
613
886
neither
66630
26655
39930
neither
negation
1992
797
1148
Civi Comments
neutral neutral toxic toxic
no identity
148186
25159
74780
identity
90337
14966
43778
no identity
12731
2111
6455
identity 17784
2944
8769
Table 9. Hyperparameters for ERM training.
Hyperparameters
Waterbirds
CelebA
ImageNet-9
MultiNLI
CivilComments
Initial learning rate
3e-3
3e-3
le-3
le-5
lc-3
Number of epochs
100
20
120
10
10
Learning rate scheduler
Cosine
Cosine
M0l6iStep140.60.80]
Linear
Linear
Optimizer
SGD
SGD
SGD
AdamW
A
Backbone
ResNet50
ResNet50
ResNet18
BERT
BERT
Weight decay
le-4
le-4
le-4
le-4
le-4
Batch size
128
128
16
16
Table 10. Hyperpaaametters for NeuronTune.
Hyperparameters
Waterbirds
CelebA
ImageNet-9
MultiNLI
CivilComments
Learning rate Number of batches per epoch
le-3
le-3
le-3
le-5
le-3
200
200
200
200
200
Number of epochs
40
40
60
60
Optimizer
SGD
SGD
SGD
AdamW
AdamW
Batch size
128
128
128
128
128
22

---

Neuron Tune: Towards Self-Guided Spurious Bias Mitigation
23
Imagga
Imagye
Teattaap
0.5
2.8
heatmaap
03
0.1
(a) Identified unbiased dimensions for non-blond hair
(b) Identified biased dimensions for non-blond hair
Figure 3. Value distributions of the correctly (blue) and incorrectly (red) predicted samples for unbiased (a) and biased (b) dimensions. along with the representative samples, respectively, based on the non-blond hair samples in the CelebA dataset.
Images
Images
hestmaps
020
0.6
1.5
Images
Imsges
135
Testmaps
0.5
Inaoes
5.6
teammaps
Teammaps
(a) Identified unbiased dimensions blond hair
Identified biased dimensions for blond hair
Figure Value distributions the correctly (blue) incorrectly (red) predicted samples for unbiased (a) biased (b) dimensions. along with the representative samples, respectively, based on the blond hair samples in the CelebA dataset.
23

---

Neuron Tune: Towards Self-Guided Spurious Bias Mitigation
Imagps
104
Imappe
1.9
Heatmays
Hestmaap
2.0
mave
Image
Mestmaps
Images
125
Hestmaps
0.3
0.2
0:2
(a) Identified unbiased dimensions for landbird
(b) Identified blased dimensions for landbird
Figure 5. Value distributions the correctly (blue) and incorrectly (red) predicted samples for unbiased (a) and biased (b) dimensions, along with the representative samples, respectively, based the landbird samples in the Waterbirds dataset.
14410
0.4
1.0
Inapes
Pertmaps
(a) Identified unbiased dimensions for waterbird
(b) Identified blased dimensions waterbird
Figure Value distributions the correctly (blue) and incorrectly (red) predicted samples for unbiased (a) and biased (b) dimensions, along with the representative samples, respectively, based the waterbird samples in the Waterbirds dataset.
24


Reviews:
[{"review_id": "m5mzKDrFky", "reviewer": "Reviewer_BqZa", "summary": "The work proposes a bias-unaware post-hoc model debiasing method. The approach\nis based on the observation that, in a setting where a high majority of\nsamples, but not all, contain a biased attribute, the neurons in the\npenultimate layer that are affected by the spurious attribute exhibit a\ndifferent behavior between correctly classified and misclassified samples.\nThe work theoretically motivates this in a linear regression setting, and\ndefines the difference between medians of classified and misclassified samples\nas an indicator for neurons that are sensitive to spurious attributes.\nThe debiasing is conducted by setting neurons in the penultimate layer above a\ncertain distance threshold to zero, and retraining the final classification\nlayer. A toy example serves to further highlight the efficacy of this approach.\nEmpirical experiments on four popular benchmark datasets show\nworst-group-accuracy values of the proposed method competitive with approaches\nfrom prior work which leverage bias labels in the validation set for model\nselection.\n\n## update after rebuttal\n\nI thank the authors for their clarifications. I am very happy to see the increased number of trials. I am convinced of the quality, novelty and significance of this work, and thus choose to revise my initial recommendation to \"Accept\".", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your thoughtful review and valuable feedback on our submission. Here are our responses to address your questions.\n\n## Evaluation Criteria\n- **Usefulness of Accuracy Gap**: We hope to explicitly provide readers with a direct view on the numeric difference between worst-group accuracy and average accuracy, making it easier to assess how consistently a model performs across different groups of data. When models with very low worst-group accuracy exhibit inflated accuracy gaps, such as the ERM models, this indeed highlights the severe robustness issue of the models. In this paper, we report the average accuracy, worst-group accuracy, and the accuracy gap between them to provide a comprehensive evaluation of each model's overall performance.\n\n## Essential References Not Discussed\n- **Connection to the two related works**: Thanks to the reviewer for giving a clear comparison between our work and the works proposed by Kim et al, 2022a and Liu et al., 2021. Indeed, our work builds on idea of these works by extending the focus from identifying bias-conflicting samples to detecting biased neurons within a model, enabling direct intervention in the model’s decision-making process.\nWe will add the above discussion to our revised manuscript to better connect our method to these two works.\n\n## Weaknesses\n\n1. **Regarding number of trials and clear improvements**:  Following your suggestion, we ran our method for five additional trials. Overall, the accuracies remained stable with reduced variance. Our previous results in the paper and the new results below clearly demonstrate the advantage of our method in the unsupervised bias mitigation setting, where no group labels are available.\n\n|| NeuronTune ||NeuronTune$^\\dagger$||\n|---|---|---|---|---|\n||WGA|Acc.|WGA|Acc.|\n|Waterbirds|92.2±0.3|94.4±0.2|92.5±0.9|94.5±0.3|\n|CelebA|83.1±1.1|92.0±0.5|87.3±0.4|90.3±0.5|\n|MultiNLI|72.1±0.1|81.1±0.6|72.5±0.3|80.3±0.6|\n|CivilComments|82.4±0.2|89.2±0.1|82.7±0.4|89.4±0.2|\n\n2. **Regarding the assumption on the training set**:  We would like to clarify that our method does not rely on any assumptions about the training set. Since our approach uses an independent identification set for bias detection, it can be applied to models trained on any datasets. Notably, if the training set contains fewer biased samples, the resulting model is less susceptible to spurious bias. In such cases, NeuronTune will identify fewer biased neurons and the model will receive less intervention for bias mitigation.\n\n## Other Comments Or Suggestions\nWe will make revisions at the suggested locations and thoroughly review the rest of our paper.\n\n## Questions\n\n1. **The error values in Table 2 are quite high. Could this be caused by the low number of trials? Would at least e.g. five trials be possible? Did you conduct a Wilcoxon-rank-sum test?** We hope to clarify that the variances of our method are relatively low compared to other methods. After running five additional trials, we observed reduced variance. Please refer to the table in Weakness 1 for the results. Our method demonstrates clear improvements in the unsupervised bias mitigation setting. Since we compare against the best-reported performance of baseline methods, a Wilcoxon rank-sum test is not directly applicable in this context.\n\n2. **The decision boundaries are different between training and testing in Figure 2a and 2c**:  This discrepancy arises because we used two separate K-nearest neighbor classifiers, each fitted to the predictions on the training and test sets separately, to determine the decision boundaries. To ensure consistency, we have now used a shared K-nearest neighbor classifier fitted on both training and test data to determine decision boundaries. The revised figure can be accessed  [here](https://anonymous.4open.science/r/NeuronTune-6CFC/synthetic_example.png).\n\n3. **Regarding the statement in l.379 left**: We hope to clarify that in Tables 1 and 2, we group methods that are directly comparable and boldface the best result within each group of methods (L345 and L398). For example, on MultiNLI (Table 2), AFR belongs to the methods that require group labels, while NeuronTune does not require group labels.\nNeuronTune achieves the highest WGA within methods that do not require group labels and even remains competitive against methods that require group labels. We will revise L379 to further clarify this in the paper.\n\n6. **When using the validation set to find neurons that are sensitive to spurious behavior, is the training set used to tune the last layer?** Yes, for \"NeuronTune,\" the validation set is used only to identify biased neurons, while the training set is reused to tune the last layer. \"NeuronTune$^\\dagger$\" is a variant where half of the validation set is used for detection and the other half for tuning.\n\n\n Please let us know if you have further questions.", "claims_and_evidence": "The work claims their proposed debiasing approach, which does not require bias\nannotations, significantly mitigates spurious biases, i.e., is competitive with\nprevious bias mitigation approaches that do require bias annotations.\nThis claim is supported by empirical evidence, comparing the\nworst-group-accuracy of various baselines. There seems to be a small issue given\nthe high error, likely caused by the small number of conducted trials. However,\nas the work does not explicitly claim state-of-the-art, but only\ncompetitiveness, this is only a minor issue.", "methods_and_evaluation_criteria": "The work conducts empirical analysis on common benchmark datasets from the model\ndebiasing literature. Here, the worst-group performance is used as an evaluation\ncriteria, which is a common metric.\nThe work also uses the *accuracy gap* to evaluate the gain between average and\nworst-group-performance. While I understand the motivation behind this metric, I\ndo not think that this is very useful, as models with very low\nworst-group-performance obtain inflated values, which still require the\ncomparison to average and worst-group-performance.\n\nThe work also proposes a synthetic dataset to visually motivate its approach.\nWhile the visualization does indeed provide a good intuition for the issue.", "theoretical_claims": "The work makes some theoretical claims on a linear regression model in order to\nmotivate its method. Here, Proposition 4.1, given separated spurious and core\nfeatures in a linear model, demonstrates the negative contribution of spurious\nneurons.\n\nGiven these negative contributions, Theorem 4.2 shows that there is then some\ndistance between the expected activations of correctly classified and misclassified samples relying on this negative contribution of Proposition 4.1.\n\nTheorem 4.3 then assumes that the neurons detecting core features are aligned\nwith the true core direction approximately to the same degree as the neurons\ndetecting spurious features are aligned to the true spurious direction, with\nwhich the proposed approach is shown to reduce the distance between the unbiased\nand the biased model.\n\nI did not rigorously check Theorems 4.2 and 4.3.", "experimental_designs_or_analyses": "The experimental design of the results discussed on Tables 1-3 is based on a\nstraight-forward comparison on previous benchmarks. The only potential issue\nhere could be the low number for trials (three).", "relation_to_broader_scientific_literature": "The work discusses most relevant works, making the common separation between\nbias-aware, bias-unaware, and semi-bias-aware mitigation approaches.\nThe idea that bias can be identified through the misclassification of\nbias-conflicting samples has been identified in prior work (Kim et al, 2022a;\nLiu et al., 2021).\nReweighing or retraining the final layer for bias mitigation has also been\nleveraged in previous work (Liu et al., 2021; Qiu et al., 2023).\nThe novelty in this work is the contribution of individual neurons to this\nmisclassification, and the subsequent, targeted removal thereof.", "essential_references_not_discussed": "This works bases its bias detection on the idea that biased models misclassify\nbias-conflicting samples. Two prior works that leverage the same idea are cited\nand compared empirically (Kim et al, 2022a; Liu et al., 2021). However, the work\ndoes not make this connection clear.", "other_strengths_and_weaknesses": "### Strengths\n\n- Unsupervised bias mitigation, not relying on known biases, is an interesting\n  and important issue\n- The linear regression example motivates the approach well. I really like\n  Figure 2, except for the issue of the decision boundary described below.\n- The theoretical analysis provides a good foundation for the efficacy of the\n  model.\n- The work is written clearly, and seems polished well.\n\n### Weaknesses\n\n- The empirical results have high error due to a low number of trials, and do\n  not show very clear improvements.\n- The framework relies on the assumption that a overwhelming majority of the\n  training set contains biased samples in order for the model to react to\n  bias-conflicting samples.", "other_comments_or_suggestions": "- In Figure 2b: The median for Dimension 2 red seems off.\n\n- l.255 left: linear combination spurious and core in embedding typically holds in embedding space, \"combination is nonlinear\" should be specified more clearly (i.e., in input)\n- l.307 left: absolution -> absolute\n\n- l.791: so that suppress[ing] them improves the ...", "questions_for_authors": "1. The error values in Table 2 are quite high. Could this be caused by the low\n   number of trials? Would at least e.g. five trials be possible? Did you\n   conduct a Wilcoxon-rank-sum test?\n2. The decision boundaries are different between training and testing in Figure\n   2a and 2c. Is this caused by errors in the contour lines? Should they not be\n   exactly the same?\n3. l.379 left states \"Our method achives highest WGA across the datasets\" for\n   Table 1 and Table 2. However, this does not seem to be fully the case. E.g.,\n   On Multi NLI, AFR is just overall better, yet NeuroTune is in bold-face.\n4. When using the validation set to find neurons that are sensitive to spurious\n   behavior, is the training set used to tune the last layer?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "9C9aWMXoPZ", "reviewer": "Reviewer_MxxR", "summary": "The authors propose to improve OOD generalization by identifying neurons that contribute significantly to misclassification on the validation set, and then retraining the output layer while setting those neurons to zero. Relative to ERM, this simple idea trades off in-distribution accuracy for significant improvement on worst-group accuracy across many image and text classification tasks.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your thoughtful comments and for recognizing both the simplicity and effectiveness of our approach. We appreciate your valuable feedback and are glad that our efforts to explain the model selection strategies were helpful. \n\n## Regarding the subpopulation shift interpretation\n\nWe appreciate your thorough understanding of our work and your insightful interpretation from the perspective of subpopulation shifts, i.e., changes in the proportion of certain subpopulations between training and testing. We would also like to clarify the distinction between mitigating **spurious bias** and **subpopulation shifts**. \n\n- Mitigating subpopulation shifts often relies on reducing spurious bias, such as decreasing the strong correlation between waterbirds and water to improve generalization for waterbirds on land. Tables 1 and 2 demonstrate NeuronTune’s effectiveness in handling subpopulation shifts on the Waterbirds, CelebA, MultiNLI, and CivilComments datasets.\n  \n- In contrast, mitigating spurious bias has broader implications beyond subpopulation shifts. For example, Table 3 highlights NeuronTune’s ability to enhance robustness on the ImageNet-A dataset, which consists of samples that are challenging for a pre-trained model but do not conform to well-defined subpopulation shifts, such as images with unique pixels. This evaluation scenario represents a broader and more complex problem than subpopulation shifts.\n\n**Therefore, by framing our method as targeting spurious bias, we address a wider scope of challenges beyond subpopulation shifts.**\n\nWith proper examples,  our current presentation aligns well with your proposed subpopulation shift interpretation. We detail the correspondence between your interpretation and our presentation below.\n\n> When you train ERM on datasets with imbalanced subpopulations, it maximizes average-case accuracy and performs very poorly on rare subpopulations. E.g. on waterbirds, it focuses on being correct on waterbirds on water, and sacrifices its ability to classify waterbirds on land. \n\nConsider that $a$ in Eq. (2) controls subpopulations in data, e.g., when $a=1$, it may represent a group of waterbirds on water, and when $a=0$, it may represent a group of waterbirds on land. The probability $p$ controls the severity of imbalance in subpopulations. When $p$ is close to one (L159 left), the data is severely imbalanced in subpopulations. After training with ERM, the model minimizes the training loss, i.e., maximizes average-case accuracy, but obtains a large nonzero weight on the spurious feature (Lemma 1) and is away from the optimal model (Corollary 1). For example, the model may focus on being correct on waterbirds on water and sacrifice its ability to classify waterbirds on land.\n\n> Then, when you look at the misclassifications on the validation set, the features that you identify are precisely the ones that make you misclassify waterbirds on land. You zero these features out, and retrain the output layer to assign higher weights to the features that would allow you to correctly classify waterbirds on land, i.e., bird features.\n\nOur principle of neuron identification (Proposition 4.1) states that when a spurious correlation breaks, the neurons that still positively contribute to mispredictions will be selected.  For example, neurons that lead to misclassification on waterbirds on land will be identified. After retraining, Theorem 4.3 indicates that weights on core features, such as bird features, will be increased (L200-201, right).\n\n> By doing this, you slightly sacrifice your ability to classify waterbirds on water, but significantly improve your ability to classify waterbirds on land.\n\nOur findings (L196-201, right) show that our method increases the weights on core features while keeping the weights on spurious features unchanged. This suggests that our approach makes a slight trade-off in average-case accuracy to achieve improved worst-group accuracy.\nFor example, our method may slightly reduce the model’s ability to classify waterbirds on water due to a relative decrease in reliance on the water feature, while significantly enhancing its ability to classify waterbirds on land.\n\n Thanks again for your thoughtful comments. We will incorporate concrete examples to provide readers with better intuition and a clearer understanding of our approach. \n\nPlease let us know if you have further questions.", "claims_and_evidence": "* The authors' main claim is that their algorithm is robust to \"spurious correlations,\" which are defined as a non-causal relationship.\n* Their approach is motivated using a DGP with causal/anticausal features, originally proposed in the Invariant Causal Prediction (Peters et al., 2015) paper.\n* Their experimental results show that their algorithm is able to effectively trade-off in-distribution performance for out-of-distribution performance in the presence of subpopulation shift.\n* I believe the connection between the claims and evidence is currently weak, and would be improved by reframing the algorithm as a way to mitigate subpopulation shift.", "methods_and_evaluation_criteria": "* Yes, the proposed methods and evaluation make sense for the problem of OOD generalization.", "theoretical_claims": "N/A", "experimental_designs_or_analyses": "* The experimental design and analysis are sound.\n* I liked the fact that the authors are very explicit about how they perform model selection using in-distribution data. This is an extremely important point that is often omitted in this research area.", "relation_to_broader_scientific_literature": "The authors primarily contextualize their work in relation to last-layer retraining with labeled spurious correlations, and domain generalization (labeled groups). I think this is fine, but I believe the work more strongly relates to subpopulation shift (see \"Other strengths and weaknesses.\")", "essential_references_not_discussed": "N/A", "other_strengths_and_weaknesses": "The proposed idea is elegant, and meaningfully beats ERM on a wide variety of very difficult benchmarks. However, I think the current causality-based presentation obfuscates the simplicity of the idea, and does not help the reader gain intuition on why it should improve OOD generalization.\n\nThis is how I understand your method. When you train ERM on datasets with imbalanced subpopulations, it maximizes average-case accuracy and performs very poorly on rare subpopulations. E.g. on waterbirds, it focuses on being correct on waterbirds on water, and sacrifices its ability to classify waterbirds on land. Then, when you look at the misclassifications on the validation set, the features that you identify are precisely the ones that make you misclassify waterbirds on land. You zero these features out, and retrain the output layer to assign higher weights to the features that would allow you to correctly classify waterbirds on land, i.e. bird features. By doing this, you slightly sacrifice your ability to classify waterbirds on water, but significantly improve your ability to classify waterbirds on land. Hence, your algorithm improves on worst-group accuracy at the expense of average-case accuracy. The same conclusion can be reached using CivilComments, which is categorized as a subpopulation shift dataset within WILDS.\n\nI think this makes much more sense than the causality-based argument which hinges on unintuitive technical assumptions about the data generating process. I am going to recommend acceptance as-is, because I understand that my interpretation may be overfit to my own personal intuitions about this research area. However, I believe this paper would be stronger with this reframing.", "other_comments_or_suggestions": "N/A", "questions_for_authors": "Let me know what you think about the subpopulation shift interpretation that I proposed in \"other strengths and weaknesses.\"", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Xbm4EY64Uq", "reviewer": "Reviewer_fYLW", "summary": "The paper proposes NeuronTune, a method for mitigating spurious correlations in neural networks by intervening directly in the latent space at the neuron (feature-dimension) level. The work addresses the common challenge that many “robustness” or “debiasing” approaches rely on knowing or inferring spurious attribute annotations (e.g., “group labels”), which can be expensive to obtain or sometimes even unknown a priori. NeuronTune provides a self-guided approach: it identifies “biased dimensions” whose activations correlate strongly with misclassifications and suppresses those dimensions through post-hoc last-layer retraining.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your detailed review and feedback on our submission. We provide our responses to your questions below.\n\n## Essential References Not Discussed\n- **Comparison with NeuroInspect**: NeuroInspect identifies neurons responsible for mistakes from the counterfactual explanation perspective and minimizes the influence of identified neurons via regularization. In contrast, our approach identifies biased neurons by probing their activation patterns and directly manipulates identified neurons by suppressing their contributions to the final predictions.\\\nNeuroInspect emphasizes on interpretable neuron-level debugging, allowing practitioners to further pinpoint the causes of the errors. Our work aims at a self-guided spurious bias detection and mitigation framework without human intervention.\\\nWe see potential in applying the interpretation techniques to our method in future work. We will cite NeuroInspect and add the above discussion to our revised manuscript. \n\n## Weaknesses\n\n1. **Limited interpretability**: We hope to clarify that our work does not focus on interpretation. Instead, we focus on a self-guided framework without human intervention. By definition, biased neurons may encode a certain portion of core features, making the interpretation of neurons nontrivial. Nevertheless, we provide visualizations in Figures 3-6 in Appendix to analyze which spurious concepts are discovered. Please refer to our response to Question 4 for details. \n \n2. **Dependence on a separate identification set**: One of our major contributions is removing the dependence on group labels in the identification set. This offers great flexibility and freedom for practitioners to select identification data based on their needs to detect and mitigate targeted spurious biases. Hence, obtaining such data is relatively straightforward. In our experiments, we used the validation set for consistency in performance comparison, but in real-world scenarios, any sufficiently representative data can serve this purpose. \n\n## Questions:\n\n1. **Iterative Tuning**: Yes, we performed multiple rounds of neuron identification and suppression (L256-259, right column). Diminishing returns will be observed after a certain number of rounds. In general, running multiple rounds is better than running a single round with improved performance, as new biased neurons will be identified.\n\n2. **Partial vs. Full Suppression**:  Although partial suppression reduces the loss in overall accuracy, its effect is similar to no suppression, as models can continue to adjust their weights on nonzero biased activations, leading to severe spurious bias. For example, on the CelebA dataset, when we multiplied the activations of biased dimensions with the masking values in the table below, we see that only when the masking value is zero (full suppression)  did the model achieve improved WGA.\n\n|Masking value|0|0.2|0.4|0.6|0.8|1.0|\n|---|---|---|---|---|---|---|\n|WGA|87.3±0.4|71.5±1.5|72.2±1.2|72.9±1.5|73.1±1.5|73.0±1.2|\n|Acc.|90.3±0.5|93.8±0.2|93.8±0.3|93.8±0.2|93.8±0.2|93.9±0.2|\n\n3. **Identification Set Size/Composition**: The identification set should consist of diverse samples that are not memorized by the model. We find that when we use less than 30\\% samples of the validation set, the obtained worst-group accuracy is low with high variance, whereas the average accuracy remains high with low variance. When we use around 30\\% of the validation set, the number of minority group samples (e.g., waterbirds with land backgrounds in Waterbirds or males with blonde hair in CelebA) is approximately 50. Therefore, to ensure the effectiveness of our method, each data group should ideally contain more than 50 samples. In a group-label-free setting, it is suggested to include many diverse samples whenever possible.\n\n4. **Interpretability**: Yes, we have shown images that highly activate the identified biased neurons along with the heatmaps highlighting regions that contribute to the neuron activations in Figures 3-6 in Appendix. The identified biased neurons mainly represent spurious features. For example, we observe that biased neurons mainly represent blue/green backgrounds for landbirds in Figure 5(b) and represent forest backgrounds for waterbirds in Figure 6(b). Note that since a neuron may represent a mixture of spurious and core features, the visualizations sometimes may not be directly interpretable.\n\n5. **Extension to Other Tasks**: Yes, NeuronTune can be extended to multi-label prediction tasks. Instead of considering all identified neurons for different labels as biased, as shown in Eq. (6), we can consider biased neurons as label-specific and mask neurons based on the labels of input samples during retraining. This is because in the multi-label setting, a biased neuron for one label may be the core contributor to another label, which is different from the assumption in a standard classification setting (L272-274).\n\nPlease let us know if you have further questions.", "claims_and_evidence": "Overall, the claims are backed by both theoretical insights and strong empirical evidence.", "methods_and_evaluation_criteria": "These criteria align with standard practices in the literature on spurious correlation mitigation.", "theoretical_claims": "Key theoretical elements include:\n\n1. A formal data model that encapsulates both core and spurious features.\n2. Proofs establishing the connection between median-activation differences and biased (spurious) neuron detection.\n3. Analysis that shows last-layer retraining plus biased-neuron suppression reduces reliance on spurious features.\n\nThe analysis is coherent for linearized settings and helps explain why high-activation, misclassification-associated neurons reveal spurious correlations.", "experimental_designs_or_analyses": "The methodology is generally rigorous. One natural trade-off is a mild drop in average accuracy for a bigger gain in worst-group performance.", "relation_to_broader_scientific_literature": "The authors contextualize well with respect to spurious bias research and last-layer retraining. They might also elaborate on recent neuron-level debugging works such as “NeuroInspect: Interpretable Neuron-based Debugging Framework through Class-conditional Visualizations” [Ju et al., 2023] to compare or contrast how each approach manipulates internal neurons.", "essential_references_not_discussed": "A potentially relevant reference is “NeuroInspect,” which also performs neuron-level interventions. Referencing it explicitly could strengthen the discussion on interpretable neuron-level debugging strategies.", "other_strengths_and_weaknesses": "## Strengths:\n- Self-contained, requiring no group-label supervision.\n- Effective across various architectures (ResNet, BERT) and data modalities (vision and NLP).\n- Theoretical rationale matches empirical success.\n\n## Weaknesses:\n- Limited interpretability: Although the approach identifies “biased neurons,” the paper does not deeply explore how to interpret or label them. (This is not a critical flaw, but might be interesting for users to see “which spurious concepts are discovered.”)\n- Dependence on a separate identification set: The paper relies heavily on the notion that a distinct dataset (or at least a validation set) is available. In real scenarios, that might be limited or small. The ablations show that using training data to identify biased dimensions is less effective (Table 4).", "other_comments_or_suggestions": "- Iterative neuron identification: After carrying out one pass of NeuronTune, it might be worthwhile to re-run the biased-neuron detection on the newly tuned model. If certain features remain entangled or if new spurious signals emerge, a second iteration could potentially refine results.\n- Interpretation: It would be interesting to present visual or textual exemplars that highlight what the suppressed neurons correspond to semantically.", "questions_for_authors": "1. Iterative Tuning: Have you explored running multiple rounds of neuron identification and suppression? If so, did you notice any diminishing returns or additional complexity? If not, do you speculate that repeated rounds might identify new “biased” neurons or further improve performance?\n\n2. Partial vs. Full Suppression: Did you consider only attenuating (rather than fully zeroing) the activations of biased dimensions? Could such a variant reduce potential losses in overall accuracy?\n\n3. Identification Set Size/Composition: Are there heuristics or rules of thumb for how large or diverse the identification set (used to detect biased neurons) should be? For instance, if only limited validation data is available, how does that affect the stability of the median-based criterion?\n\n4. Interpretability: Have you tried mapping the suppressed neurons to specific “concepts” or features (e.g., through visualization)? If so, does this reveal anything interesting about the nature of spurious correlations the model relies upon?\n\n5. Extension to Other Tasks: The paper focuses on classification tasks with known spurious correlations. Could the NeuronTune concept extend to multi-label or structured prediction tasks? If so, would that require any modification to the methodology?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: This paper received three Weak Accept recommendations. 

The reviewers acknowledged the theoretical analysis that serves as a foundation for the proposed approach, its novelty in terms of the identification of individual neurons affected by bias, suppressing their contributions for bias mitigation, and experiments including CNNs and BERT on both image and text classification tasks.  On the other hand, reviewers raised concerns about the assumption that the vast majority of the training samples are bias-aligned, low number of trials, connection between claims and provided experimental evidences, more in-depth analysis on the identified biased neurons as well as the dependence on a validation set, which may be hard to obtain in a realistic scenario. 

In their rebuttal, the authors clarify how they do not use group labels in the validation set, they perform more trials to provide more indication on the method’s robustness, as well as clarifying that the proposed method does not assume a specific bias-aligned/bias-conflicting ratio in the training set, but would likely find less biased with the final model receiving less intervention, and provide more details on the setting and design choices. 

After reading the paper, the AC has a further comment on the related work section, specifically in the unsupervised Spurious Bias Mitigation subsection, which appears not articulated enough, as it only reports one work and misses significant contributions in this field. Especially as the paper fits into this bias mitigation framework, describing and comparing against more of these works would be useful to assess the performance of the proposed approach correctly.  

Considering the reviewers’ scores and carefully evaluating the strengths and weaknesses of the proposed approach, the AC agrees that the proposed method is interesting and novel in its identification of biased neurons, with experiments showing its effectiveness. Therefore, they recommend the acceptance of this paper.
Decision: Accept (poster)