SEBRA : Debiasing through Self-Guided Bias Ranking
Authors: Adarsh Kappiyath
Abhra Chaudhuri
AJAY KUMAR JAISWAL
Ziquan Liu
Yunpeng Li
Xiatian Zhu
Lu Yin
Venue: ICLR 2025 Poster (2025)
Keywords: Spurious Correlations
Bias Mitigation
Sub-population shift
Fairness
Primary area: alignment, fairness, safety, privacy, and societal considerations
TL;DR: A method to sort/rank the data in the decreasing order of spuriosity without human supervision.

Abstract: Ranking samples by fine-grained estimates of spuriosity (the degree to which spurious cues are present) has recently been shown to significantly benefit bias mitigation, over the traditional binary biased-vs-unbiased partitioning of train sets. However, this spuriousity ranking comes with the requirement of human supervision. In this paper, we propose a debiasing framework based on our novel Self-Guided Bias Ranking (Sebra), that mitigates biases via an automatic ranking of data points by spuriosity within their respective classes. Sebra leverages a key local symmetry in Empirical Risk Minimization (ERM) training -- the ease of learning a sample via ERM inversely correlates with its spuriousity; the fewer spurious correlations a sample exhibits, the harder it is to learn, and vice versa. However, globally across iterations, ERM tends to deviate from this symmetry. Sebra dynamically steers ERM to correct this deviation, facilitating the sequential learning of attributes in increasing order of difficulty, ie, decreasing order of spuriosity. As a result, the sequence in which Sebra learns samples naturally provides spuriousity rankings. We use the resulting fine-grained bias characterization in a contrastive learning framework to mitigate biases from multiple sources. Extensive experiments show that Sebra consistently outperforms previous state-of-the-art unsupervised debiasing techniques across multiple standard benchmarks, including UrbanCars, BAR, and CelebA.

Published as a conference paper at ICLR 2025
SEBRA: DEBIASING THROUGH SELF-GUIDED BIAS RANKING
Adarsh Kappiyath', Abhra Chaudhuri', Ajay Jaiswal Ziquan Liu', Yunpeng Li", Xiatian Zhu', Lu Yin!*
University of Surrey, Fujitsu Research of Europe, University of Texas at Austin 'Queen Mary University of London
ABSTRACT
Ranking samples by fine-grained estimates of spuriosity (the degree to which spu- rious cues are present) has recently been shown to significantly benefit bias mit- igation, over the traditional binary biased-us-unbiased partitioning of train sets, However, this spuriosity ranking comes with the requirement of human supervi- sion. In this paper, we propose a debiasing framework based on our novel Self- Guided Bias Ranking (Sebra), that mitigates biases (spurious correlations) via an automatic ranking of data points by spuriosity within their respective classes. Sebra leverages key local symmetry in Empirical Risk Minimization (ERM) training - the ease of learning sample via ERM inversely correlates with its spuriousity; the fewer spurious correlations sample exhibits, the harder it is learn, and vice versa. However, globally across iterations, ERM tends to devi- ate from this symmetry. Sebra dynamically steers ERM correct this deviation, facilitating the sequential learning of attributes in increasing order of difficulty. ie., decreasing order of spuriosity. As a result, the sequence which Sebra learns samples naturally provides spuriousity rankings. We use the resulting fine- grained bias characterization in a contrastive learning framework to mitigate bi- ases from multiple sources. Extensive experiments show that Sebra consistently outperforms previous state-of-the-ar unsupervised debiasing techniques across multiple standard benchmarks, including UrbanCars, BAR, CelebA, MultiNLI, and ImageNet- IK. Code, pre-trained models, and training logs are available at nttps::/kadarset github..bhhb....... i0/ htt../thh...../b.r.r............ sebrraaierrrr55
INTRODUCTION
Distribution shifts driven by spurious correlations (aka biases or shortcuts) are arguably one of the most studied forms of subpopulation shift (Koh et al. 2021 Yang et al. 2023). Models trained on data that have certain easy-to-leard attributes, spuriousy correlated with labels, can overly rely on such spurious attributes, resulting in suboptimal performance during deployment (Geirhos al. 2019). Both supervised (Sagawa et al. 2020 [drissi et al. 2022) and unsupervised (Nam et al.) 2020 Liu et al. 2021 Li al. 2022 Park al.] 2023) methodologies for making neural networks robust to spurious correlations, a task also known as debiasing, have been developed. To get around the ex- pensive human labor involved in acquiring bias labels for training supervised debiasing algorithms, unsupervised methods typically take two-stage approach: an initial stage for bias identification and a second stage for bias mitigation. Unsupervised bias identification often relies on certain character- istics of spurious attributes, such as their relative ease of learning compared to target attributes (Nam) ett al. 2020). formation of clusters feature space (Sohoni et al. 20200, adherence to a low-rank property (Huh et al. 2023). etc. This is followed by the mitigation step via resampling (Idrissi et al.) 2022), contrastive learning [Zhang et al., 2022). and pruning (Park et al. 2023). etc.
Existing bias identification methods typically categorize data points into two (Nam et al. 2020 Liu ct al. 202T) or more discrete groups (Sohoni et al.) 22020; Yang et al. 2024). However, they do not offer insights into how the strength of spurious correlations varies across the identified groups, do they account for the variation in the strength of spurious correlations across instances within each
"Corresoonding author.

---

Published as a conference paper at ICLR 2025
group. Recent works, such as Singla & Feizi (2022); Moayeri et al. (2023). address these limitations by ranking data points based on spuriosity the degree to which common spurious cues are present. However, these spuriosity / bias ranking methods rely on human supervision or auxiliary biased models to identify biased features. Furthermone, they heavily rely on the interpretability of neural features extracted from adversarially trained encoders and the effectiveness of the interpretability techniques employed, which limits their applicability.
To address these limitations, we present Self-Guided Bias Ranking (Sebra), a spuriosity ranking algorithm without the need for human intervention. Sebra is based on the observation of a local symmetry in ERM (Empirical Risk Minimization) training in a given iteration, the hardness of learning a sample is inversely correlated with the amount of spurious features it contains. In other words, the lower the amount of spurious features, the harder a sample is to learn, and vice versa. We call this, the Hardness-Spuriosing Symmetry (Assumption [D. which consequently gives rise to a corresponding conservation law (Theorem [T) relating the hardness of learning a sample to a measure of its spuriosity. This implies that the spuriosity ranking can be derived by looking at the trajectory (through the sample space) of a model that learns attributes sequentially in increasing order of hardness. We empirically confirm the validity of the Hardness-Spuriosity Symmetry assumption in Appendix G.2
However, when training a neural network samples with varying levels of spuriosity, globally across iterations, ERM tends to deviate from this trajectory due to (a) reliance on spurious features, since higher spuriosity samples are known to inhibit the learning of those with relatively lower levels of spuriosity QQu et al. (2024) and (b) non-uniform gradient updates received for samples of different levels of spuriosity due different values of the task loss (influenced by their levels of spuriosity), leading to non-determinism in the order in which samples are learned. Sebra corrects this devi- ation by steering the optimization pathway of a neural network by dynamically modulating ERM through a pair of controller variables to follow the conservation law corresponding to the Hardness- Spuriosity Symmetry, while minimizing the interference caused by samples of one spuriosity level on the learning of the other, thereby enabling the network to learn attributes in increasing order of dif- ficulty. Consequently, a readout of the order in which samples are learned along this pathway serves as our predicted spuriosity ranking, requiring no human supervision. By leveraging the fine-grained spuriosity rankings obtained through Sebra and incorporating them into a simple contrastive loss, outperform previous state-of-the -het unsupervised and supervised debiasing techniques across multiple benchmarks, UrbanCars, BAR, CelebA, MultiNLI, and ImageNet-IK.
To summarize, we: 1 Introduce novel self-guided bias ranking framework, Sebra, dynamically rank the data points of each class on the decreasing order of the strength of spurious signals, without any human supervision; @ Utilize these derived rankings to enable debiased learning using a sim- ple contrastive learning framework; @ Empirically demonstrate the effectiveness of our proposed approach across multiple datasets with spurious correlations, including UrbanCars, CelebA, BAR, MultiNLI and ImageNet- 1K.
RELATED WORKS
Bias Identification: A plethora of methods assume knowledge of bias either in the form of bias labels Lee et al. () Idrissi al. (2022) type of bias Geirhos al. (2019): [hang et al. 1021 Even though these methods produce superior debiasing results, obtaining bias annotations for all biases identifying the type of bias requires significant human efforts. This led to the development of various inductive biases suitable for bias identification. One of the most commonly used inductive biases for bias identification is the property of bias being easier to learn. In In/Nam al.) (220)). bias is identified by obtaining bias-only model through upweighting data points that are easy to learn. Another popular bias identification strategy relies on training a model with a limited network capacity using empirical risk minimization Liu et al.] (202T), the hypothesis being that model with a small capacity would face difficulties in learning complex features and thus prefer to learn easy spurious features. Such simple bias identification schemes has shown to be very useful for datasets with single bias attributes but encounter Whac-A-Mar AMole dilemma Li et al. (2023) when faced with datasets with multiple spurious correlations. In Sohoni et al. (2020); Yang et al. (2024), clusters based on biased attributes in the feature space are utilized for bias identification but provide no means to characterize the nature of clusters discovered or how the strength of spurious attributes varies
2

---

Published as a conference paper at ICLR 2025
proposed a method to identify spurious and core attributes by analyzing neural features of adversarially trained encoders using interpretability techniques like GradCAM and feature attacks. Building these insights, MMoyeri et al. (202)) rank instances within a class based on the presence of these identified attributes, sorting data in decreasing order of spuriosity. Although these methods offer detailed characterization of spurious attributes in the dataset, their dependence on human supervision and the quality of interpretability techniques used can restrict their applicability. In contrast, our proposed ranking framework orders data in decreasing order of easiness to learn as perceived by an ERM model, supery
across these discovered clusters. Recently, Singlaa & Feizi 2022)
without relying on human supervision or fragile interpretability techniques.
Some of the simple bias mitigation strategies involve up-weighting bias conflicting points and down-weighting bias aligned points. thereby promoting the model to learn target features from the data Liu et al. (202T): [drissi et al. (2022) Lee et al. (202T). Other approaches include obtaining a debiased model by training a model to learn different mechanisms to that of a bias-only model Nam al. (2020), pruning Park et al. (2023) forgetting the bias information from biased model Tiwari & Shenoy (2023). In the presence group labels either inferred or via supervision, a debiased model is obtained by minimizing worst group risk Sagawa et al. (2020). Although simple upweighting methods have shown to be very effective in debiasing they lead to the underutilization of diversity of the training data resulting in suboptimal performance. With the more
Bias Mitigation:
bias identification scheme, we utilise the available data more efficiently using contrastive loss to facilitate debiasing. Contrastive learning effectively debiases data Zhang et al. (2022) y: Jung et al. (2023). but our ranking scheme refines pair selection, boosting debiasing performance and scalability to diverse and larle-scale datasets.
3 METHODOLOGY
In this section, we introduce a novel spuriosity-ranking framework, Sebra, designed to rank or or- der data points in decreasing order of spuriosity. At its core, the framework integrates self-guided weighting mechanisms into the standard Empirical Risk Minimization (ERM) using cross-entropy loss, creating an objective that prioritizes data points by their spuriousness. These self-guided weighting mechanisms guide ERM consistently along a pathway wherein attributes are learned sequentially in the increasing order of hardness. As a result, the order in which instances transi- tion from unlearned to learned naturally reflects the spuriosity of the data point. We demonstrate the effectiveness of this ranked dataset for debiasing within contrastive learning framework. Our approach is formalized in Section 3.1 with diagrammatic illustration in Fig.
3.1 SEBRA: SELF-GUIDED BIAS RANKING
Intuition behind Sebra: Following the example in Fig. consider the problem of classifying "cows" and "camels", where in the train set, cows are spuriously correlated with "green" back- grounds (such as grasslands) in "daylight", and camels are spuriously correlated with the "desert" background at "nighttime". Now, a model trained with ERM tends to classify the training datapoints first based on the background, i.e., cows on grasslands v.s. camels on deserts, which implies that is the easiest attribute to learn Nam et al. 2020). However, when samples exhibiting the background spurious correlation are dropped out from the training set, ERM learns to classify based the light- ing conditions, i.e., cows in daylight v.s. camels at nighttime. Finally, it is only when these are also dropped from the training set does the model finally capture the core attributes of cows and camels. Thus, when controlled with an appropriate steering mechanism (dropping of training samples corre- sponding to the already-learned spurious attribute), the sequence in which ERM learns data points follows high-spuriosity to low-spuriosity pathway, naturally providing spuriosity ranking. This fine-grained ordering can then be exploited through contrastive learning for debiasing.
Notations: Given train set X {(x;,, v:) with N data points across C' classes, we aim to rank them in decreasing order of spuriosity, i.e., if x; exhibits spurious cues than Ej, then p(z:) < p(z,), where p(z) is integer in [0, .N indicating the spuriosity rank of x. We use neural network fo with parameters @ to drive the ranking process. All proofs and derivations are provided in Appendix A
3

---

Published as a conference paper at ICLR 2025
Sebra
Contrastive DeBiasing
Training
posilive pair
nogative pair
Epocha
Bar Anitiaa (flaeg- Bis Annbacc
Ranking
(land)
Spriagted Truang
Spuriosity (Low Kank)
Spurosity (High Rinky
Ranked List
Figure 1: In each step of Self-guided bias ranking (Sebra), datapoints are upweighted with u, and then trained via ERM. Following this, we estimate for each sample to select them for subsequent training. Samples for which v; transitions from to 0 ranked at each step and eliminated from subsequent training. Any unranked samples are appended to the ranked list at the end of the training phase. In the mitigation phase, negative pairs are formed using samples with the same rank, while positive pairs are obtained using samples with higher rank than the reference samples.
Definition 1. For a sample X, let Fr be the set of all features types / attributes in x. An attribute space A is the exhaustive collection of all feature types across all x E X, i.e.,
A= U F
Definition 2 (Attribute Types and Spuriosity Ranking). A causal attribute ae E A is one that is responsible for determining the label y of datapoint 2, Vx X. A spurious attribute a, A non-causal attribute that does not determine the label y of any sample x E X, but co-occurs frequently with a, in X. We call the subspace of A covering all spurious features, As, the spuriosity basis. The spuriosity measure μ(x) on X is the fraction of spurious attributes in A, spanned by the feature-type set F& of a sample x E X. A spuriosity ranking p(z) is an ordering on X such that:
p(x:) < p(x,) Vi.j N p(x:) > u(z;)
In other words, samples with high levels spuriosity appear earlier in the ranking via p than samples with lower levels of spuriosity.
Assumption (Hardness-S) (Hardness-Spuriosstty Symmetry). The hardness of learning a sample, and its corre- sponding spuriosity measure, are symmetric to each other the harder it is to learn sample, the lower its spuriosity measure, and vice versa.
Implementation of p(z): We leverage the Hardness- Hardness-Spuriosity Symmetry to design form of self- guided bias identification, steering ERM consistently along high- sphrrispptistty low-spuriosity path- way.
This results in the rank of data point z; being the epoch in which its cross-entropy loss (or mono- tonically increasing function of it) drops below certain threshold, or in other words, its predicted probability Py (or a monotonically increasing function of it) of the correct class y exceeds a certain threshold, determined by hyperparameters, discussed below.
Fine-Grained Rank Resolution: Note that this specific criterion of ranking maps the N datapoint to M buckets, where M < N. In other words, multiple datapoints get mapped to the same rank bucket, they transition below the loss probability threshold together in the same iteration. However, in implementation,
also provide information about the spuriosity measure μ(z) for every data point I; through a weighting factor called u, x μ((z). Since p(x) is continuous- valued function, sorting in the decreasing order of u(x:) provides straightforward mechanism for
4

---

Published as a conference paper at ICLR 2025
collision resolution and obtaining fine- fine-grained ranking among data points that inhabit the same coarse-graine caanse-rraied rank bucket.
3.1.1 FORMULATION
Our ranking algorithm involves the following three key phases in each epoch:
We design the selection mechanism shift the model's focus new subgroup once a particular subgroup has been learned, for it to capture attribute types in order of increasing diffi- culty across iterations. The training is partitioned into samples that have been learned, i.e. easier samples with high spuriosity, and those that have not yet been learned, i.e. difficult samples with low spuriosity. The latter are carried forward for further updates based selection serves a dual purpose: it mitigates the influence of highly spurious features on the learning of the less spurious ones, and promotes the learning of attributes increasing order of difficulty. To implement this, we introduce binary selection variable u; for each point Zú, which identifies a minimal subset of data points that maximizes the cross-entropy loss:
! Selection:
via ERM. This segregation:
min max Σ (olfce(So(or)).u Aua}
of (fo((f( ys) responsible for selecting points that have not yet been learned (i.e., those with high Loe), while the -xx; prevents the trivial solution where all u/s are to 1, minimizing the number points that are selected in a single epoch.
Furthermore, to mitigate the influence of previously learned highly spurious attributes on subsequent learning, we condition the optimization on the state of vs in the previous iteration, i.e., on of as follows:
N of (of - Aul}
min max
and additionally restrict the domain of of to {0, of (instead of the general binary {0, 1)). where of 1, Vi E [1, NJ. This dynamic domain constraint follows from the order on X induced by the measure It effectively implements the inductive bias that points with higher bias would always be learned before points with fewer spurious features, leading to the result that once something has been learned and ranked (with their corresponding o/ set to 0), they need not be considered anymore. Note, however, that before vf ((=1)) becomes 0, (i.e., while is still 1), solving for the optimal og is still effectively general binary optimization problem {0, 1}.
2 Upweighting: Next, counteract the non-uniform gradient updates inherent in ERM, and to facilitate the ranking of points with high spuriosity before those with low spuriosity, we utilize the inductive bias that ERM has lower local risk, in any given iteration, for high spuriosity samples relative to their lower spuriosity counterparts (Assumption[]] D.. We do so by introducing a weighting variable us for each point z; proportional to the value of the spuriosity measure for that point, p(x:) as follows:
min max 8.
2 - (o/unCe((foor) v.) AuA}
This essentially results in the selection of those points with the lowest Lee and having them ranked before any of the other points with higher values of Log (more difficult-to-learn points, and hence, with fewer spurious features). In principle, following from Assumption u could be any monoton- ically decreasing function of LeE-
However, shortcut solution to minimizing u is to set all u, O. We prevent this shortcut by incor- porating the inductive bias that u, & j(x;)) into the optimization objective. For our specific case, we use u, = e (((ce(ff((::)))))
which has the effect that samples with high learnability spuriosity are upweighted by exponential function of their spuriosity, where ((x) is monotonically increasing function of x. We incorporate this constraint into the objective as follows:
min max 8,1
Σ o 1-1 (or - Auf +g(44))
(1)
5

---

Published as a conference paper at ICLR 2025
where 3 is a hhperparameter determining the weight of this constraint, and g(u)) is convex function meant to impose the constraint, whose form we uncover next.
Theorem (Hardness-Spuun Spuriissitt Conservation). Iff the spuriosity measure uf -=f((caa(s((r)).))) where t(x)) is a monotonically increasing function of x, the variable in Eq. ( across all values
of LCr(Jo (o(()) (11). yc), satisfies the following conservation law:
"uLCe([J (o((()(())) (xx), We) B(u, In us us) C,
such that un is the minimizer of the conserved function.
Intuition: Theorem arises as a consequence of the Hardness-Spuriosity Symmetry (Assump- tion D. which requires the measures of hardness (Lcr) and spuriosity (u4) to balance each other out. It states that, for the solution to Eq. (1) to have the form e=((Gs((((()))))) the quantity u; ( ys) + S(u, In uy u4) should be conserved, i.e., a constant, for all valid choices of ug. The implication is that the optimization on should be restricted to the space of those values that follow the conservation law. It formalizes the constraint that we need impose on in order to avoid the shortcut of setting all = 0.
In other words, the solution to in Eq. [ the minimum in the space of all values that satisfy the conservation law. Based on this, we use g(u)) B(u, In u4) in Eq. (I) to enforce the conservation criterion, and obtain our final objective, which we optimize for all three sets of variables 0, u, and v:
Cranking u0 = Σ of Lof - de!- - -Bu + Bu, In mz}
min max rraninng (0, u, v),
3 Ranking: Finally, samples with high spuriosity, i.e., the ones that have been already learned and dropped out of the training set in the selection phase, are appended to the rank list. Specifically, in every epoch i, we select those s; for which vi- 0 from the selection step, and append them rank list X rank (which is initially empty) as:
X Pankod = X
Tamked R
where || is the concatenation operator between that < Z, = u(xx)) 2 u(x;): Vz,, 2 E R and "x²) 1, 0z) O; Vx E R. Below we discuss how Sebra progressively orders training samples based on spuriosity by optimizing the variables associated with the above three phases.
lists, R is an ordered list of data points such
3.2 OPTIMIZATION
Based on formulation in Section 3.1.1 Sebra is parameterized by set of three variables, 0, u. and v. respectively corresponding to the Selection, Upweighting, and Ranking phases. Since they are all independent, one can optimize Cranking w7f each of the variables by keeping the others fixed. In each iteration, we first solve for of to select the points that have not yet been sufficiently learnt, compute their corresponding u,s, with which we upweight and minimize LCE wrt 0, (which non- zero for only those samples that have been selected by in the beginning of the iteration). and finally, set aside and rank samples whose U,S switched from to 0 in this iteration to avoid interfering with subsequent rankings.
Selection: We start by maximizing Cranking( (0, u, v) wri w. Note, here, that solving for of a discrete optimization problem, since of E {0. of 1--1) 1. Let k u.Cce(forr, vi) X. This partitions the search space into two halves, Le., 1 2 0 and ie < 0, follows:
max Cranking (v 0, max
of (u:Len(aauss) {} +Bu In ua
The optimal i; can be obtained in terms of the predicted probability of the correct class Py as:
0,
if Py > Periticals
(2)
1, otherwise.

---

Published as a conference paper at ICLR 2025
Once the of for data point F; has been set 0, consider it as learned, and by the design of the optimization objective, it does not influence the subsequent learning of the remaining points.
Upweighting and Training: We then solve for the minimization Lranking (0, 2, v) in
Lranking (u 0)
Σ of (oluffce u..) Au Buu + Bug In ui}
Since L(u 0, v) a convex function, can be minimized by equating derivative wvr a to 0 and solving the resulting equation (when of = 1), which gives us the minimizer of L(u 0, 0, as:
Py
(3)
We then optimize the parameters of the neural network 0 as follows via regular mini-batch stochastic gradient descent:
min Franking (0 u, 0r 0z V rrakknn 0z u, VoLCE((o(ss)) vi)
(4)
Note how the gradients of the (ce(fo(x;)) ye) wrr are upweighted compared to vanilla SGD, by an exponentially decreasing factor of LCu (fa (z4), Va), i.e., Py This helps the model to converge and rank datapoints with higher spuriosity before it moves on to those with lower levels of spuriosity.
Ranking: Finally, we set aside the samples for which ii ((-1)) 1, 0, and consider them as ranked by appending them to X in decreasing order of their corresponding u,s, thus allowing for fine-grained rank resolution. We provide the pseudocode for Sebra Algorithm TT Based on the ordering obtained, proceed in the next section, with formulating contrastive learning based objective for learning a metric space devoid of spurious correlations.
3.3 CONTRASTIVE DEBIASING
The ranking objective introduced in Section 3.1 generates class- wise ordering of data points in order decreasing spuriosity. This fine-grained ranking can be leveraged for efficient debias- ing. To demonstrate its effectiveness, contrastive learning has proven effective for debiasing (Zhang et al., 2022). the fine-grained bias characterization offered by Sebra enables the selection of more informative contrastive pairs. This approach surpasses traditional methods, which rely on simpler bias identification mechanisms, such as GCE partially trained ERM models. Additionally, contrastive learning- based approaches able the utilization of the entire training dataset, unlike methods such as DFR Kirichenko et al. 2023). which focus on the least spurious examples. Although the fine-grained bias identification generated by sebra could be integrated into other debiasing strategies like DFR, we opt for a con- trastive learning framework to showcase the full potential of Sebra.
adopt contrastive loss-based debiasing approach. While
Given a randomly sampled data point with rank r from class we sample another instance x, from the same class and rank r to form a negative pair (I,, x; This is motivated by the ranking objective, which assigns the same rank to data points with similar levels of spurious correlations. To form a positive pair, we pair 2; with an instance of higher rank than as such instances are less likely to share the same spurious features as 24. Using these contrastive pairs, we learn debiased representation by optimizing the contrastive loss while simultaneously updating the full model via cross-entropy loss. For classifier fo with encoder Senc, which maps a data point x to its representation Sene (x)) the training objective is:
Č(fo xz y) Lup (fenci x, v) (CEE(f((z
(5)
where 7 is weighting coefficient. The supervised contrastive loss with M positive pairs and N negative pairs is:
exp(2 zm/TT
Csup (2;; Senc)
E log EMe exp(= T 2 + En exp(= T
where T is the temperature coefficent, 2ih, 2n and 2 are the embeddings of positive, negative, and reference samples respectively.

---

We compute Published as a conference paper at ICLR 2025
4 EXPERIMENTAL SETUP
This section outlines the experimental framework for evaluating the effectiveness of the proposed spuriosity ranking and debiasing approach. We outline the datasets, evaluation metrics, and baselines used, with implementation details and hyperparameters in Appendix
Datasets: We evaluate the proposed ranking and debiasing strategy on one synthetic and four natural datasets with spurious correlations. The synthetic dataset, UrbanCars Li al. (023), focuses on car-type classification with spurious correlations involving the background and co- co-occurring objects. For natural datasets, we use CelebA Liu et al. (2015), which addresses spurious features like age and gender in predicting emotions (smiling/sad) as per Hu et al. (2023). These datasets contain multiple spurious correlations and bias annotations, enabling the definition of a ground truth rank order of spuriosity and evaluating method's 's effectiveness. Additionally, we test on two natural vision datasets, BAR (Nam et al., 2020) and ImageNet- 1K [Dng et al. (2009), and the natural language dataset MultiNLI to demonstrate scalability across domains. Sample images and dataset details in Appendix B
Evaluation Metries:
Quantitatively comparing spuriosity rankings is challenging due to the ab- sence of ground truth rankings. For datasets with bias annotations, such as Urban Cars and CelebA, where two biases are present (with Bias A being stronger than Bias B), define the ground truth ordering as follows: (Bias A aligned, Bias B aligned), (Bias A aligned, Bias B conflicting), (Bias A conflicting, Bias B aligned), (Bias A conflicting. Bias B conflicting). Kendall's 's tau correlation Kendall (1938) to compare our ranking with this ground truth.
For datasets without bias annotations (e.g., BAR), we propose Performance Disparity (PD):
PD Accuracy Bottom &s (Drest) Accuracy k (Dust)
(6)
PD measures ranking quality by assessing accuracy differences between models trained on highly vs. minimally spurious samples. High PD indicates effective separation of spuriosity levels.
To evaluate debiasing, we adopt and extend three UrbanCars metrics Li et al. (2023): BG Gap, CoObj Gap, and BG CoObj Gap, which quantify accuracy drops when spurious features unaligned. These metrics generalize to other datasets, e.g., Age Gap, Gender Gap, and Age + Gender Gap for CelebA. Avg. GAP aggregates robustness across shifts. For BAR, report test accuracy on bias-conflccting samples, following Li et al. (2022). Further metric details are in Appendix
We compare the performance of the proposed approach with a supervised approach Group DRO (Sagawa et al. 2020) and five popular unsupervised approaches ERM (Vapnik 1999). LIF (Nam et al. 2020). JTT (Liu et al. [202T), Debian (Li et al.) (0022) and DFR (Kirichenko et al., 2023). The supervised approaches assume the availability of shortcut labels for all the spurious attributes while the unsupervised methods have access only to target labels. Both classes of methods further assume access small supervised validation set for hyperparameter tuning.
Baselines:
4.1 RESULTS
In this section, we compare the performance of the proposed method with various baselines and datasets described in Section to demonstrate its effectiveness in spuriosity ranking and debiasing.
Ranking Evaluation. As observed in Table [ the proposed method produces superior ordering of data points as indicated by a higher value of Kendall's tau-b coefficient for both datasets. The inferior performance of Spuriosity Ranking (Moayeri et al.) 2023) despite human supervision could be attributed the fact that biased attributes like background encompass multiple sub-attributes like lighting. sky, terrain, etc, and different sub-attributes are captured by different neurons rather than one or few neurons. Since these concepts are distributed across multiple neurons they need not be contained in top-k activations Fig. 4 resulting in many of these attributes being not considered while sorting the data.
Debiasing Evaluation. As shown in Table our proposed Sebra outperforms all the unsuper- vised methods in simultaneously mitigating multiple biases, as evidenced by the lower average gap (Avg. GAP) metric across datasets. While Sebra may not always achieve the best performance
8

---

Published as a conference paper at ICLR 2025
individual Bias GAP metrics, this is due the whac-a-mole dilemma observed in previous meth- ods, where mitigating one bias attribute exceptionally well can amplify the other bias attribute.
This can result in a very low Bias GAP for one attribute, even though the model remains highly biased overall. Furthermore, the proposed method consistently sur- passes previous approaches. Addi- tionally, our method performs com- parably to prior single-bias unsu- pervised methods in single-bias set- tings, highlighting its effectiveness. An extended version of Table 2 is provided in Appendix D.2
Table 1:
Quantitative comparison Sebra with various base- lines. The results are shown in terms of Kendall's s 7 for Urban Cars and CelebA, and Performance Disparity (PD) for BAR.
Method
Urban Cars
CelebA
BAR
Metric
Kendall's T (t)
Kendall's (t)
PD (t)
Random Ordering ERM-based Ranking
0.02
-0.01
0.25
0.12
0.14
4.55
Spuriosity Ranking
0.40
0.38
28.88
Sebra (Ours)
0.85
0.69
32.32
Table 2: Performance comparison across UrbanCars, CelebA, BAR, ImageNet, and MultiNLI datasets. Sup.: Whether the model requires group spurious attribute annotations (X: not quired, v: required). I.D. Ace. measures performance without subpopulation shift, while Avg GAP does in its presence. All results reported mean (standard deviation).
Methods
Sup.
UrbanCars
CelebA
Aec. Acc. (1)
WG Acc. (1)
Avg GAP (1)
I.D. Acc. (1)
WG Acc. (1)
Avg (1)
Test Acc. (1)
Group DRO /
91.60 (1.23)
75.70
(1.35)
90.08 (0.70)
(1.9
-5.79 (1.63)
FRM
97.60 (0.86)
33.20 (0.86)
31.90
96.43 (0.13)
(1.0
(2.883
68.00 (0.43)
LIF
97.20 (2.40)
35.60 (2.40)
31.06
95. (0.35)
(2.0
-22.57 (1.26)
68.30 (0.97)
JIT
95.80 (1.45)
-20.50
(1.48)
-26.81
(0.28)
Debian
98.00 (0.89)
30.10 (0.89)
-31.40 (1.44)
96.28 (0.37)
41.1 (4.3)
-2..66 (0.54)
69.88 (2.92)
89.70
(2.61)
(3.27)
(1.25)
Sebra (Ours)
92.54 (2.10)
73.8 (3.28)
-1.557 (1.72)
88.61 (3.36)
65.3 (4.1)
-9.82 (3.06)
75.36 (2.23)
Method
Sup.
ImageNet-IK
MultiNLI
L.D. Ace. (f)
IN-W Gap (f)
IN-9 Gap (1)
IN-R Gap (?)
Carton Gap
WG. Acc (1)
LLE ERM
V
76.25
-6.18
-3.82
-54.89
+10
76.13
26.664
-5.53
-55.96
+40
66.8
LIF
70.26
-17.57
-8.10
-56,54
+40
63.6
JIT
75.64
-15.74
-6.75
-55.70
+32
69.1
Debian
74.05
-20.00
-7.29
-56.70
+30
Sebra (Ours)
74.89
-14.77
-3.15
-54.81
+25
72.3
4.2 ANALYSIS AND ABLATION STUDIE
In this section, present comprehensive set of analyses and ablation studies to provide deeper insights into the performance of Sebra. Specifically, we investigate how the training dynamics of model optimized using the proposed ranking objective differ from those of standard empirical risk minimization (ERM)-based model. This comparison elucidates how the proposed selection and weighting mechanisms modulate the ERM training dynamics facilitate spuriosity ranking. Furthermore, we conduct ablations on the various components of our framework to quantify their contributions the overall ranking quality. Additional ablation studies are provided in Appendix G
Analysis of Ranking Dynamics: In Section 3 we introduced Sebra, which integrates targeted modifications to ERM to systematically rank data points in the decreasing order of spuriosity. To rigorously assess the impact of these modifications, we conduct detailed analysis of the training dynamics under the Sebra objective compared to standard ERM. Specifically, we leverage the Ur- banCars dataset, which includes bias annotations, enabling a detailed evaluation of how spurious and intrinsic features are differentially learned across the two training paradigms. In Fig. 2 we plot the accuracy of three visual cues-object (e.g., car body type), background, and co-occurring objects-oo on the unbiased validation set by comparing the model's {urban, country) predictions to the corresponding labels. As shown in Fig. 2 (Left), both models initially prioritize the easiest bias attribute (background). However, as training progresses, the sebra objective induces more pro- nounced forgetting of learned attributes compared to ERM, likely due to the selection mechanism

---

Published as a conference paper at ICLR 2025
Background Bias
Co-Oecurrng Object Bias
Cur Type Core Attribute
75
Azcuracy %
70
70
65
50
60
55
0
9
12
15
18
15
01
3
6
9
12
15
18
Epochs
Epochs
Epocks
Ouzs ERM
Figure 2: Training dynamics of Sebra and ERM monitored in terms of accuracies of background bias (left), co-occurring object bias (center), and core attribute (right).
through i that refocuses the model's attention on other subgroups. The aggressive forgetting un- der the sebra framework overcomes the slowdown in the convergence of difficult attributes in the presence of simpler correlated features (Qiu et al., 2024), as evidenced by the higher peaks for both target and co-occurring object attributes. Another interesting observation is that, for relatively difficult bias attributes, such as co-occurring objects, the naive ERM formulation struggles to dif- ferentiate them from core attributes, as indicated by a simultaneous increase in accuracies Fig. 2 (center and right). Sebra effectively addresses this challenge by leveraging the upweighting factor U4, which amplifies the influence of highly spurious instances, thereby facilitating the progressive learning of these attributes. Additionally, the self-guided mechanism driven by enhances overall performance, as demonstrated by the ranking improvements shown in Table The non-overlapping peaks indicate that instances with higher prevalence of respective attributes are assigned differ- ent ranks. Therefore, the ranking objective of sebra leads to segl-segeeeated sequential learning of different shorteuts in the decreasing order of spuriosity. This analysis confirms intuition and provides empirical evidence that the sebra efficiently ranks data in decreasing order of spuriosity.
Effect of Loss Components: To evaluate the con- tribution of various loss components, we conduct an ablation study by systematically removing compo- nents and measuring their impact on the quality of the resulting rankings using Kendall's coefficient. The results of this analysis are shown in Table 3 When using only LeE, corresponding to standard ERM training, we observe that ERM cannot alone rank data points effectively. To address this, we define proxy ranking based on the epoch which the predicted probability of the target attribute surpasses fixed threshold. As shown in Table 3 the model trained with naive cross-entropy loss exhibits a low correlation with the ground truth rank- ing, indicated by the low Kendall's This suggests that naive cross-entropy fails to capture the underlying spuriosity of the data. The slightly positive value of r likely reflects ERM's inherent, albeit weak, capacity asynchronously learn different attributes. When U; is added to the objective function, the ranking quality improves significantly. This suggests that ERM's poor bias ranking per- formance may be due interference from the easiest attributes when learning more complex ones. By incorporating both i/ and un into the training objective, the ranking quality further improves to 0.85, underscoring their importance enhancing performance.
Table 3: Ablation study of different compo- nents used in Sebra.
(c)(
Comiia (m.)
fanaii (14)
Kendall's r (?)
0.12
0.79
0.85 (Sebra)
CONCLUSION AND FUTURE WORKS
We propose a novel debiasing strategy, Sebra, based on a fine-grained ranking of data points in decreasing order of spuriosity, obtained without any human supervision. Sebra facilitates spuriosity ranking by modulating the training dynamics of a simple ERM model to iteratively focus on highly spurious data points while simultaneously excluding already ranked datapoints from the ranking process. We further demonstrate how this fine- -rained bias ordering enhances bias mitigation, by considering contrastive learning- based approach an exemplar various datasets. Future work could explore bias mitigation strategies tailored to Sebra's 's rankings, refine the ranking scheme, and develop unsupervised metrics for evaluating spuriosity rankings.
10

---

Published as a conference paper at ICLR 2025
6 ACKNOWLEDGMENTS
Adarsh Kappiyath thanks Silpa Vadakkeeveetil Sreelatha for invaluable conversations and feedback throughout this project. Additionally, we would like to thank the University of Surrey for providing the valuable computing infrastructure needed for this project.
REFERENCES
C. Chann,, G. Adam, and A. Goldenberg. Towards robust classification model by counterfactual and invariant data generation. In CVPR, 2021.
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In ICPR, 2009.
Robert Geirhos, Patricia Rubisch, Claudio Michaelis, Matthias Bethge, Felix A. Wichmann, and Wieland Brendel. Imagenet- traiiee CNNs are biased towards texture; increasing shape bias im- proves accuracy and robustness. In ICLR, 2019.
Rui Hu, Yahan Tu, and Jitao Sang. Echoes: Unsupervised debiasing via pseudo-biat labeling in an echo chamber. In Proceedings of the 31st ACM International Conference on Multimedia. ACMM, 2023.
Minyoung Huh, Hossein Mobahi, Richard Zhang, Brian Cheung, Pulkit Agrawal, and Phillip Isola. The low-rank simplicity bias in deep networks. TMLR, 2023.
Badr Youbi Idrissi, Martin Arjovsky, Mohammad Pezeshki, and David Lopez-Paz. Simple data balancing achieves competitive vorst-group-aceuracyy In CLeaR, 2022.
Yconsung Jung, Hajin Shim, June Yong Yang, and Eunho Yang. Fighting fire with fire: Contrastive debiasing without bias-free data via generative bias- maaatransformmtion. In ICML. PMLR, 2023.
Maurice G Kendall. A new measure of rank correlation. Biometrika, 1938.
Polina Kirichenko, Pavel Izmailov, and Andrew Gordon Wilson. Last layer re-training sufficient for robustness to spurious correlations. In The Eleventh International Conference on Learning Representations, 2023. URL (etphttsss. http://pprreviveeeeeeee net
Pang Wei Koh, Shiori Sagawa, Henrik Marklund, Sang Michael Xie, Marvin Zhang, Akshay Balsub- ramani, Weihua Hu, Michihiro Yasunaga, Richard Lanas Phillips, Irena Gao, Tony Lee, Btienne David, Ian Stavness, Wei Guo, Berton Earnshaw, Imran Haque, Sara M Beery. Jure Leskovec, Anshul Kundaje, Emma Pierson, Sergey Levine, Chelsea Finn, and Percy Liang. Wilds: bench- mark of in-the-wild distribution shifts. In ICML, 2021.
Jungsoo Lee, Eungyeup Kim, Juyoung Lee, Jihyeon Lee, and Jaegul Choo. Learning representation via disentangled feature augmentation. In NeurIPS, 2021.
Zhiheng Li, Anthony Hoogs, and Chenliang Xu. Discover and Mitigate Unknown Biases with Debiasing Alternate Networks. In The European Conference on Computer Vision (ECCV), 2022.
Zhiheng Li, Ivan Evtimov, Albert Gordo, Caner Hazirbas, Tal Hassner, Cristian Canton Ferrer, Chenliang Xu, and Mark Ibrahim. A whac-a-mole dilemma: Shortcuts come in multiples where mitigating one amplifies others. In CVPR, 2023.
Yong Lin, Shengyu Zhu, Lu Tan, and Peng Cui. Zin: When and how to learn invariance without environment partition? In NeurlPS, 2022.
Evan Z Liu, Behzad Haghgoo, Annie S Chen, Aditi Raghunathan, Pang Wei Koh, Shiori Sagawa, Percy Liang, and Chelsea Finn. Just train twice: Improving group robustness without training group information. ICML. PMLR, 2021.
Ziwei Liu, Ping Luo. Xiaogang Wang, and Xiaoou Tang. Deep learning face attributes in the wild. In Proceedings of International Conference on Computer Vision (ICCV), 2015.
11

---

Published as a conference paper at ICLR 2025
Mazda Moayeri, Wenxiao Wang, Sahil Singla, and Soheil Feizi. Spuriosity rankings: Sorting data to measure and mitigate biases. NeurlPS, 2023.
Junhyun Nam, Hyuntak Cha, Sungsoo Ahn, Jaeho Lee, and Jinwoo Shin. Learning from failure: Training debiased classifier from biased classifier. In Advances in Neural Information Processing Systems, 2020.
Geon Yeong Park, Sangmin Lee, Sang Wan Lee, and Jong Chul Ye. Training debiased subnetworks with contrastive weight pruning. In CVPR, 2023.
GuanWen Qiu, Da Kuang, and Surbhi Goel. Complexity matters: Feature learning in the presence of spurious correlations. In ICML, 2024.
Shiori Sagawa, Pang Wei Koh*, Tatsunori B. Hashimoto, and Percy Liang. Distributionally robust
neural networks. In ICLR, 2020.
Sahil Singla and Soheil Feizi. Salient imagenet: How to discover spurious features in deep learning? In ICLR, 2022.
Nimit Sohoni, Jared Dunnmon, Geoffrey Angus, Albert Gu, and Christopher Ré. No subclass left behind: Fine- tine-graieed robustness in coarse-grained classification problems. In NeurlPS, 2020.
Rishabh Tiwari and Pradeep Shenoy. Overcoming simplicity bias in deep networks using feature sieve. In ICML, 2023.
Vladimir Vapnik. An overview of statistical learning theory. IEEE Trans. Neural Nenworks, 1999.
Adina Williams, Nikita Nangia, and Samuel Bowman. A broad-coverage challenge corpus for sen- tence understanding through inference. In Marilyn Walker, Heng Ji, and Amanda Stent (eds.), Proceedings of the 2018 Conference of the North American Chapter of the Association for Com- putational Linguistics: Human Language Technologies, Volume (Long Papers). New Orleans, Louisiana, June 2018. Association for Computational Linguistics. doi: 10.18653/11/N18.. 1101. URL https: //a 1aw.tholoo..orr//111111111 1101/
Yu Yang, Eric Gan, Gintare Karolina Dziugaite, and Baharan Mirzasoleiman. Identifying spurious biases early in training through the lens of simplicity bias. In Proceedings of The 27th Interna- tional Conference on Artificial Intelligence and Statistics. PMLR, 2024.
Yuzhe Yang, Haoran Zhang, Dina Katabi, and Marzyeh Ghassemi. Change is hard: A closer look at subpopulation shift. JCML, 2023.
Michael Zhang, Nimit S Sohoni, Hongyang R Zhang, Chelsea Finn, and Christopher Re. Correct- n-contrast: contrastive approach for improving robustness to spurious correlations. In ICML, 2022.
12

---

Published as a conference paper at ICLR 2025
A PROOFS AND DERIVATIONS
A.I PROOF OF THEOREM 1
Proof. We start by first proving the case when in = e e-((Ce(e..v.99) leads to the conservation law, u.LCr(us, Ve,, 01, + 3(u; In Us)
L.., un e -(((en(z2.31000
min (T Vs. Xu; + Bg(u,) = (a ye. 0) + 3g' (u:) -
g((,,) CC Vi. un = ( Vs. 0))
Now, we know that un e ((Cnlzz::)))) Considering $2, we have, (x))- = E² g((x) In Then,
Inu," (cr(Is: yr. y1 = fimu" du, - / CcE(dt. v.. du, u,Len(Ii. Vr. 0) B(u us - uz) Xuh,
-1
since Xo was the constant that vanished under the derivative. This proves the statement in the direction uf ===((c((x..0000 = u,CcE(I,, Vr. 0) + Buu, In u.)
Next, we prove the statement in the other direction, i.., when minimizing the conserved function leads to the exponentially decreasing characteristic of u²- Solving for the minimizer of the conser- vation expression, we get:
u min u, Cee Tz: Us. 8) + B(u; In uz) - Ax = LEE(Fs. Vi, 0) 3 In = 0
= u4 = Lcu (CE(E,,
1
= u = (eal(r) = (((Cos(tt.210))
-
where f Jz. This proves the statement in the direction u, u, (2,, Vi 0) + B(u, In u4 u.)
ui = = =c(((cc(ee.).0)) and completes the proof of the theorem.
L
A.2 SOLUTION FOR U4
0
Cranking (u 02 =0 = CfCcn Vi. 3 B[1 + In u;] =
Du:
1 Impp Inu Inpy 1/3 uj p 1/3
Py 3 In
In
3
A.3 SOLUTION FOR
Case 1 (k > 0): When & 2 0, the optimal solution is o/" 1, ensuring L(v 0, 0,u) 2 0. Otherwise, Cranking 1 0, u) = 0, which is always less than or equal to when of = 1. Thus, of = is the maximizer of (1 (v 0, u) when k 2 0.
Below, we derive the condition for optimality in terms of the predicted probability of the correct class Py (this applies only when of- 1):
UzLoel yı, 0) A
In I -X Py Se- A/m4
Case 2 (Is < 0): When < 0, the optimal solution is rs = 0. If of = 0. Cranking (v 8, u) = u would be negative. Thus, of maximizes Cranking (v 0, in this case. Similarly to Case derive the condition for optimality when & < 0 in terms of the predicted probability of the correct class py:
Py > N/u.
13

---

Published as a conference paper at ICLR 2025
We consider the inqquality Py e>ppy where À and 3 are constants, and aim to rewrite the relationship in an analytically solvable form. Taking the natural logarithm of both sides of the inequality, we obtain
In(py)
(7)
1/8
Py
By multiplying through by py (valid for > 0), this simplifies to
p In(p)) -
(8)
To simplify further, introduce the substitution into the inequality gives
1/2
which implies Pv 29. Substituting
A
<ln(2)
8
At this point, we solve for explicitly. The equation z II((2) = C. (where C= is well-known form that can be solved using the Lambert W-function. The Lambert W-function defined as the inverse of yeu z, such that (z)). Rewriting In(z) in exponential form, obtain
In(z) W
(10)
Exponentiating both sides gives the explicit solution for
= c (-
(11)
1/8
Returning the original substitution 2 Py we substitute back express Py in terms of the Lambert W-function:
p. e w-
(12)
Finally, raising both sides to the power B gives the solution for Py:
Py = (-x())))²²
(13)
This expression represents the critical value of where the equality Py = e." N/p/// holds. Thus, the inequality Py > -//2// can now be interpreted as
Pg (c)(-
(14)
The Lambert W-function provides explicit solution for equations where variable appears both inside and outside of a logarithmic or exponential function. This result establishes the threshold Peritical- defined
Peritical = (e
(15)
which represents the critical value of Py where equality holds. Due to the monotonically decreasing nature of exponential, the inequality
Py PW
can thus be rewritten as:
Py Peritiall where Peritical depends only on the constants À and through the Lambert W-function. This formulation establishes that for the inequality > e -//p/// to hold, must strictly exceed the threshold Peritical-
(16)
1/0
14

---

Published as a conference paper at ICLR 2025
B DATASETS
We evaluate the proposed method across three distinct datasets, each designed to explore different facets of bias and debiasing techniques. Below, we provide a suceinct overview of each dataset:
UrbanCars Li et al. (2023): This synthetic dataset purposefully crafted to investigate debiasing methodologies amidss multiple spurious correlations. Comprising two classes UrbanCar and Country Car each class encompasses 4000 samples. The dataset is char- acterized by two biased attributes: Background and Co-Occurring Object. UrbanCars fea- ture city-like backgrounds with co-occurring objects such as traffic signs and fire hydrants, while Country Cars are set against rural backgrounds, predominantly featuring animals. UrbanCars is publicly available on Kaggle.
2. CelebA: A versatile dataset featuring celebrity faces alongside 40 binary attributes. We focus on the 'smile' attribute as the target, with biases introduced by age and gender. This configuration was introduced in Hu et al. (2023), and we employ their open-source code to obtain the data.
3. Biased Action Recognition (BAR) Nam et al. (2020): The Biased Action Recognition (BAR) dataset contains real-world images categorized into six action classes, each bi- ased towards particular locations. The dataset includes six prevalent action-location pairs: Climbing on Rock Wall, Diving underwater, Fishing
Water Surface, Racing on Paved Track, Throwing on Playing Field, and Vaulting into the Sky. The testing set is composed exclusively of samples with conflicting biases. Therefore, achieving higher accuracy on this set signifies improved debiasing performance.
4. ImageNet-IK Deng et al. 2009): ImageNet- 1 a large-scale dataset consisting of over million high-resolution images categorized into 1,000 distinct object classes. The dataset includes a diverse range of real-world objects across various categories such as animals, vehicles, and everyday items, making it a benchmark for visual recognition tasks. ImageNet-IK is widely used for training and evaluating deep learning models, and achiev- ing high accuracy on this dataset demonstrates strong generalization and recognition capa- bilities, with an emphasis on overcoming challenges posed by large-scale, varied data.
5. MultiLII Williams et al. (018):: The MultiLI (Multi-Genre Natural Language Infer- ence) dataset is large-scale benchmark for evaluating models on the task of natural lan- guage inference (NLI). It consists of 433k sentence pairs drawn from wide variety of genres, including government, fiction, and telephone conversations, among others. Each pair is annotated with one of three labels: entailment, contradiction, neutral. MultiNLI is designed to test a model's ability to generalize across diverse linguistic contexts, and achieving high performance this dataset indicates model's l robustness in understanding complex sentence relationships and overcoming genre-specific biases natural language processing tasks.
15

---

Published as a conference paper at ICLR 2025
4.25%
0.25%
90,25%
0.25%
Santing
Urtan
Nat--ralinng
Country Car
BiasA unalign BinsB align-G)
BiasA align-Gi
BiasA align BiasB unalign-G;
BiasA & B unalign-G,
Climbing
Diving
Fiehing
Racing
Throwing
Vaulting
Figure 3: Dataset samples: Images from various datasets with multiple spurious correlations used in experiments are shown below. For CelebA and UrbanCars dataset each column depicts multiple groups categorised based on biased features, as well as their proportions in the training set, each row displays samples from various classes. Images at the bottom demonstrates samples from BAR dataset from 6 classes. The images with red border lines belong to BAR evaluation set, and others belong to BAR training set.
C BASELINES
We evaluate the proposed method against series of unsupervised and supervised bias mitigation techniques. Below, we provide concise overview of each method:
GroupDRO: Sagawa al. (2200) supervised bias mitigation technique leveraging group labels to identify and mitigate biases across various groups in the training data. The objec- tive minimize the worst group accuracy across the identified groups.
2. ERM Vapnik (999): Empirical Risk Minimization, employing cross-entropy loss and lz regularization.
3. Learning from Failure (LfF): Nam et al. 2020) This approach utilizes the Generalized Cross-Entee Easss-y (GCE) loss derive a bias-only model. Subsequently, learns debiased model by reweighting the bias-conflicting points to learn debiased model.
4. JTT: Liu et al. (2021) This method uses ERM model trained for few epochs and identifies the misclassi iisscaasffiiaiionss obtained by the model as bias conflicting samples and is upweighted for debiased learning.
5. Debian: Li et al. (2022) Introducing a novel bias identification scheme relying the equal opportunity violation criteria, followed by bias mitigation strategies.
6. DFR: (Kirichenko al. 2023) demonstrates that ERM model captures non-spurious at- tributes even when trained with biased training data and thus simple last layer retraining with unbiased data is sufficient for debiasing.
16

---

Published as a conference paper at ICLR 2025
D RESULTS
D.I QUALITATIVE RESULTS
Darkness
Pointy Structures
Text
Stone Buidings
Dark Sky
Figure 4: Top spurious concepts discovered using Spuriosity rankings introduced Moayeri et al. 2023). As observed, the identified neurons capture only subset of features corresponding to the spurious attribute 'background', ; thus, ranking relying on top-k highly activating neurons would only rely partial characteristics of spurious features.
Class 0 : UrbanCars
Top Ranked
Bottom Ranked
Class
Country Cars
Top Ranked
Bottom Ranked
Figure 5: Qualitative Analysis UrbanCars Dataset: Examples of top-ranked (high-spurious) and
bottom-rankedd ranked samples ranked by Sebra, showcasing a range of samples from both the classes.
Low Spuriosity
High Spuriosity
Figure 6: Qualitative Analysis on the ImageNet-Ile 1K Dataset. Sebra identifies spurious correlations in three classes. In the "Carton" class, Sebra detects watermark shortcuts, while in the 'Shark' and 'Jackfruit" classes, deep-sea and tree backgrounds are flagged as spurious features.
17

---

Published as a conference paper at ICLR 2025
Diving
Top Ranked
Bottom Ranked
Throwing
Top Ranked
Bottom Ranked
Racing
Top Ranked
Bottom Ranked
Pole Vaulting
Top Ranked
Bottom Ranked
Climbing
Top Ranked
Ranked
Fishing
Top Ranked
Bottom Ranked
Figure 7: Qualitative Analysis on BAR Dataset: Examples of top-ranked and bottom-ranked samples as ranked by Sebra, showcasing a range of samples across different classes.
18

---

Published as a conference paper at ICLR 2025
D.2 QUANTITATIVE RESULTS
Table 4: Performance comparison on the UrbanCars dataset. Sup.: Whether the model requires group spurious attribute annotations in advance (X: not required, v: required). The best- performing results among unsupervised methods are marked in bold.The The baseline results are taken
from Li 2023
Methods
Sup.
L.D. Acc. (f)
BG GAP (t)
CoObj GAP (f)
BG + CoObj GAP (f)
Group DRO
91.60(1.23)
-10,90 (1.08)
(3.600
-16.40 (2.80)
ERM
97.60 (0.86)
-15.30 (1.35)
-11.20 (5.07)
-69.0 (5.34)
97.20 (2.40)
-11.60 (1.23)
-18.40 (4.01)
-63.20 (2.21)
JTT
95.80 (1.45)
-8.10 (L.8)
-13.30 (4.28)
-40.10 (2.48)
Debian
X
98.00 (0.89)
-14,90 (1.08)
-10.50 (1.47)
-69.00 (1.78)
DIR
89.70 (1.21)
-10.70 (1.85)
-6.90 (2.56)
-45.20 (3,42)
Sebra
92.54 (2.10)
-6.54 (1.38)
-7.84 (1.38)
-17.34 (2.40)
Table 5: Performance comparison on the CelebA and BAR. Sup. indicates whether the method is supervised for bias (V) not (X). The best results among unsupervised methods marked in bold.
Methods
Sup.
CelebA
BAR
Ace (1)
Gender GAP (1)
Age G1)
Gender+Age GAP (T)
Test Acc. (?)
Group DRO
90.08 (0.70)
-5.67 (2.23)
(2.4
-9.11 (3.34)
(0.13)
LIF
95.12 (0 (0.35)
-24.14 28)
(1.25
-4.326(1.32)
68.30 (0.97)
JTT
91.86 (1.48)
-31.07
-3.51
-45.85
68.14
Debian
96.28 (0.37)
(1.263
(1.65
-42.41 (0.49)
69.88 (2.92)
DFR
(1.12
(1.316
-17.36 (3.23)
-2..96
69.22 (1.25)
(2.64)
75.36
E METRICS
This section provides detailed mathematical description the evaluation metries used throughout the paper.
In-Domain Accuracy (L.D. Ace): This metric represents the weighted average accuracy across groups, where the weights determined by the correlation strength (i.e., fre- quency) of each group in the training data. It is designed to assess model performance under conditions where group distribution remains consistent with the training set.
I.D. Acc
Σ W/ Acc;,
(17)
where denotes the weight of group i, and Acc; represents the accuracy for group i.
2. Bias GAP:
This metric captures the difference between In-Domain Accuracy (I.D, Acc) and the accuracy groups where the specific bias is less pronounced. It quantifies the model's performance drop when tested on groups that diverge from the biases present in the training data.
Bias GAP = I.D. Acc ACCuncummon:
(18)
where AcCuncommon represents the accuracy groups with less prevalent bias.
3. Kendall's Tau Coefficient: Kendall's Tau a non-parametric statistic that assesses the ordinal association between two variables. It ranges from -1 (perfect negative correlation) to (perfect positive correlation), with 0 indicating no correlation. Particularly suitable for ranked data, Kendall's Tau is more robust than Pearson's 's correlation when the data distri- bution is non-nonne non-normal or the relationship between variables is non-linear. The coefficient is computed by comparing the number of concordant and discordant pairs in the dataset.
19

---

Published as a conference paper at ICLR 2025
F APPLICATIONS BEYOND DEBIASING
This section highlights the wide-ranging applications of Sebra, demonstrating its utility beyond just debiasing tasks. In particular, we demonstrate that sebra faciliates additional utilities beyond debiasing like outlier and noise detections well discovery unknown biases.
F.1 OUTLIER AND NOISE DETECTION
Training datasets often consist of samples from various sources, annotated by individuals with dif- fering expertise and background knowledge. As result, it is common for such datasets to contain outliers mislabeled instances. When these corrupted samples are incorporated into the training process, they can negatively impact model performance, especially if the label noise is prevalent or severe.
Sebra's proposed ranking scheme provides natural mechanism to address these issues. Specifi- cally, in datasets containing outliers mislabeled samples, Sebra assigns the highest ranks these corrupted instances. This ranking system makes it easy identify and segregate noisy data. For in- stance, Living17 dataset, we demonstrate how Sebra effectively ranks samples across several classes, identifying both the highest- and lowest-ranked samples, as shown in Fig. 8
The ability to segregate noisy data facilitates an efficient filtration process, which mitigates the negative impact of corrupted samples model training. This leads to enhanced model robustness and ensures better performance in downstream tasks by focusing on cleaner, more reliable data.
Class Salmander
Low Ranked
High Ranked
Class: : Parrot
High Ranked
Class Butterily
Low Ranked
High Ranked
Class Tuntl
Ranked
High
Figure 8: Qualitative Analysis the Living17 Dataset. Examples of the least- and highest-ranked samples from select classes the Living17 dataset. Sebra assigns higher ranks to mislabeled and outlier samples, enabling their identification and removal in downstream task processing.
F.2 DISCOVERY OF UNKNOWN BIASES
Datasets often contain various biases, some of which are difficult to identify due to their inherent nature. The Spuriosity ranking generated by Sebra facilitates the discovery of such previously un-
20

---

Published as a conference paper at ICLR 2025
known biases. Through qualitative inspection of the various segregations identified by Sebra, uncovered two previously unknown biases in widely used synthetic datasets for debiasing studies, shown in Fig. | Specifically, in the UrbanCars dataset, observed that the color and appear- ance the car are spuriously correlated with the "UrbanCars' category. Similarly, in the Landbirds dataset, spurious correlation between the color of the birds and the 'Landbirds' category iden- tified. These subgroups were flagged as relatively high in spuriosity by Sebra, revealing biases that were previously apparent.
Unknown Discoverad: Color! Sporty
Cnknown Discovared: Color
Unknown Blas Discovered: Color
Figure 9: Discovery of Unknown Biases in Synthetic Datasets. Examples of spurious correlations uncovered by Sebra in two widely used synthetic datasets for debiasing studies. the Urban- Cars dataset, spurious correlation between color and the 'UrbanCars' category was identified. Similarly, in the Landbirds dataset, a spurious correlation between bird color and the 'Landbirds' category was observed. Sebra's Spuriosity ranking facilitated the identification of these previously unknown biases, aiding in their detection and mitigation.
Algorithm 1: Pseudocode of Sebra
Input: A neural network fe. Xurin {(r, 34) ) where yi E (1,...,
C), maximum rank and upweighting and selection hyperparameters and X, respectively.
wyteecioonyypeepaaaaaii Output: Xnaked = {(Xe, p(x,))
Initialize t= 0
while < R do
Obtain Pv = fo(2, y) compute of using equation 3
> Up-weighting
Update the model parameters trained with upweighted points using equation
DTraining
Compute vf using equation]) 2 to select samples for subsequent training
Deelectio
if of 0 and o!-
then
L p(T.) Inerement = f++1
>Rankiig
G ADDITIONAL ABLATION STUDIES AND EMPIRICAL VALIDATIONS
G.1 EFFECT OF VARYING
The Sebra objective introduced in Section 3.1 involves two key hyperparameters. À and B. In this section, we investigate the sensitivity of the proposed ranking scheme to different values of 3. Specifically, we plot the variation of Kendall's 7 metric as a function of increasing 3 values in Fig. 10 As shown, the ranking quality demonstrates an almost linear decreasing trend as B in- creases, suggesting that smaller values of B are preferable for optimal performance. This behavior simplifies the typerparameter search, as the optimal 3 appears to lie within the range (0, 1), reducing the computational cost associated with hyperparameter tuning.
21

---

Published as a conference paper at ICLR 2025
9.44
902
Figure 10: Sensitivity of ranking quality to /
G.2 EMPIRICAL VALIDATION OF SEADNESS-SPPRROSIT SYMMETRY
Lin et al. (2022) et al. have theoretically demonstrated that unsupervised bias discovery is funda- mentally impossible without the incorporation of additional inductive biases meta-data. In this work, we leverage the concept continuous measure of spuriosity. This symmetry has been explored in prior studies, such as Nam iet al. (2020); Qiu et al. (2024). Here, we refine and formalize this concept, proposing a method to quantitatively assess spuriosity.
Hardness-S Sppuriiotty Symmetry as an inductive bias to derive
To empirically validate this assumption, we present a plot of the training loss for samples with and without spurious correlations, generated by training model using Empirical Risk Minimization (ERM) on the Urbancars, Celeb A and BAR dataset. As shown in Fig. 11 samples containing spuri- correlations (i.e., bias attributes) exhibit a rapid decrease in loss, whereas non-spurious samples, which lack such shortcut attributes, show a much slower decline loss. This discrepancy provides empirical support for our hypothesis that the difficulty of learning from a sample inversely related to its spuriosity.
be
1055 tpechs for Offerent Sample bpes
Lase as toace for fiseet Semple lppe
Non Spuriuu Semple
Surin
MonSpriuss
Squiices
Spalbat
Sp. Sp
1.
64-
Eacchs
tpabs
(a) Urban Cars
(b) CelebA
(c) BAR
Figure 11: Empirical Validation of Hardness-Spuriosse Symmetry: Training loss vs. epochs for samples with and without spurious correlations on the UrbanCars, CelebA and BAR dataset. Sam- ples with spurious correlations demonstrate rapid decrease in loss compared to samples without such correlations, suggesting that higher spuriosity corresponds to easier learning.
H COMPUTATIONAL COST
The computational cost of debiasing via self-guided bias ranking can divided into two compo- nents: the cost of spuriosity ranking with Sebra and the cost of contrastive debiasing. Sebra's 's low computational complexity arises from the closed-form solution for the weighting variable and the progressive removal of data points during ranking, which accelerates the process. However, the cost of bias mitigation using contrastive learning is higher due to its reliance on the full diversity of data rather than limited subset. To quantify Sebra's computational complexity, breakdown of the time (in wall-clock minutes) required for both the ranking and debiasing phases of our proposed framework in Table For comparison, also include the corresponding time for
provide detailed
22

---

Published as a conference paper at ICLR 2025
ERM on the UrbanCars dataset. All the measurements were done using single Nvidia RTX 3090 GPU.
Method
Time
Sebra Ranking
minutes
Sebra Contrastive Debiasing (Bias Mitigation)
52 minutes
ERM (Empirical Risk Minimization)
32 minutes
Table 6: Time breakdown of Sebra's ranking and bias mitigation steps compared to ERM.
I LIMITATIONS
While Sebra demonstrates strong bias ranking capabilities and superior debiasing performance, it remains sensitive to label noise. Another limitation arises datasets with multiple sub-population shifts, such as class imbalance. In such cases, the model may overemphasize a particular class, sulting in an increasingly unbalanced dataset during training. This imbalance can lead to learning collapse and a failure in ranking performance. Extending Sebra to handle these more complex sce- narios, such as bias ranking in the presence of multiple sub-population shifts, could be promising direction for future research.
J REPRODUCIBILITY
In this section, we outline the hyperparameters used in our proposed approach across various datasets. The optimal hyperparameters obtained for various datasets are summarised in Table 7 All experiments were conducted using a single RTX 3090 GPU. To facilitate reproducibility, we intend to release a user-friendly version of the code publicly along with the pre-trained models post- acceptance. provide implementation details and hyperparameters to facilitate reproducibility in Table 7 All the datasets used are publicly available or can be generated with publicly available resources.
Implementation Details: We use the same architectures and experimental setups as previous studies Li al. (2022); Nam et al. (2020) to ensure fair comparisons. Specifically, we utilize ResNet-50 for the UrbanCars, and ResNet- 18 for CelebA and BAR datasets. The optimal hyperparameters selected based on experiments conducted on a small validation set with bias annotations, following the approach in Liu et al.) (202T): Li et al. (202)) for CelebA and UrbanCars. For BAR, bias annotations are used, even during validation and validation set is obtained by random split of training set in 80:20 ratio. To ensure statistical robustness, perform four independent trials with different random seeds and report the mean and standard deviation of the results.
Optimal parameerrss for the BAR, UrbanCars, CelebA, and ImageNet datasets deter- mined through Syper--paraeed search.
Table 7:
Parameter
UrbanCars
BAR
CelebA
ImageNet
Learning Rate (LR) batch Size optimiser
1.0 x 10
1.0 10
1.0 x 10 3
1.0 10 10-3
128
64
64
512
SGD
Adam
SGD
SGD
momentum
0.1
0.8
0.5
weight decay
0.001
0
1.0 x 10-4
1.0 x 10-4
Peritical
0.75
0.75
0.7
0.1
B
1.25
1.42
1.25
1.42
?
0.5
0.5
0.05
0.15
0.05
0.1
23


Reviews:
[{"review_id": "9NSfr6dt32", "reviewer": "Reviewer_GZFw", "summary": "This paper aims to rank samples by the degree of spuriosity and debias accordingly. The authors propose an unsupervised framework, Sebra, which ranks samples automatically by spuriosity in their classes. An important assumption of this paper is the negative correlation between spuriosity and the ease of learning a sample via ERM. Further, they dynamically steer ERM to satisfy this assumption during iteration training and sequentially learn samples in increasing order of difficulty. Experiment results show superiority of Sebra over other unsupervised debiasing methods.", "questions": "- Can this method be extended to semi-supervised, especially when encountering new classes?", "limitations": "", "rating": 6, "confidence": 2, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "- The motivation of this paper is simple and clear: biased samples are harder to learn.\n- The design of methods is well described and easy to understand.\n- Performance improvement is significant.", "weaknesses": "- The method parts are a little too long and not eye-catching, written in a way that follows the authors' thought processes. I would prefer it to be more concise and add more examples or illustration figures.\n\n- Some sentences are way too long to understand, e.g., lines 340-343. Also, plz pay attention to distinguishing \\cite and \\citep.\n\n- Class labels are required in this method.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "n4mOf2BAF8", "reviewer": "Reviewer_CwjJ", "summary": "The authors mainly aim to address spurious correlations by ranking instances according to the extent of biases automatically and conducting contrastive learning with these rankings. Specifically, based on the observation that the extent of biases is proportional to learning speed, they modify the learning process of ERM to progressively capture biases according to their extent. This modification process consists of three phases: selection, upweighting & training, and ranking. Then, with acquired rankings, they conduct contrastive learning to debias models by constructing negative pairs with similar levels of spurious correlations and positive pairs with different levels of spurious correlations. In experiments, the authors demonstrate the effectiveness of their method.", "questions": "* Would the authors provide how much additional training time is required compared to training ERM. It would be helpful if you could provide a breakdown of the time spent on ranking and contrastive learning separately. Although there is a discussion in the appendix, specific values are not provided.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "* The proposed method can acquire the ranking of instances based on the extent of spurious correlation without human annotations.\n* The authors show that their ranking is similar with the ground truth ranking and the overall framework mitigates performance gaps.\n* The paper is well-written.", "weaknesses": "- The paper lacks a detailed discussion on the utility of ranking instances according to the strength of spurious correlation. In this paper, the ranking is used for debiasing models via contrastive learning, yet it remains unclear how this approach offers high-level advantages over debiasing methods that do not utilize ranking. It would enhance the paper to discuss potential applications of their fine-grained spuriosity rankings beyond the contrastive learning framework. For example, the recent method, B2T [1], can identify and mitigate biases without human supervision by leveraging keywords. Since keywords are natural language, humans can interpret identified biases as well. Would the authors clarify the advantages over B2T?\n- The experimental validation of the proposed method is not convincing. First, the comparison does not include recent debiasing methods [1, 2, 3]. Additionally, the model is validated only on a few datasets within the visual domain. The paper would be strengthened by including experiments on other widely-used datasets for spurious correlation, such as Waterbird, and NLP tasks like MultiNLI and CivilComments. Finally, the metrics are not commonly used in the spurious correlation domain, so reporting worst-group accuracy would enhance comparability.\n\n[1] Kim, Younghyun, et al. \"Discovering and Mitigating Visual Biases through Keyword Explanation.\" CVPR, 2024.\n\n[2] Liu, Sheng, et al. \"Avoiding spurious correlations via logit correction.\" ICLR, 2023.\n\n[3] Deng, Yihe, et al. \"Robust learning with progressive data expansion against spurious correlation.\" NeurIPS, 2023.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "0mQez976Dv", "reviewer": "Reviewer_RG64", "summary": "This paper proposes a bias mitigation method by ranking data points based on the assumption that the harder a data point is to learn, the less spuriosity it has and the overall ranking process is achieved by selecting and upweighting the samples that have not yet been learned, which relive the need of human supervision compared to existing method used to identify biased features.", "questions": "- When selecting points that have not yet been learned, which are defined by large loss, how can the method avoid selecting data points that are outliers or affected by label noise issues?\n\n- Is Assumption 1 (Hardness-Spuriousness Symmetry) supported by any literature or empirical evidence?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "- The paper addresses the bias mitigation problem by introducing a fine-grained estimation method instead of the traditional binary biased-*vs*-unbiased partitioning approach, providing a more precise way to evaluate spuriosity and the combination of the contrastive learning framework is a nice attempt to further reduce the reliance on the few unbiased samples compared to existing methods by utilizing all the samples in the dataset.\n- The intuition behind the proposed method is clearly with the definitions and theorems. Although I think the assumption (assumption 1, Hardness-Spuriosity Symmetry) is strong, it still demonstrates a clear logic explaining why the ranking mechanism is designed in this way.", "weaknesses": "- The proposed method heavily relies on the assumption 1 (Hardness-Spuriosity Symmetry), which states that samples with large losses (difficult to learn) are likely to be less spurious. This raises concerns about the method’s effectiveness when the samples are outliers or have label noise issues....so if the large loss is caused by these samples, then utilizing them for training would adversely affect the performance on the downstream tasks. I think such condition should be discussed. \n- The proposed method has three parameters to optimize, so when the dataset has high dimensionality, the optimization process may become very slow, and potential convergence issues should be discussed.\n- The overall writing of this paper is not very straightforward, as it repeats the main concepts too many times (e.g., the concept of spuriousness, the assumption, selection, upweighting.....) and the method section (3.1 to 3.3) has very weak connections, making each part appear independent of the others, which weakens the logical flow when introducing the method. I suggest the authors consider merging some of the repeated content (or summarize when it repeat) and adding some transitional words between sections to improve coherence.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "g9lcjv2gMf", "reviewer": "Reviewer_APoD", "summary": "This work introduces SEBRA (Self-Guided Bias Ranking) to reduce bias in machine learning models without human intervention. Typically, models can develop biases by over-relying on spurious patterns in the data. SEBRA works by identifying these spurious patterns and ranking data points based on how likely they are to contain these misleading cues. During training, SEBRA adjusts the order in which samples are learned, focusing first on those with stronger biases and gradually moving to unbiased samples. This method allows the model to better distinguish between real patterns and biases, resulting in a more accurate and fair model.\n\nSEBRA further incorporates this ranking into a contrastive learning framework, which contrasts data points with strong biases against those with fewer biases, enhancing the model’s ability to learn unbiased representations. Tests across datasets such as UrbanCars, CelebA, and BAR show that SEBRA outperforms other debiasing methods, particularly when multiple sources of bias are present.", "questions": "Can you delve on the motivation a bit? Why is this kind of de-biasing important?\n\nIs Assumption 1 a part of known literature or is it something discovered during the behavioral exploration of this work? If it is a known fact in debiasing literature, please cite relevant works.", "limitations": "", "rating": 5, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "The paper is easy to understand, especially for readers unfamiliar with the area. The approach is interesting and addresses a problem of importance.\n\nThe proposed algorithm seems simple, intuitive yet effective. \n\nThe illustration of results in the appendix is quite helpful in building intuition.", "weaknesses": "The findings in the SEBRA heavily rely on Assumption 1, termed the Hardness-Spuriosity Symmetry, which states that the difficulty of learning a sample and its spuriosity (presence of misleading features) are inversely correlated. SEBRA uses this assumption to rank and prioritize samples in training, guiding the model to focus on samples with high spuriosity initially. The approach optimizes a ranking objective derived from this assumption, implying that SEBRA's effectiveness in reducing bias is largely contingent on this assumption holding true in practice. If assumption 1 does not hold, SEBRA’s ranking mechanism and bias-mitigation strategy might lose validity, as the model's ranking would no longer accurately reflect the spuriosity of samples. \n\nPlease add more references in the related work section.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)