A Simple Remedy for Dataset Bias via Self-Influence: A Mislabeled Sample Perspective
Authors: Yeonsung Jung
Jaeyun Song
June Yong Yang
Jin-Hwa Kim
Sung-Yub Kim
Eunho Yang
Venue: NeurIPS 2024 poster (2024)
Keywords: Debiasing
Spurious correlation
Robust learning
Dataset bias
Primary area: fairness
TL;DR: We propose a simple yet effective method for remedy through fine-tuning that utilizes a pivotal set constructed using Bias-Conditioned Self-Influence to recover biased models.

Abstract: Learning generalized models from biased data is an important undertaking toward fairness in deep learning. To address this issue, recent studies attempt to identify and leverage bias-conflicting samples free from spurious correlations without prior knowledge of bias or an unbiased set. However, spurious correlation remains an ongoing challenge, primarily due to the difficulty in correctly detecting these samples. In this paper, inspired by the similarities between mislabeled samples and bias-conflicting samples, we approach this challenge from a novel perspective of mislabeled sample detection. Specifically, we delve into Influence Function, one of the standard methods for mislabeled sample detection, for identifying bias-conflicting samples and propose a simple yet effective remedy for biased models by leveraging them. Through comprehensive analysis and experiments on diverse datasets, we demonstrate that our new perspective can boost the precision of detection and rectify biased models effectively. Furthermore, our approach is complementary to existing methods, showing performance improvement even when applied to models that have already undergone recent debiasing techniques.

A Simple Remedy for Dataset Bias via Self-Inftuence: A Mislabeled Sample Perspective
Yeonsung Jung"! Jaeyun Song" June Yong Yang! Jin-Hwa Kim 23 Sung-Yub Kim Eunho Yang!
Korea Advanced Institute of Science and Technology, 2NAVER Al Lab Seoul National University, "AITRICS {ys. jung, (ercery)@kaist.. @kaist. ac.kr'
Abstract
Learning, generalized models from biased data is an important undertaking toward fairness in deep learning. address this issue, recent studies attempt to identify and leverage bias-conflicting samples free from spurious correlations without prior knowledge bias or an unbiased set. However, spurious correlation remains an ongoing challenge, primarily due to the difficulty in precisely detecting these samples. In this paper, inspired by the similarities between mislabeled samples and bias-confficting samples, approach this challenge from novel perspective mislabeled sample detection. Specifically, we delve into Influence Function, of the standard methods for mislabeled sample detection, for identifying bias-conflicting samples and propose a simple yet effective remedy for biased models by leveraging them. Through comprehensive analysis and experiments diverse datasets, we demonstrate that our new perspective can boost the precision of detection and rectify biased models effectively. Furthermore, our approach is complementary to existing methods, showing performance improvement even when applied models that have already undergone recent debiasing techniques.
1 Introduction
Deep neural networks have demonstrated remarkable performance in various fields of machine learning tasks comparable or superior to humans on well-curated benchmark datasets 7 60 [4). Nevertheless, the efficacy of these models trained on unfiltered, real-world data remains an open question. In this scenario, significant concern arises due the presence of dataset bias [52) where task-imelevant attributes are spuriously correlated with labels only in the training set. This can lead to models that rely on misleading correlations rather than learning the task-related features, resulting in biased models with poor generalization performance [62 10)
To prevent models from learning detrimental bias, various methods are proposed encourage models to prioritize learning task-relevant features. Recent studies enhance task-related features by first identifying conflicinn (unbiased) samples through loss 440 36], gradients [ or bias prediction techniques [35] during training, using an auxiliary biased model trained with Empirical Risk Minimization (ERM). Then, they amplify bias-conflicinng samples by counteracting the bias- -alinnd (biased) samples through loss weighting [40 or weighted sampling [35 The effectiveness of such methods largely depends on their precision of bias-conf sample detection. Specifically, there risk of erroneously amplifying malignant bias instead of task-relevant features when bias-align -aigned samples are inaccurately identified as bias-conflicting. Due to the limited detection performance of it presents crucial challenge that remains unresolved.
previous methods 40 36 35
*Equal contribution.
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

In this paper, we address this challenge from novel perspective of mislabeled sample detection. Inspired by the similarities between mislabeled samples and crafffiifi samples, we delve into Influence Functions (IF: (7): one of the standard methods for mislabeled sample detection [55] 57 [29], to identify binffconn samples and propose a simple yet effective approach for biased models by leveraging them.
We first conduct a comprehensive analysis to explore the efficacy of Self-Influence (SI) variant of IF, in biased datasets. SI estimates how removing a specific training sample during training influences the prediction of the sample itself with the trained model (see Section. 2.2 By measuring
SI, we can identify a minority sample that, if removed from the training set, increases the likelihood of incorrect predictions of itself by the trained model due to their discrepancies with the majority samples. In this context, leveraging SI to biased datasets is promising as bias-con cinffccofi samples constitute the minority and contradict the dominant malignant bias learned by the model. However, we observe that unlike in mislabeled settings, directly applying SI to biased datasets is not effective (Figure ((a) ((d). Therefore, we investigate the differences between mislabeled samples and bias- conflicting samples and reveal the essential conditions for SI to effectively identify bias-confficting samples. Note that we denote SI under found conditions as Conditioned Self-Influence (BCSI).
Building on our analysis, we propose a simple yet effective method for rectifying biased models through fine-tuning. We construct small pivotal subset with a higher proportion of bias-conflicting samples using BCSI. While not perfect, this pivotal set can serve as an effective alternative to an unbiased set. Leveraging this pivotal set, we rectify biased model through fine-tuning with only a few additional iterations. Extensive experiments demonstrate that our method can effectively rectify even after models are already debiased by recent methods.
Our contributions are threefold:
We conduct a comprehensive analysis to explore the efficacy of SI in biased datasets and reveal the essential conditions for SI to accurately differentiate bias-conflicting samples, leading to Bias- Candioondt Self-Influence (BCSI).
We propose a simple yet effective remedy through fine-tuning that utilizes a pivotal set constructed using BCSI to rectify biased models across varying bias severities.
Our method is complementary to existing methods, capable of further rectifying models that have already undergone recent debiasing techniques.
2 Background
2.1 Learning from biased data
We consider a supervised learning setting with training data D {2n 1N sampled from the data distribution Z : (X, Y), where the input X is comprised of X x (S, O) where $ is the task- related signal, B a task-irrelevant bias, and target label of the task, where the label is y E (1...., C}. When the dataset is unbiased, ideally, model learns to predict the target label using the task task-relevant signal: PY|
the other cask-independent feature. Also, Y the
SS) B, O) = PSS)).) However, when the dataset is biased, the task-irrelevant bias B is highly correlated with the task-relevant features S with probability py, i.e., P(B= Under this relationship, data sample x b, o) is bias-aligned if (6 by) A (s 8y) and, bias-conflicttig otherwise, where A denotes the logical conjunction. When B is easier to learn than S for model, the model may discover a shorteut solution to the given task, learning to predict (X = P(Y|B) instead of XY = PSY However, debiasing a model inclines the model towards learning the true task-signal relationship PY P(Y[S).
= b, = 8y) = Py, where Py 2 d.
2.2 Influence Functions
Influence Function (IF; [I] (7)) estimates the impact of an individual sample from the training set on the model parameters, which in turn influences model predictions. A brute-force approach to assess the influence of a sample is to exclude the data point from the training set and retrain the model to compare differences in performance, referred to as leave-one-out (LOO) retaining. However, performing L.0 retraining for samples is computationally challenging: as an alternative, influence functions have been introduced as efficient approximated method.

---

11-
I Irarain
Prectsion (95)
Precision (36)
Preciaion 86
15
1.55
Proctdion
10
15
175-
train
(a) CMNIST (1%)..
(b) CIFARIOC (1%)).
(c) BFFHQ.
(d) Waterbird.
Figure Precision of detecting bias-conflictigg samples among Loss, Gradient Norm, Influence
function training set (II uain), and Self- (SI). The precision is evaluated with the ground truth number of bias-conflicting samples. The average precision of loss value, gradient norm, SI, and IF are presented in bars across three runs.
Here, we review the formal definition of influence funetion, Given a training dataset (=n ). where (In, Vn), model parameters are learned with loss function C:
6" = argmin L(D,0) argmin
EZzn. f)
where (zz,0) -og lo (ym (.)) is the cross-entropy loss for on with parameter
To measure the impact of single training sample on model parameters 0, we consider the retrained parameter 02, obtained by up- up-weighting the loss of by
6" argmin (CDD,)) ((2,6)).
0
Then, IF, the impact of on another sample z', defined as the deviation of the retrained loss E(z', 0%,)) from the original loss 1(''00%
Ix(2, 2' E(2', (2,), 0(21-00)
For infinitesimally small e, we have
I(2,2') dIc dIc(2,2²) de
(1)
c=0
where H V3C(D, RPPP is the Hessian of the loss function with respect to the model parameters at 0". Intuitively, the influence Z(z, z²) estimates the effect of z on z' through the learning process of the model parameters. Note that IF commonly computed once model has converged since Equation I approximates more accurately when the average gradient norm of the training set is sufficiently small.
Influence function also can calculated on itself to measure the Self-influence of 2:
[=))+(=) ~ V Vol(2,0')T 'oo(2(2,
which approximates the difference in loss of when z itself is excluded from the training This metric is commonly used for detecting mislabeled training samples in the noisy label setting 27 31
55 57
29
important samples in data pruning for efficient training 40 59
3
An analysis of Self-Influence in bias-conflicting sample detection
In this section, we conduct comprehensive analysis to delve into the efficacy of SI in bias-conflicting sample detection. First, we examine the process of identifying thas-conflicting sample detection through the perspective of mislabeled sample detection (Section [.1)) Next, we introduce essential conditions required
to effectively identify bias-conflicting samples by analyzing the differences between mislabeled and bias bias-conflicting samples (Section 3.2). We term the SI calculated under these conditions as Conditiondd Self-Influence (BCSI) and demonstrate that BCSI outperforms SI in detecting bias-conflicting samples.

---

Conventional
Biased model
Training data
Self-Influcnce
Pivotal set
Blas-sligned
Amplity
Banseligrad Bizsccoullitinn
D
9
Blas conditions
Identify Blss-ooutleting samples
Cias-Conditioned Self-Influence (BCSI)
Remainder
9
9
Bias conflicting
Post-training recovery
Bizsalianed Bizz-conflistin
Figure 2: The overview of our method. We compute Cadd-Condiiooeed Self-Influence (BCSI) of the training data and construct small but concentrated pivotal set with high ratio of bias-conflicting samples. Then, we remedy biased models through fine-tuning that utilizes the pivotal set and remaining samples.
3.1 Bias-confficting sample detection from the perspective of mislabeled sample detection
IF is one the standard methods for mislabeled sample detection (27] The use of influence functions for mislabeled sample detection generally involves approaches: computing influence scores using a clean validation set or computing self-influence scores. The former, I(2; V). utilizes a validation V free of mislabeled samples measure the impact validation loss, identifying samples whose removal reduces this loss as likely mislabeled. The latter, Self-influence I(z;, 2;), estimates how If removing sample
the loss of sample 2; changes when it is removed from the training significantly increases its own loss, it indicates that the sample is likely mislabeled, as normal samples can still be correctly predicted using the remaining samples. For instance, in a task classifying dogs and cats, if dog image is mislabeled as a cat, removing this mislabeled sample from the training set decreases the likelihood of correctly predicting
cat.
In this context, mislaheled samples and bias-conflicting samples share a key characteristic that both minority samples contradicting the dominant features learned by the model. Mislabeled samples have incorrect labels that conflict with the learned features, making them easily identifiable through SI. Similarly, bias-conflictnng samples contradict the malignant bias that model learns from biased dataset. Despite the different contexts, both types of samples can be detected through the same underlying principle of IF.
In summary, given the similarities between mislabeled samples and bias-conflicting samples, it is promising to leverage the perspective and methodology of mislabeled sample detection to identify bias-conflicting samples. However, in real-world scenarios, preemptively identifying malignant bias and constructing unbiased validation set mitigate the bias problem is impractical. Therefore, using self-influence offers a more feasible and practical solution for addressing bias-conflicting samples instead of using influence scores validation set. Consequently, center approach on SI to effectively detect bias-conflicting samples.
3.2 Bias-Conditioned Self-Influence (BCSI)
To validate Self-Influence (SI) in detecting bias-conflicting samples, we conduct experiments on benchmark datasets with diverse bias types and severities: Colored MNIST, Corrupted CIFAR10, Biased FFHQ (BFFHQ), and Waterbird. These datasets feature bias related color, synthetic corruption, gender, and place background, respectively (details in Appendix N.I. In contrast to mislabeled setting, we observe that directly applying SI to detect bias-conflicting samples in biased datasets often fails. In Figure the detection precision of SI is significantly low, mostly below 25%. Note that since an unbiased validation set is unavailable in our target problem, we additionally estimate the influence score the training set, indicated IFriin in Figure] ]

---

CINSST
Preditin P50
Pression
B
nffunnce
CF
Sample quatt
3.8
1
Epoch
100
BCC (Oun))
(a) Clussilication Acc.
(b) Detection precision Fiiin
(c) Detection precision of SI
(d) Histograms of SI and BCSI.
Figure 3: A comprehensive analysis of Influence function on the training set (IFuaia) and Self- Influence (SI) biased datasets. Figure 3(a) shows the classification accuracy of bias-aligned and bias-conflicting samples over training epochs. Figure ((b) and [(c) depict the detection precision of IF Irain and SI across training epochs for varying ratios of bias-conflicting samples in CIFAR Figure ((d) shows histograms of sample distribution in CIFARIOC (1%) and each bar indicates the number samples within specific range.
This is due the inherent differences between mislabeled samples and bias-conflicting samples. While mislabeled samples strongly conflict with the dominant features learned by the model due to their incorrect labels, bias-conflicting samples share task-related features with bias-aligned samples. For instance, in biased dataset where seagulls are spuriously correlated with sea backgrounds, seagull image against a desert background still retains the features of a seagull. Despite the dominance of malignant bias, these features are still partially utilized. Therefore, bias-cue clas-confiiciin samples do not exhibit clear contrast with the dominant features of a biased model, posing a challenge for using SI.
To address this challenge, we introduce essential conditions that enable SI to accurately detect bias- conflicting samples. The key concept is restrict the model from learning task-related features and instead induce the model to focus more on the malignant bias to achieve better separation. A simple but effective way attain this is by leveraging models in the early stages of training, since malignant bias is learned first, followed by task-related features later, according to Nam et al. [40]. In Figure ((a) experiments on CIFAR IOC and CMNIST demonstrate that the classification accuracy of bias-aligned samples increases rapidly, while that of bias-conflicting samples shows a slower rise. In addition, as shown Figure ((b) and [(C) our experiments on CIFARIOC with diverse ratios of bias-conflicting samples (0.5%, %6 2%, and 5%) demonstrate a significant decline in detection precision of IF and SI as training epochs increase, since the model gradually learns task-related features. Therefore, computing SI with models in the early stages of training can achieve better separation. Formally, given a model parameterized by 0 at an early epoch we compute the self-influence Iaant (z) as:
Iaair
V ((z T,, Vo, ((2,
where H, is the Hessian of the loss function at the parameter 0,.
To further enhance the separation of SI, we employ Generalized Cross Entropy (GCE) [61] to induce the model to focus more on the tasier-to-learn bias-aligned samples, resulting in more biased model. GCE emphasizes samples that are easier to learn, thereby amplifying the model's s bias by tending to give more weight to bias- aligned samples in the training set.
90
90
90
70
Precision (%)
So
50
50
50
10
10
10
employ the model trained under these conditions to measure SI and refer SI estimated by this heav- ily biased model with Equation as Bias- Conditioned Self-Influence (BCSI). Since we induce the model to heavily exploit bias and discourage the model from learning task-related features, BCSI can effectively detect bias-conflieting samples. To avoid the impracticality of manually searching epoch for each dataset, we base our method on the well-known findings of Frankle et al.
Consequently,
BCSI
SI
BCS
BCSI
BCS
CMNIST (1%)
CIFAR1OC (1%)
BFFHO
Waterbrid
Figure 4: Comparison of average precision between SI
and BCST across diverse datasets over three runs.
5

---

*
(a) Bias-conflicting with high BCST
(b) Bias-conflicting with low BCST
(c) Bias-aligned with high BCST
(d) Bias-aligned with low BCST
Figure 5: Example images from BFFHQ ranked within the top 100 by BCSI score. (a) and (b) are bias-conflicting samples with high and relatively lower BCSI scores, respectively. (c) is bias-aligned sample with high BCSI score, while (d) a bias-aligned sample with low BCSI score.
that the primary directions of the model's parameter weights are determined during 500 to 2,000 iterations. Thus, we set epoch i within this range according to the mini-batch size of each dataset. Specifically, we used f=5 for all datasets to ensure practicability and consistency across experiments, but fine-tuning the epoch t for each dataset can yield further improvement.
We validate the efficacy of BCSI in detecting bias-conflicting samples. Since calculating H-1 (D))(D)0)) is computationally expensive for large networks due to their extensive number of parameters, we calculate H and the loss gradient the sample 2, Vgl(z, (*), by using the last layer of the model following Koh and Liang [27]. Pruthi al. 43 In Figure 4 BCSI outperforms conventional SI in detection precision.
Additionally, Figure 3(d) demonstrates that BCSI has notable tendency for bias- samples to exhibit larger scores compared to bias-aligned samples, in contrast to SI. This trend also observed in other biased datasets, as shown in Appendix A and C These experimental results support that BCSI can serve as an effective indicator for identifying bias-conflicting samples. To further analyze the qualitative characteristics of bias- bias-confficting and bias-aligned samples within the top 100 samples ranked by BCSI, we examine BCSI on BFFHQ, as illustrated in Figure In BFFHQ, gender serves as the bias attribute and age the target attribute, leading spurious correlations between 'young' and 'woman' as well as 'old' and 'man'. For bias-conflicting samples, F(a shows that BCSI assigns high scores clear counterexamples, such boys very elderly women. In contrast, Figure 5(b) exhibits relatively lower BCSI scores for cases like slightly older young men or elderly women who appear younger, indicating that BCSI prioritizes samples with stronger opposition spurious correlations. A similar trend is observed for bias-aligned samples in Figure 5(c) and Figure 5(d) enhancing that BCSI effectively distinguishes between varying degrees of alignment with the spurious correlations.
4 Remedy biased models through fine-tuning
In this section, we propose a simple but effective remedy that first utilizes BCSI to construct concentrated pivotal subset abundant in bias-conflictngg samples and then employs for rectifying biased models via fine-tuning without leveraging the supervision of bias or an unbiased validation set. Our method is complementary existing methods, capable of rectifying models that have already undergone other debiasing techniques. The overall pipeline is described in Figure
Constructing a pivotal subset. We select the top-k subset of samples from each class, based on their BCSI scores, to form pivotal subset of bias-conflicting samples as follows: Zp

---

86 18 88
Confict
Accurac (%)
Acuraay
8
DE 8 8
Algn acc
Fine Lastlayer rebrning Vonill
20-
3
50
100
3 Bias confficting ralio (96
Pivotal set ratio (96)
Bias conficting ratio (%)
(a) Acc. of last layer retraining.
(b) Acc. fine-tuning.
(c) Performance gain.
Figure 6: Test accuracy under varying bias-confiicting ratios. Figure 6(a) shows the accuracy for last layer retraining across varying bias ratios in pivotal sets. Figure ( depicts performance changes of last layer retraining and fine-tuning under diverse bias ratios. In Figure [[c] performance gains are provided. We present the average accuracy with the error bars indicating the standard error across three runs.
U { [EBCSt rank(()) n== where C is the number of classes and BCSI-rank(n,, c) is the dataset index of the 1-th training sample of class sorted by BCSI score. Due to the unknown ratio of binfcconng samples beforchand, determining a proper through syper-parameter tuning for each dataset is computationally expensive. To mitigate this, repeat the selection process three times with different randomly initialized models and use the intersection of the resulting sets as the pivotal set. This ensures high likelihood of selecting bins-conf samples, as they are consistently identified by three runs. We provide the resulting bias-conflicting ratios of the pivotal sets across various datasets in Appendix E and confirm that this process improves detection precision by 16. % on average. Note that we set k = 100 across all datasets in our experiments. For computational costs, since we only train models for five epochs, this iterative approach incurs negligible cost compared to full training, as commonly done in previous studies [40] 31 [9]. We confirm that in Appendix The detailed filtering process outlined in Algorithm which can be found in Appendix B
Efficient remedy via fine-tuning. Recent works [26] 32 show that retraining specific layers of
model using small unbiased set can effectively mitigate bias in biased models, overcoming the inefficiency of retraining models from scratch [40 [3] 19] However, preemptively identifying the bias and curating an unbiased set very costly, making impractical solution. Instead, our method leverages the pivotal set which has a high proportion of bias-conflicting samples, as a practical alternative. While not perfect, our method can efficiently remedy biased models with just a few additional training iterations without the need for prior knowledge of bias or unbiased datasets. As shown in Figure [(a)] even without a perfect pivotal set, its concentration facilitates its applicability in fine-tuning. Note that contrary to the claims of Kirichenko et al. (26), we observed that in highly biased scenarios, the feature extractor also becomes biased, making last- retraining insufficient, as demonstrated in Figure/o(b) Therefore, we fine-tune all the parameters in the models.
In addition, we formulate counterweight cross-entropy loss by drawing a mini-batch from the remaining training set. In real-world scenarios, the unpredictability of bias severity necessitates ro- bustness across wide range of bias severities. However, previous methods often assume sufficient presence of bias-alignee samples in the training set, which limits their performance in low-bias sce- narios. Despite its significance, the study on both low and high-bias scenarios has been underexplored, and to the best of our knowledge, we are the first to bring this issue.
We then train the model using both the cross-entropy loss the pivotal subset and the counterweight loss on the remaining training set:
C(Zp, ZR) := LcE(Zp) + ALce ZR),
where Zp is the pivotal subset, ZR ~ Z Z,7 Zp a randomly drawn mini-batch from the remaining training set, and LCE is the mean cross-entropy loss.
To this end, our method efficiently remedies bias through fine-tuning that utilizes a pivotal set constructed via BCSI across varying bias severities. Additionally, our approach complements existing methods, capable of further rectifying models that have already undergone recent debiasing techniques. The overall process is described in Algorithm? 2 which is included in Appendix B
7

---

Table 1: The average and the standard error over three runs. Ours indicates our method applied to model initially trained with the prefix method. The best accuracy is annotated in bold. o indicates that given method uses bias information while x denotes that a given model does not use any bias information.
Bias
CMNIST
CIFARIOC
BFFHQ
Method
Info
0.5%
1%
2%
5%
0.5%
1%
2%
5%
0.5%
GROu
79.57
90.50
94.89
97.54
33.44
38.30
45.81
57.32
54.80
ERM
x 71.7611.a4
86.47.0s:
93.8719.32
96.28 1o:0
20.501o.s4
10.as
28.99 o..
10.24
53.53.2..5
LFF
X
89.08 89.0661..7
89.50+2
85.74+4.37
94.30ee.07
25.28+2.s0
31.15+107
38.64-0.0
46.15
35.33+2..0
DFA
84.71=1.00
90.20+1
92.31
94.33=1.28
27.13=1.6
31.26+2.71
37.96=0.71
44.99
52.07=1.01
BPA
73.34
87.21 م0
89 89.4223.37
17.13
25.50
27.4
34.29
51.40.a.0s
DCWP
85.16
89.68+6.5
89 49,42+4.25
95.17
31.27=0024
+0.87
=0.47
52.86+1.24
57.33e1.76
SelecMix
54.4620.5
95.75
28.09
37.63
40.
10.1422.42
47.54_0.50
54.86.0.7
63.07
Ours ERM
75.87=1.60
89.69
95.08+0.17
96.79.0.1s
26.61+0.se
33.47+0.20
40.75=0.a7
49.30+0.40
56.00=1.07
LIF
10.7
94. 10 oe
92.95a1.17
95.59
27.63=1.00
35.29
43.36.0.7
51.95
57.13
Ours DEA
88.39
+208
95.67+0.12
97.52=0.00
25.66
25.66+0.ss
33.53+2.01
42.80-0.a1
+0.51
56.60=2.8s
Ours SeleMix
X
95.35 1.
97.15
98.13
38.74
46.182o.as
52.70=0.10
59.66.0.31
63.80
Experiments
In this section, we present experiments applying our method to models trained with ERM and recent debiasing methods. We validate our method and its individual components by following prior conventions. Below, we provide brief overview of our experimental setting in Section 5.1 followed
by empirical results presented in Section 5.2 5.3 and (5.4
5.1 Experimental settings
We now describe datasets, baselines, and evaluation protocol. Detailed descriptions about these are provided in Appendix N
Datasets. For a fair evaluation, we follow the conventions of using benchmark biased datasets 440 Colored MNIST dataset (CMNIST) is a synthetically modified MNIST 66, where the labels are correlated with colors. We conduct benchmarks on bias ratios of r E (0.5, 0.1, 0.2, 5). CIFARIOC is synthetically modified CIFARIO 10 (30 dataset with common corruptions. To evaluate our method in low-bias scenarios, we expand our scope and conduct experiments with varying bias ratios r E {0.5, 0.1. 0.2, 20, 30, 50, 70, 90(unbiased)), ). Biased FFHQ (BFFHQ) [3]] curated Flickr- Faces-HO (FFHQ) (22) dataset, which consists of facial images where ages and genders exhibit spurious correlation. The Waterbirds dataset [53] consists of bird images, to classify bird types, but their backgrounds are correlated with bird types. Non-L.I.D. Image dataset with Contexts (NICO) [16] a natural image dataset for -uitt--distiibutiinn classification. We follow the setting of (54, inducing long-tailed bias proportions within each class, simulating diverse bias ratios in single benchmark. Additionally, to demonstrate the effectiveness of our method on NLP datasets, we conduct experiments on CivilComments 2 28 and MultiNL [58 45 as detailed in appendix F
Baselines. Since our goal addressing the dataset bias without leveraging any prior knowledge of bias an unbiased set, we evaluate our method with such baselines. GroupDRO 44 uses bias
supervision to debias models. LIF [4)] BPA [48), and DCWP [4] adjust the loss function to amplify the learning signals for bias-conflictngg samples. DFA 31 and SelecMix [9] augment samples possessing various biases different from the original data.
Evaluation protocol. Following other baselines, we calculate the accuracy for unbiased test sets in CMNIST, CIFARIOG IOC, and NICO. We measure the minority- accuracy in BFFHQ, and the worst-group accuracy in Waterbird. Note that use the models from the final epoch for all
8

---

Table 2: The average and the standard error over three low-bias scenarios.
Table 3: Accuracy Waterbirds, NICO
CIFARIOC
Method
Method
Waterbird
NICO
20%
30%
50%
70%
90
ERM
19.47.10.5
65.64_0..1
71.33_
74.90.0.25
16.03
ERM
68.74_a..5
39.5611.77
59.78
60.56 96
60.35
62.5220.49
63.4240.64
75.27 34.5621.47
DFA
60.34+0..0
64.2420.44
65.97=1.80
64.97
20.500
DEA
77.57.11.0
44.59
SelecMix
62.05
22.17.
62.52=1.64
26.23
65.8 20.81
SelecMix
74.72=1.14
33.87
Ours BaM
62.78
65.61
70.61
73.20
73.57
Ours BAM
87.64:1.10
43.54
Qursum
64,461105
64.40
65.82_..15
67.29
68. 1510.76
Oursua
87.85_o.0s
40.1819.51
Our
66, 40.30
68.1320.45
73.5620.1 =2.79
70.36
Ours
87. 32.1220.66
45.69
Ours
66.67+0.45
64.51=1.44
66.45-0.25
69.97
69. +0.77
Ours Sokeblis
89.67
44.33
CIARRIIC Welzted RETNN
Arcinacy
Accursic
Acturaic
CIPARII urblese
CIFRR 0.5%
Virdil u4Ous
0
Selectia
300
100
200
05
61 67
03
06
05
Dio Codliciin Rallo
Top-K
Cross ermopy
Epoch
Acc. bias ratios.
(b) Acc. on varying k.
(c) Acc. varying A.
(d) Acc. epochs.
Figure 7: Figure ((a) displays the test accuracy for SelecMix and our method at different bias ratios.
Figure 7(b)
7 and depict the unbiased evaluation under varying the size k in the pivotal set, X, the number of epochs for detection models, respectively. We present the average accuracy with the error bars indicating the standard error across three runs.
experiments to evaluate performance. We report the average value and the error bars denote standard errors across three runs.
5.2 Results on highly biased scenarios
We evaluate our method to measure the degree of rectification of baseline models when combined with ours on benchmark datasets. In Table we significantly enhance the performance baselines on the majority of datasets under various experimental settings. Ours seleMix achieves state-of-the-art accuracy on CIFAR1OC. Also, we observe that performance gain is larger the ratio of bias- conflicting samples increases in CIFARIOC. We conjecture that fine-tuning becomes more effective in CIFARIOC (2%) and (5%) since the bias-conflicting sample purity of the pivotal set increases, as shown in Section 4 In Figure [(c) performance gain mainly stems from the increased performance of bias-conflicingg samples.
5.3 Results on low-bias scenarios
We validate the baselines CIFAR1OC under varying ratios of bias-conficting samples in Table and 3 Baselines exhibit performance deterioration compared to ERM when the bias-confficting ratio high. In contrast, mildly biased datasets except for ERM. Albeit there is a slight decrease in performance for ERM, the accuracy gap is much lower than other baselines. Since the innate nature of fine-tuning can minimize friction by training from pre-trained parameters, our method can remedy bias in a wider range of bias ratios, in Figure ((a) The results for other methods provided in Appendix D
method can significantly rectify remaining bias within a model, even in
5.4 Ablation study
We examine the sensitivity of hyperparameters such as the number of selected samples per class (k) in the pivotal set, the weight for the remaining data in fine-tuning (A), and the number of epochs

---

to train detection models. In Figure 7(b) there is a slight performance decrease as k increases in CIFARIOC (0.5%). In contrast, the accuracy CIFARIOC (5%) increases. Since there are a few bias-conflicting samples per class in CIFAR I0C (0.5%), additional usage of samples dilutes the ratio of bias-conflicting data in the pivotal set, leading to a performance drop. Figure [7()] we observe marginal accuracy drop as À increases in CIFAR 10C (0.5%), CIFAR10C (90%) experiences performance increase. These results indicate that learning the remaining samples is beneficial in CIFARIOC (90%), fostering the model to capture task-relevant signals. For the number of epochs used to train the model for the detection, we compute the final performance when combining SelecMix in Figure [(d) Except for insufficiently trained epoch, the performance is not sensitive to the number of epochs between 3 and 11 epochs. We note that the analysis for intersections is provided in Appendix H
6 Related work
Debiasing deep neural networks. Research on mitigating bias has centered on modulating task- related information and malignant bias during training. Early works relied on human knowledge
through direct supervision or implicit information of bias 44 33 12,, which is often impractical
due to its acquisition cost. Thus, several studies have focused on identifying and utilizing bias- conflicting samples without relying on human knowledge. Loss modification methods [40] [3 41] amplify the learning signals of (estimated) bias-conflie chas-conflicting samples by modifying the learning objective. Sampling methods [35 II overcome dataset bias by sampling (estimated) bias-conflicting
data more frequently. Data augmentation approaches [1 34 20 19 synthesize samples with various
biases distinct from the inherent bias of the original data. Recently, based on the observation that bias in classification layers is severe compared to feature extractors, several approaches focus on rectifying
the last layer (3 39 26 Similarly, Lee et al. [32] demonstrated that selectively fine-tuning a subset of the layers with unbiased dataset can match or even surpass the performance of commonly used fine-tuning methods. However, identifying the bias and curating an unbiased set is very costly, making it an impractical essential condition.
Influence functions. Influence function (IF) and its approximations +3 2
have been utilized
in various deep learning tasks by measuring the importance of training samples and the relationship between them. One application of IF is in quantifying memorization by self- influnnfueeee, which is the increase in loss when training sample is excluded [43] g.. IF can be used to estimate the significance of samples, enabling the reduction of less important ones for efficient training 49 59 Recent works utilize IF identify and relabel mislabeled samples in noisy label settings 22 51
55 57 29). Furthermore, IF has also been applied in 3D domains like NeRF, where it measures pixel-wise distraction caused by unexpected objects, aiding in the identification and mitigation of such distractions (21)
7
Conclusion
In this work, we introduce a novel perspective of mislabeled sample detection on biased datasets. By conducting a comprehensive analysis of Sclf-Influence in detecting bias-conflicting samples, discover essential conditions required for SI to effectively identify these samples, which denote as Bias-Conditionel Self-Influence (BCSI). Building on our analysis, we propose a simple yet effective remedy for biased models through fine-tuning that utilizes small but concentrated pivotal set constructed via BCSI. Our method is not only capable of further rectifying models that have already undergone recent debiasing techniques but also demonstrates better generalization wide range of bias severities compared to previous studies.
Limitations. In this work, we rectify biased models via simple fine-tuning approach. However, this is the basic method; more sophisticated techniques such as sample weighting or curriculum learning possible. We believe that our introduction of this novel perspective will pave the way for more advanced future work.
Broader impact. Our work aims to learn unbiased deep learning models without bias annotations. Since filtering every training data under every given circumstance, the social impact of the ability to debias biased deep learning model after its training is much needed in terms of faimess.
10

---

Acknowledgements
This work was partly supported by Institute of Information communications Technology Planning & Evaluation (ITTP) grants
Meta-learning applicable to real-world problems, No.2022-0-00984 Development of Artificial Intelligence Technology for Personalized Plug-and-Play Explanation and Verification of Explanation, National Research Foundation of Korea (NRF) grants (RS-2023-00209066 A Study on Optimization and Network Interpretation Method for Large-Scale Machine Learning) funded by the Korea government (MSIT), and KAIST-NAVER Hypercreative Al Center.
Artificial Intelligence Graduate
School Program(KAIST)
References
[1] Sumyeong Ahn, Seongyoon Kim, and Se- Young Yun. Mitigating dataset bias by using per- sample gradient. In The Fleventh International Conference on Learning Representations, 2023.
URL https:://pperrrrier
/tt///orrmmimmnneeeee 7GMv
[2] Daniel Borkan, Lucas Dixon, Jeffrey Sorensen, Nithum Thain, and Lucy Vasserman. Nuanced metrics for measuring unintended bias with real data for text classification. In Companion proceedings of the 2019 world wide web conference, pages 491-500, 2019.
[3] James Bradbury, Roy Frostig, Peter Hawkins, Matthew James Johnson, Chris Leary, Dougal Maclaurin, George Necula, Adam Paszke, Jake VanderPlas, Skye Wanderman- and Qiao Zhang. JAX: composable transformations of Python+NumPP programs, 2018. URL
http://githut com/ com//ooole/jar
[4] Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners. Advances in neural information processing systems, 33:1877-1901, 2020.
[5] Alexandra Chouldechova. Fair prediction with disparate impact: A study of bias in recidivism prediction instruments. Big data, 5(2):153-163, 2017.
[6] Li Deng. The mnist database of handwritten digit images for machine learning research [best of the web]. IEEE signal processing magazine, 29(6):141-142 2012.
[7] Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby. An image is worth 16x16 words: Transformers for image recognition at scale. In International Conference Learning Representations (ICLR), 2021. /
net wwwteeetttooee "rerum?id-YicbbaNNT
[8] Vitaly Feldman and Chiyuan Zhang. What neural networks memorize and why: Discovering the long tail via influence estimation. Conference on Neural Information Processing Systems (NeurIPS), 33:2881 1-2891, 2020.
[9] Jonathan Frankle, David J. Schwab, and Ari S. Morcos. The early phase of neural network training. International Conference on Learning Representations, 2020. URL https //openreview Met//forum'dd-HHI1RRRNM5
[10] Robert Geirhos, Jörn-Henrik Jacobsen, Claudio Michaelis, Richard S. Zemel, Wieland Brendel, Matthias Bethge, and Felix A. Wichmann. Shortcut learning in deep neural networks. Nat. Mach. Intell., 2(1 1):6656673, 2020.
[11] Frank R Hampel. The influence curve and its role in robust estimation. Journal of the american statistical association, 69(346):383-393 1974.
[12] Xiaochuang Han and Yulia Tsvetkov. Influence tuning: Demoting spurious correlations via in- stance attribution and instance-driven updates. In Findings of the Association for Computational Linguistics: EMNLP 2021, pages 4398-4409, 2021.
[13] Moritz Hardt, Eric Price, and Nati Srebro. Equality of opportunity in supervised learning. Advances in neural information processing systems, 29, 2016.
11

---

[14] Kaiming He. Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Delving deep into rectifiers: Surpassing nevell performance on imagenet classification, In Proceedings of the IEEE international conference computer vision, pages 1026-1034, 2015.
[15] Kaiming Hc, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In IEEE Conference Computer Vision and Pattern Recognition (CVPR), pages 770-778. IEEE Computer Society, 2016.
[16] Yue He, Zheyan Shen, and Peng Cui. Towards non-iid image classification: A dataset and baselines. Pattern Recognition, 110:107383, 2021.
[17] Dan Hendrycks and Thomas G. Dietterich. Benchmarking neural network robustness common corruptions and perturbations. CoRR, abs/1903. 12261, 2019. URL httpt:://aaa....rrrrrrt.////// abs/ 1903. 12261
[18] Youngkyu Hong and Eunho Yang. Unbiased classification through bias and bias- balanced learning. In M. Ranzato, A. Beygelzimer, Y. Dauphin, P.S. Liang, and J. Wortman Vaughan, editors, e Advances in Neural Information Processing Systems, volume 34, pages 26449
htttts:://ppccceeednneeeooee neurips..... cttccc/cppp.. cc/paper. 26461. Curran Associates, Inc., 2021. URL httpp// files http:p/pw.8880/.2222222266666662
[19] Inwoo Hwang, Sangjun Lee, Yunhyeok Kwak, Seong Joon Oh, Damien Teney, Hwa Kim, and Byoung-Take Tak Zhang. Seleemix: Debiased learning by contradicting- sampling. In Conference on Neural Information Processing Systems (NeurlPS), volume 35, pages 14345- 14357, 2022.
[20] Yeonsung Jung. Hajin Shim, June Yong Yang, and Eunho Yang. Fighting fire with fire: Con- trastive debiasing without bias-free data via generative bias-transformation In International Conference on Machine Learning, ICML 2023, 23-29 July 2023, Honolulu, Hawaii, USA, volume 202 of Proceedings of Machine Learning Research, pages 15435- PMLR, 2023.
[21] Yeonsung Jung, Heecheol Yun, Joonhyung Park, Jin-Hwa Kim, and Eunho Yang. PruNeRF: Segment-centric dataset pruning via 3D spatial consistency. In Proceedings of the 41st Inter- national Conference on Machine Learning, volume 235 of Proceedings of Machine Learning mlr
Research, pages 22696-22709 22709, PMLR, 21-27 Jul 2024. URL httpss:/ press/v235/ wwguub.gttaa.
[22] Tero Karras, Samuli Laine, and Timo Aila. A style-based generator architecture for generative adversarial networks. Proceedings of the IEEF/CVF conference on computer vision and pattem recognition, pages 4401-4410, 2019.
[23] Nayeong Kim, Sehyun Hwang, Sungsoo Ahn, Jaesik Park, and Suha Kwak. Learning debiased classifier with biased committee. In ICML. 2022: Workshop Spurious Correlations, Invariance and Stability, 2022. URL htt.:ttts///ppereeeeee wtt.:enrrewweeeeeee/rrtuormmmeee www.rruuunniiddbbuuuuuuuauaa.
[24] Sung Yub Kim, Kyungsu Kim, and Eunho Yang. GEX: flexible method for approximating influence via geometric ensemble. In Conference on Neural Information Processing Systems (NeurIPS), 2023. URL http http:/ewwww...etteoroourrrarrae.
[25] Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In Inter- national Conference on Learning Representations (ICLR), 2015. URL http:///://arrt org/ abs/1412.6955
[26] Polina Kirichenko, Pavel Izmailov, and Andrew Gordon Wilson. Last layer re-training is sufficient for robustness spurious correlations. In International Conference on Learning Representations (ICLR), 2023. URL http::/pppppooooooro.ooooorrrraa -bG.AA-Fgcckke
[27] Pang Wei Koh and Percy Liang. Understanding black-box predictions via influence functions. In Doina Precup and Yee Whye Teh, editors, International Conference on Machine Learning (ICMI., volume 70 of Proceedings of Machine Learning Research, pages 1885-1894. PMLR, 06-11 Aug 2017. URL
.ww..ag.mmmrrrrrmmrrmmm..m / html
12

---

[28] Pang Wei Koh, Shiori Sagawa, Henrik Marklund, Sang Michael Xie, Marvin Zhang, Akshay Balsubramani, Weihua Hu, Michihiro Yasunaga, Richard Lanas Phillips, Irena Gao, et al. Wilds: A benchmark of in-the-wild distribution shifts. In International conference on machine learning, pages 5637-5664. PMLR, 2021.
[29] Shuming Kong, Yanyan Shen, and Linpeng Huang. based data relabeling. In International Conference on Learning Representations (ICLR), 2022. URL /tLLitttpp///opnneeeeeee Hsst/
Resolving training biases via influence-
EskfHObuNVn
[30]
Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. 2009.
[31] Jungsoo Lee, Eungyeup Kim, Juyoung Lee, Jihyeon Lee, and Jaegul Choo. Learning debiased representation via disentangled feature augmentation. In Conference on Neural Information Processing Systems (NeurlPS), pages
2021.
[32] Yoonho Lee, Annie S Chen, Fahim Tajwar, Ananya Kumar, Huaxiu Yao, Percy Liang, and Chelsea Finn. Surgical fine-tuning improves adaptation to distribution shifts. In The Eleventh International Conference on Learning Representations, 2023. URL https://openreviie
net/forum?2dd-APPPRy.
[33] Yi Li and Nuno Vasconcelos. REPAIR: removing representation bias by dataset resampling. In IEEE Conference on Computer Vision and Patern Recognition (CVPR), pages 9572-9581, Computer Vision Foundation/ / IEFE, 2019.
[34] Jongin Lim, Youngdong Kim, Byungjai Kim, Chanho Ahn, Jinwoo Shin, Eunho Yang, and Seungju Han. Biasadv: Bias-adversarial augmentation for model debiasing. In IEEE/CVF Conference on Computer Vision and Putlern Recognition, CVPR 2023, Vancouver, BC, Canoda, June 17-24, 2023, pages 3832-3841. IEEE, 2023.
[35] Evan Zheran Liu, Behzad Haghgoo. Annie S. Chen, Aditi Raghunathan, Pang Wei Koh, Shiori Sagawa, Perey Liang, and Chelsea Finn. Just train twice: Improving group robustness without training group information. In International Conference on Machine Learning (ICML), volume 139 of Proceedings of Machine Learning Reseurch, pages 6781-6792. PMLR, 2021.
[36] Sheng Liu, Xu Zhang, Nitesh Sekhar, Yue Wu, Prateck Singhal, and Carlos Germandez-Grandee Avoiding spurious correlations via logit correction. In International Conference on Learning Representations (ICLR), 2023. httptttt/pppptereeeeteee
net wet./forummiddSSaaCC/
[37] Ilya Loshchilov and Frank Hutter. SGDR: stochastic gradient descent with warm restarts. In International Conference on Learning Representations (ICLR). OpenReview.net, 2017. URL
https://openrevie net f orum? -ieeddddddSsssee -kg889sx
[38] Xingjun Ma, Hanxun Huang, Yisen Wang, Simone Romano, Sarah Erfani, and James Bailey. Normalized loss functions for deep learning with noisy labels. In International conference machine learning, pages 6543-6553, PMLR, 2020.
[39] Aditya Krishna Menon, Ankit Singh Rawat, and Sanjiv Kumar. Overparameterriattin and worst- case generalisation: friend or foe? In International Conference on Learning Representations (ICLR), 2021. URL httptt:://ppprreeeeeeee
[40] Jun Hyun Nam, Hyuntak Cha, Sungsoo Ahn, Jaeho Lee, and Jinwoo Shin. Learning from failure: De-biasing classifier from biased classifier. In Conference on Neural Information Processing Systems (NeurIPS), 2020.
[41] Geon Yeong Park, Sangmin Lee, Sang Wan Lee, and Jong Chul Ye. Training debiased subnet- works with contrastive weight pruning. In IEEE Conference on Computer Vision and Pattern Recognition ( (CVPR), pages 7929-7938. IEFF, 2023.
[42] Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Köpf, Edward Z. Yang. Zachary Vetito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. Pytorch: An imperative style, high-performance deep learning library. In Conference on Neural Information Processing Systems (NeurIPS), pages 8024-8035, 2019.
13

---

[43] Garima Pruthi, Frederick Liu, Satyen Kale, and Mukund Sundararajan. Estimating training data influence by tracing gradient descent. Conference on Neural Information Processing Systems (NeurIPS), 33:19920-19930, 2020.
[44] Shiori Sagawa, Pang Wei Koh, Tatsunori B. Hashimoto, and Percy Liang. Distributionally robust neural networks. In International Conference on Learning Representations (ICLR), 2020.
[45] Shiori Sagawa*, Pang Wei Koh*, Tatsumori B. Hashimoto, and Percy Liang. Distributionally robust neural networks. In International Conference on Learning Representations, 2020. URL
https://openreviev..n ntt/: htt.://aw..aiaaaa..oooooooooooaa
[46] Andrea Schioppa, Polina Zablotskaia, David Vilar, and Artem Sokolov. Scaling up influence functions. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 36, pages 8179-8186, 2022.
[47] Ramprasaath R Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedantam, Devi Parikh, and Dhruv Batra. Grad-cam: Visual explanations from deep networks via gradient-based localization. In Proceedings of the IEEE international conference on computer vision, pages 618-626, 2017.
[48] Seonguk Seo, Joon- Youn-Yung Lee, and Bohyung Han. Unsupervised learning of debiased repre- sentations with pseudo-atrribues. IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2022, New Orleans, LA, USA, June 18-24, 2022, pages 16721-16730. IEEE, 2022.
[49] Ben Sorscher, Robert Geirhos, Shashank Shekhar, Surya Ganguli, and Ari Morcos. Beyond neural scaling laws: beating power law scaling via data pruning. Advances in Neural Information Processing Systems, 35:19523-19536 2022.
[50] Haoru Tan, Sitong Wu, Fei Du, Yukang Chen, Zhibin Wang, Fan Wang, and Xiaojuan Qi. Data pruning via moving-one saani-----------oouueee Advances in Neural Information Processing Systems, 36, 2023.
[51] Daniel Ting and Eric Brochu. Optimal subsampling with influence functions. Conference on Neural Information Processing Systems (NeurIPS), 31, 2018.
[52] Antonio Torralba and Alexei A. Efros. Unbiased look at dataset bias. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 1221-1223 IEEE Computer Society, 2011.
[53] Catherine Wah, Steve Branson, Peter Welinder, Pietro Perona, and Serge Belongie. The caltech- uesd birds-200-2011 dataset. 2011.
[54] Tan Wang, Chang Zhou, Qianru Sun, and Hanwang Zhang. Causal attention for unbiased visual recognition. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 3091-3100, 2021.
[55] Tianyang Wang. Jun Huan, and Bo Li. Data dropout: Optimizing training data for convolutional neural networks. In IEEE 30th International Conference on Tools with Artificial Intelligence, ICTAI 2018, 5-7 November 2018, Volos, Greece, pages 39-46. IEEE, 2018
[56] Yisen Wang, Xingjun Ma, Zaiyi Chen, Yuan Luo, Jinfeng Yi, and James Bailey, Symmetric cross entropy for robust learning with noisy labels. In Proceedings of the IEEE/CVE international conference computer vision, pages 322-330, 2019.
[57] Zifeng Wang. Hong Zhu, Zhenhua Dong, Xiuqiang He, and Shao-Lun Huang. Less is better: Unweighted data subsampling via influence function. In The Fourth AAAI Conference on Artificial Intelligence, AAAI 2020, The Thirty-Second Innovative Applications of Artificial Intelligence Conference, IAAI 2020, The Tenth AAAI Symposium Educational Advances in Artificial Intelligence, EAAT 2020, New York, NY, USA, February 7-12, 2020, pages 6340-6347. AAAI Press, 2020.
14

---

[58] Adina Williams, Nikita Nangia, and Samuel Bowman. A broad-coverage challenge corpus for sentence understanding through inference. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume (Long Papers), pages 1112-1123 New Orleans, Louisiana, June 2018. Association for Computational Linguistics. doi: (0.18653/1//118-11000 URL https:
aclanthology.oo www.ooyy...g.----oo -1101
[59] Shuo Yang, Zeke Xie, Hanyu Peng, Min Xu, Mingming Sun, and Ping Li. Dataset pruning: Reducing training data by examining generalization influence. In International Conference on Learning Representations (ICLR), 2023. URL https: /ttt::://openrvvee fet
4w21AXD29TQ
[60] Qian Zhang, Han Lu, Hasim Sak, Anshuman Tripathi, Erik McDermott, Stephen Koo, and Shankar Kumar. Transformer transducer: A streamable speech recognition model with trans- former encoders and RNN-T loss. In IEEE International Conference Acoustics, Speech and Signal Processing (ICASSP), pages 7829-7833. IEEE, 2020.
[61] Zhilu Zhang and Mert Sabuncu. Generalized cross entropy loss for training deep neural networks with noisy labels. In Conference on Neural Information Processing Systems (NeurIPS), vol- ume 31. Curran Associates, Inc., 2.18. URL httphtttp////rrreeednngg mew..ddddddnnnnnnen............. cc/ /epparr._
pdf
[62] Zhuotun Zhu, Lingxi Xie, and Alan L. Yuille. Object recognition with and without objects. In International Joint Conference on Artificial Intelligence (IJCAI), pages 3609-3610 ijeai.org, 2017.
15

---

A Distribution of Self-Influence and bias-conditioned Influence
In Figure [[c] and Figure I(d] of the main paper, we have shown the influence histogram of naive self-influence and bias-condittonee self-influence (Ours) for the training set of CIFARIOC (1%). In this section, we show the histograms of self-influence and olas-conditioned self-influence for the training sets of an extended variety of bias ratios and datasets. Figure shows the influence histograms for CIFAR 1OC, and BFFHQ. Figure shows
influence histograms CMNIST, Waterbird, and NICO. In accordance with the main paper, we observe that bias clas-conditioned self-influnnce generally exhibits better separation compared naive self-influence, deeming a better option to detect bias-conflicting samples.
b snvaal gred b im-contlccing
Ham-4lghed
bann igned b me-conilccing
bran-e igned mab-cordicctin
100
100
bian- men-conticinng
121
104
201
120
Sample
30
109
100
Self-inlluece
022
Self-infleenca
0.6
0.5
0.8
1.0
2.5
2.5
0.00
0.25
0.30
0-75
1.00
1,2
(a) SelF.IF CMNIST (0.5%)
(b) iC CMNIST (0.5%)
(c) Self in CMNIST (1%).
(d) BCST CMNIST (1S.).
gigeed confliting
lgned conflictnng
alined conflicting
algeed
1.*.
Sample: count
Sample count.
Sample countt
104
2.2
104
2.0-
201
10a
Selfinfluence
le5
Self-influence
1e7
Selfinftunnce
1e2
Selfinfluece
1e6
(e) Self-IF in CMNIST (2%).
(f) BCSI in CMNIST (2%).
(g) Self-IF in CMNIST (5%).
(h) BCSI in CMNIST (5%)
bas aigeed
Was lgned tas:conffddigg
bls alined blas con-confiictnng
alsallgeed
bas-conficting
10"
103
was-confiict nt
10a
107-
Sample count
Semple count
Semple count
1007
30a-
100
107
10a 10
109
Selfinfluence
(Ours)
32
Selfinfluence
36
S.
1:0
0.0
05
(Ours)
2.0
75
(i) Self-IFin Waterbirds (5%).
(j) BCSI Waterbirds (5%).
(k) Selt-IF NICO.
(1) BCSI NICO.
Figure 8: Histogram of self-influence and bias-conditioned self- self-influence for CMNIST, Waterbird, and NICO.
16

---

aecaa
igned mncconilcingg
stsseligee
Nasw
10r
an-conficting
10+
104
104
ban-condiiting
Sample count
Sample count
Sample count
12
Sample
10
10
201
16
100
109
109
109
Selfiiffuence
1e5
0.5
BCSI (Ours)
1e7
Self-influence
le6
2a
BCSI (Ours)
1c7
Self-TF CIFARIOC (0.5%)
(b) BCST 10C (0.5%)
(c) SelFTF CIFAR 10C (2%).
(d) BCST CIFARIOC (2%)).
bas al gred b ban-conticting
as aigned b bee-ondliciigg
bas aignel baseond nt
Dias sls-algneed Sien-conti eing
Sam counte
Sample count
201
100
109
10
10a
10"
0.00
0.25
Solt
0:50
077
1.00
BCSI (Ours)
1e7
0.00
Solt
0.50
0:75
BCSI (Ours)
1e7
(e) Self-IF CIFRRIC (5%).
(1) BCSI CIFARIOC (5%)
(g) Seif-IF in CIARIIC (219%).
(h) BCSI CIFARIOC (200%)
algnnd
confictiing
10f
104
conf
10'
conf
30-
Sample courtt
Sample court
Sample court
100
104
200
10a
101
Self influence
1e6
BCSI (Ours)
1e7
Selt influence
1:00
1e6
BCSi (Ours)
1e7
Seif-IF CIFARIOC (30%).
BCSI in CIFARIOC (30%)..
(k) Seif-IF in CIFARIOC (50%).
(I) BCSI in CIFARIOC (50%)
Was lgned Mas-confliiinn
10"
bla
104
bas aggeed
300
Sample count
1.2
Sample count
10
Sampler count
10
1.
109
10P.
10a
109
8700
Selrinfluence
1n6
BCSI (Ours)
106
Selfinfluence
164
BCSI (Ours)
165
(m) Self-IF in CIFARIOC (70%). (n) BCSI CIFARIOC (70%).
(0) Self-IF BFFHQ (0.5%).
(p) BCSI in BFFHQ (0.5%))
Figure 9: Histogram of self-influence and bias-conditioned self-influence for the CIFAR10C dataset
with varying bias-conflicting ratio and BFFHQ.
17

---

B
Algorithm
Algorithm Construct a pivotal set
Algorithm 2 Post-training with the pivotal set
I: Input: model parameters 6, GCE LocE, number epochs Repoch. learning rate p. number of classes
I: Input: pre-trained model parameters 0*, CE LeE, number of iterations Tliers learning rate p. train set
C. train set Z, number of topk Thopk
Z, pivotal set Zp, weight of remaining set À
2: Initialize: Model parameter 0.
2: Initialize: Last-layer of model Baatdyyrr
3: for 0, 2, -.. Repooh do
ZR <- Z Zp
4: @ 4 - pV efcce (Z, 8)
4: np <- [Pp
5: end for
5: for i = 0, 0, 2, .... Raarr do
6: # Select samples with high self-influence
6: # Sample data from remaining samples
7: Zp 6
Zs 50
8: 11c=001,1 0.
do Z: - [(T,1)] E7yy) =3
for
1,2, ... mp do
9:
9:
~~ 7R
10:
for j = 0,1,2, Maople do
10:
7s 7s {2}
11:
Shighest argmaxx, I (2)
11:
end for
Ze Ze (highhes
C LrF (Zr,
13:
Zy - Zp
/highees
13:
++ L + XLcL (Zs, 0*)
14:
end for
14:
8r 8-+ - -VVol
15:
end for
15:
end for
16: Output: Ze
Output: 8"
C Detection precision for other datasets
We now describe the detailed experimental setting used in Figure 4 of the main paper. We first train ResNet18 [15] for five epochs and then compute self-influence, and conditiodd self-influence. Note that only the last layer when computing self-influence, and bias-conditions influence. Subsequently, we sort the training data in descending order based on the values obtained by each method, selecting samples ranging from the highest the k-th sample, where k is the number of total bias-conflicting samples the training set. We then calculate the precision in detecting bias-conflicting samples within the selected data.
self-
To further demonstrate the effectivenesss of biab- conditiond self-influence in detecting bias-conflicting samples, we compare bias-conditioned self-influence with self-infuennce other datasets including CMNIST (0.5%, 2%, 5%), CIFARIOC (0.5%, 2%, 5%, 20%, 50%), NICO. As shown in Table 4 cinstcondi self-influence exhibits superior performance is comparable to self-influence in most cases. This observation is consistent with the result in the main paper.
Table 4: Comparison of bias-conflicting sample detection precisions hetween self-influence (ST), and bias- conditioned self-influence (BCSI) across various datasets. The average and the standard error of precision over three provided.
CMNIST
CIFARIOC
Method
NICO
0.5%
5%
0.5%
2%
5%
20%
50%
SI BCSI
31.94
39.35
37.91
23.33
20.19:2:2
42.17
41.1120.08
38.7320.30
89.37
82.86+1.a8 92.08
44.00
38.33
50.45+0.84
60.48
+9.48
71.78 90.86=0.80
18

---

D Performance with respect to the bias-conflicting ratio
In Figure 4.2 of the main paper, we showed the unbiased accuracy trends of the CIFARIOC dataset with respect to the bias-conflicinng ratio for SelecMix and SelecMix with our method. In Figure T0 we provide the CIFARIOC accuracy trends of LfF and DFA [3] alone and with our method.
80
80
Accuracy (%)
60
Accuracy (%)
60
50
50
Vanilla Vanilla+Ours
Vanilla
40
40
Vanilla+Ours
LFF
DFA
LFF +Ours
DFA AOurs
20
20
0.5
10
30 50
70
90
0.5
2
10
30
50
70
90
Bias confiicting ratio (%)
Bias conflicting ratio (%)
(a) The performance of LIF and Ours.
(b) The performance of DFA and Ours.
Figure 10: Performance of the other baselines and Ours on the CIFARIOC dataset with varying bias ratio. The performance of LfF 40 is shown in Figure 10(a) Figure T0(b displays the performance of DFA 31
E Bias-conflicting ratio of the pivotal set
We provide the resulting bias canss-ooffciin ratios (i.e, bias- detection precisions) of the pivotal set produced across a variety of datasets. Table 5 and Table show the bias-conflicting ratios for CMNIST (0.5%, 1 1%, 2%, 5%), CIFARIOC (0.5%, 1%, 2%, 5%, 20%, 30%, 50%, 70%), BFFHQ. Waterbirds, and NICO.
Table 5: The average and the standard error of detection precision three Note that pute the precision of the pivotal sets across varying ratios of bias-conflicting samples in CIFARIOC.
CIFAR I0C
0.5%
1%
2%
5%
20%
30%
50%
70%
Accuracy 45.57=1.63
68.18 86. 13 96.60+o.11
99.94 20.96 99.88=0.12 98.30+a.30
85.81=2.08
Table 6: The average and the standard error of detection precision over three runs. Note that we compute the precision of the pivotal sets CMNIST, BFFHQ, Waterbirds, and NICO.
CMNIST
BFFHQ
Waterbirds
NICO
0.5%
1%
2%
5%
0.5%
5%
Accuracy 68.19 84.61 11.6 97.2740.51
13.2416.55
66.32
64.2641.93
94.01 12.57
F
Results on natural language processing datasets
To verify the effectiveness of our method on NLP datasets, we conduct experiments on two widely used benchmarks, CivilComments and MultiNLI. CivilComments contains user-generated comments labeled as either toxic or non-toxic. The spurious attribute in this dataset indicates whether comment mentions one of the protected attributes, such as male, female, LGBB, black, white, Christian, Muslim, or other religions. These attributes disproportionately associated with toxic comments, creating
19

---

spurious correlation. Similarly, the MultiNII dataset comprises pairs of sentences with labels denoting their relationship as contradiction, entailment, or neutral. The spurious attribute in MultiNLI is the presence of negation words, which are more frequently observed in the contradiction class, Both datasets are structured into groups based on combinations of the label and the spurious attribute s, resulting in four groups in CivilComments and six in MultiNLI.
As shown in table our method demonstrates its effectiveness in further rectifying models previously debiased with JTT, achieving increases in worst-group accuracy of 3,4% and 14.8% on MultiNLI and CivilCommenss, respectively.
Table 7: The average and the worst-group accuracy on NLP datasets.
MultiNLI
CivilComments Avg.
Method
Avg.
Worst-group
Worst-group
ERM
82.4
67.9
92.6
57.4
JTT
80.0
70.2
92.6
63.7
Ours JIT
79.8
73.6
86.9
78.5
G Comparison of time costs
In this section, we analyze the time cost of our method and compare with the other baselines. For practical and tangible comparison, we measure the wall clock time for the CIFARI0C (0.5%) dataset. We run our experiments with a machine equipped with Intel Xeon Gold 5215 (Cascade Lake) processors, 252GB RAM, Nvidia GeForce RTX2080ti (1 IIGB VRAM), and Samsung 860 PRO SSD. For self-influence calculation, we utilize the JAX K library for fast Hessian vector product calculation. For all other deep learning functionalities, we utilize Pytorch 42). In Table & the wall-clock duration of each component our method is shown. We observe that the self-influence calculation step takes longer time compared to the fine-tuning step due to the intersection process. However, this can be executed in parallel, which reduces the time cost self-influence calculation approximately threefold. In Table 9 a wall-clock time comparison with the other baselines is shown. Our method consumes a significantly lesser amount of time, dropping to less than half the time of ERM full training when the self inffinnfueece caleulation is executed in parallel. Reflecting on these results, we assert that the time cost of our method is rather small or even negligible compared to the full training time of other baselines.
Table The average and the standard error of computational costs over three runs We measure the computing time for full training the wall-clock time of each component. Self-influence (parallel) represents calculating the cind-conditi seff-influence in GPU Paraalell.l. Note that t indicates that corresponding methods JAX while others utilize PyTorch.
Component Self-influence
Self-influence (paallel)
Fine-tuning
Time (min.) 11.46
3.86f 10.03
1.08
Table 9: The average and the standard error of computational costs over three runs. We measure the computing time for full training the wall-clock time of each method. Ours (parallel) presents method which computes bias-come bias-conditioned self- influence in GPU-parallel. Note that indicates that corresponding methods use JAX while others utilize PyTorch.
Method
ERM
LIF
DFA
SelecMix
Ours
Ours (ponille)
Time (min.) 22.55
33.64
53.18
352.53#n.18
12.54 +0.0s
4.94 +0.03
H Analysis of intersections within pivotal sets
In this section, we analyze the effects of intersections between pivotal sets obtained from various random initializations of models. For the comparison, we provide the number samples, detection precision, and performance after fine-tuning models across different numbers of the intersections
20

---

in Table 10, Table 11 and Table 12 We observe that the detection precision increases as the number of intersections rises, while the number of samples in the pivotal decreases. For the performance, higher number of intersections shows effectiveness in the highly-biased scenarios, as bias-conflicting samples are scarce, and intersections reduce the size of the pivotal set. contrast, fewer intersections exhibit superior performance in low-hased scenarios as there are abundant bias-conflicting samples. Note that, to observe the trend across varying ratios of bias bias-conflicting samples, conduct experiments on CIFARIOC (0.5%, 1 1%, 2%, 5%, 20%, 30%, 50%, 70%).
Table 10: The average and the standard error of the number of pivotal sets over three runs considering numbers of intersections.
Number of Intersections
CIFARIOC
0.5%
1%
2%
20%
30%
50%
70%
1
1000
1000
1000
1000
1000
1000
1000
1000
2
322.67
38..67
503.67.aw.rs
577.00178.46
554.00_s0.ow
421.67 12.17
309.67.m..o
290.00_7m..z
3
201.67 #4.91
267.00 388.33 19..0 a0..0
452.00
281.00 =01.00
144.67
141.67=m..p
Table 11: The average and the standard error of detection precision over three runs considering numbers of intersections.
Number of Intersections
CIFARIOC
0.5%
1%
20%
30%
50%
70%
13.27
27
47.071..65
76.27 10.50
97.10
97.60
91.27 12.02
10.8
2
31.6821.81
52.83 1.
75.17
92.01 20.80
99.77 19
99.0220.74
96.0020.17
83.68200.0
3
45.57=1.6a
68.18+0.0
16.13 96.60=a.11 99.94
99.88
98.30+0.30
85.01
Table 12: The average and the standard error of classification accuracy of 'Ours+SelecMix' over three runs considering numbers of intersections.
Number of Intersections
CIFARIOC
0.5%
1%
5%
20%
30%
50%
70%
20.44
40.76
49.57+0.41
59.31
67.99
67.04.0.65
67.39
70.09
2
38.85 10.62
43.47
10.21
51.43
60.22 10.19
66.96.0.15
65.90
15.901
66.77.1.40
69.92:a.5a
3
38.74
46.18
#o.33
52.70+0.40
59.66=0.s1
66.66=a.45
64.51
+1.44
66.45
69.97=a.a1
I Improving performance in low-bias settings
In CIFAR-1OC, as the bias severity decreases from 30% to 90%, the dataset gradually transitions into the low-bias domain, approaching an unbiased state 90%. This reduction undermines the ussumption that the bias is sufficiently malignant, reducing the effectiveness of debiasing methods and allowing ERM achieve better performance. In this context, to improve the performance of our method when applied to which leverages large number of conflicting samples- is necessary to increase the size of the pivotal set, thereby expanding the number conflicting samples. As shown in Table 13, expanding the pivotal set can improve performance in low-bias settings. This result implies that we could further enhance performance by adjusting the top-k value if we had access to information regarding bias severity.
21

---

Table 13: The average and the standard error of classification aceuracy three runs.
CIFARIOC
Method
30%
50%
70%
90%
ERM
t0.64
71.33
74.90=u.25
76.03
Ours EXM (k= 100)
65.61 +0.77
70.61 #0.62
73.20=u.33
73.57=t0.66
Ours ERM ER (k=2000)
71.25 1:.34
74.46.0.34
75.84
76.14.0..
J Qualitative analysis using Grad-CAM
This section provides qualitative results of method using Grad-CAM 47 BFFHQ and Waterbird. For BFFHQ, the target attributes are [young, old} and the bias attributes are (man, woman). For Waterbird, the target attributes are (waterbird, landbird), and the bias attributes (water, land]. In Figure (a) and (c), ERM focuses on biased features such as gender and background. However, ERM combined with our method tends focus more on task-relevant features including age-related facial features and the birds themselves. This implies that our approach effectively guides the model in prioritizing target attributes over biased ones.
ERM
(b) Ours ERM
(c) ERM
(d) Ours ERM
Figure 11: The Grad-CAl CAM of ERM and ERM+Ours on BFFHQ and Waterbird. (a-b) show results on BFFHQ, while (c-d) display results on Waterbird. (a) and (c) represent the Grad-CAMs for ERM, and (b) and (d) correspond to Grad-CAMs for ERM combined with our method.
22

---

K
Ablation study on the loss function of the detection model
In this section, conduct ablation study the learning objectives of the detection model. Our method uses Generalized Cross Entropy (GCE), a commonly adopted loss function in debiasing tasks acquire biased models. However, conceptually, our method can be applied to any loss function to obtain biased models. To demonstrate the generality of our approach with different loss functions, evaluate BFFHQ and Waterbird using alternative objectives, such GCE, SCE [56], and NCE+RCE (38] which are designed for handling noisy label environments, In Table [4] both SCE and NCE+RCE demonstrate performance comparable to GCE. Since these objectives encourage models to focus more on the majority samples, our method combined with these loss functions also achieves similar results. Note that, naive cross- entropyy which does not promote majority sample utilization, fails on the BFFHQ dataset.
Table 14: The average and the standard error over three runs.
BFFHQ
Waterbirds
Method
0.5%
5%
SelecMix
63.07
74.72=1.14
Ours w/ CE SelecMix
62.73
88.73
Ours w/ GCE SelecMix
63.8013.12
89.67
Ours w/ SCE SelecMix
66.2011.5s
89.46=00.36
Ours w/ NCE+RCE SelceMax
67.73+1.99
89.72=0.41
Ablation study on Influence estimation methods
We conduct an ablation study on other influence estimation methods. We leverage the fundamental form of Influence Functions to demonstrate the generalizability of our approach. However, other estimation methods are compatible. To further show this, we evaluate our method on BFFHQ and Waterbird using MoSo 50). TracIn
4. and Arnoldi 466 As shown Table TracIn outperforms
the basic IF, while MoSo and Arnoldi exhibit comparable performance. These results indicate that our method can enhance performance across various estimation approaches.
Table 15: The average and the standard error of detection precision over three runs.
BFFHQ
Waterbirds
Method
0.5%
5%
SelecMix
63.07+2.32
74.72=1.14
Ours SeleMMix
65.80
89.67
Ours w/ MoSo SelecMix.
63.1313.27
89. 89.72.1111
Ours w/ TracIn SoleeMix
90.39 69.20
Ours w/ Arnoldi SeleeMix 66.40+3.12
71.08
M Evaluation with fairness metrics
To further demonstrate the effectiveness of our method, we evaluate it using fairness metrics including
demographic parity (DP) 5 and equalized odds equal opportunity (EOP) [3] on Waterbird. In Table 16, our method significantly improves performance in both DP and EOP, indicates that our method also addresses the faimess problem.
23

---

Table 16: The average and the standard error of demographic parity (DP) and equalized odds equal opportunity (EOP) over three runs.
Waterbirds
Method
DP (L)
EOP (L)
ERM
0.182610.0044 0.2731 10.0187
SelecMix
0.1146
0. 081885 00.0100
Ours SelecMix
0.024210.0055 0.0099 000004
Experimental settings
N.1
detailed description of benchmark datasets
/
2 3
4
5
6
7 7
8 8
9
0
2
3
4
5
6
9
2
3
4
5
7
8
9
Colored MNIST.
(b) Corrupted CIFARIO.
Figure 12: Example images of CMNIST and CIFARIOC. Images in the first and second rows are bias aligned and images in the third row are bias- caas-oofflictin.
(a) Biased FFHQ
(b) Waterbirds.
Figure 13:
Example images of BFFHQ and Waterbirds. The red-bordered images are bias- dlas-aligned and the blue-bordered images are hias-confficting
Colored MNIST. Colored MNIST (CMNIST) is a synthetically modified version of MNIST (6] where the digit is the label and the color is the bias. For example, an image of digit 0 correlated with the color red. We use the following bias-conflicting ratios: r E {0.5%. 1 1%, 2%, 5%). The images are in x 28 resolution and are resized 32 32. There are approximately 55,000 training, 5,000 validation, and 10,000 test samples. Examples are shown in Figure 12(a)
synthetically modified version of CIFARIO [30 proposed by Hendrycks and Dietterich [17] with the following common corruptions as the bias: (Snow, Frost, Fog, Brightness, Contrast, Spatter, Elastic transform, JPEG, Pixelate and Saturate). We use the following bias-conflicting ratios: r E (0.5%, 1%, 2%, 5%, 20%. 30%, 50%, 90% (unbiassss))) The images are in 32 x resolution. There are approximately 45,000 training, 5,000 validation, and 10,000 test samples. Examples are shown in Figure (2(b)
Corrupted CIFAR10. Corrupted CIFAR10 (CIFARIOC)
Biased FFHQ. Biased FFHQ (BFFHQ) 31 is curated Flickr-Faces-HHC (FFHQ) 22 dataset, which consists of images human faces. The designated task label the age fyoung, old} while the bias attribute is the gender {man, woman]. The bias-confiictin ratio is e (0.5%). The images

---

are in 128 x 128 resolution and are resized to 224 x 224. There are approximately 20,000 training. 1,000 validation, and 1,000 test samples. Examples are shown in Figure 33a)
Waterbirds. Waterbirds is proposed by Sagawa et al. which synthetically combines bird images from the Caltech- UCCD Birds-200-2011 (CUB) with place background as bias. It consists of bird images to classify bird types (waterbird, landbird], but their backgrounds {water, land} are correlated with bird types. The bias-conflicting ratio E (5%}. The images are in varying resolutions and are resized to 224 224. There are approximately 5,000 training, 1,000 validation, and 6,000 test samples. Examples are shown in Figure 33(b
NICO. NICO is a dataset designed to evaluate non I.I.D classification by simulating arbitrary distribution shifts. To evaluate debiasing methods, a subset composed of animal classes label is utilized, as in (54). The class labels (e.g. "dog") are correlated to spurious contexts (e.g. "on grass", "in water", "in cage", "eating". "on beach". "lying", "running") which exhibits long- tail distribution. The images are in varying resolutions and are resized to 224 224. There are approximately 3,000 training, 1,000 validation, and 1,000 test samples.
N.2 Baselines
We validate our method by combining various debiasing approaches. ERM is the model trained by cross-entropy loss. GroupDRO [44] minimizes the worst-group loss by exploiting group labels directly. LIF 40] detects bias-conflicting samples and allocates large loss weights on them. DFA [3]
augments diverse features by swapping the features obtained from the biased model and concatenating the feature from the debiased model with the exchanged feature. BPA [8]] utilizes clustering method to identify pseudo-attribute cluster size and its loss. DCWP [4]] debiases network by pruning biased neurons. SelecMix [19] identifies and mixes bhas-contradicting pair within the same class while detecting and mixing a bias-aligned pair from different classes. Note that we adopt SelecMix+Lff rather than SelecMix since SelecMix+Lff exhibits superior performance than SelecMix 19].
using a clustering approach and adjusts loss weights according to the
N.3 Evaluation protocol
We provide experimental setups for evaluation. We use JAX 3 and PyTorch 422 for the experiments.
We conduct our experiments with a machine equipped with Intel Xeon Gold 5215 (Cascade Lake) processors, 252GB RAM, Nvidia GeForce RTX2080ti (IIGB VRAM) (or Nvidia GeForce RTX3090
(24GB VRAM).. and Samsung 860 PRO SSD. constructing pivotal sets, we adopt ResNet18 [5] as the base architecture for all datasets. For optimization, learning rate of 0.001, and train the models for 5 epochs. To calculate self-influence, we only utilize the last layer of the models. In fine-tuning, we deploy ResNet18 for CMNIST, CIFAR10C, BFFHQ, and NICO while ResNet50 is used for Waterbirds as following other baselines 40
employ the Adam optimizer (25 with
31 36). We adopt
the Adam optimizer for CMNIST, CIFAR1OC, BFFHQ, NICO while SGD is used for Waterbirds. For the learning rate, we use 0.001 for CMNIST, CIFAR OC, Waterbirds, and 10-1 for BFFHQ. We apply cosine annealing (37) to decay the learning rate to 10-3 of the initial value. We utilize weight decay of 10-4 for all datasets. We fine-tune the pre-trained models for 100 iterations. We set A = 0.1 for all experiments. For baselines [31 19], we use the officially released codes. For our method, we adopt k = 100, x= = 0.1 for all datasets.
N.4 Licenses for existing assets
Flickr-Faces-HH (FFHQ) 22 is high-quality image dataset of human faces, originally created as a benchmark for generative adversarial networks (GAN). The individual images were published in Flickr by their respective authors under either Creative Commons BY 2.0, Creative Commons BY-NC 2.0, Public Domain Mark 1.0, Public Domain CCO 1.0, or U.S. Government Works license. NICO [16] dataset does not own the copyright of images. Only researchers and educators who wish to use the images for non-commercial researches and/or educational purposes, have access to NICO.
JAX 3 has Apache License.
25

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification:
We reflect all our assertions and contributions on Abstach and Introduction.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results
be expected to generalize other settings.
is fine to include aspirational goals as motivation as long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification:
We discuss the limitations of our work in Section i
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model sell-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken low lighting. Or speech-to-text system might be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and faimess.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory Assumptions and Proofs
Question:
For each thcoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
26

---

Justification:
We do not include theoretical results in our paper.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless whether the code and data are provided or not)?
Answer: Yes)
Justification:
We provide all the experimental details in Section 5.1 and Appendix N.3
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make clear how to reproduce that algorithm. (b) the contribution is primarily a new model architecture, the paper should describe the architccture clearly and fully.
(c) If the contribution is new model (e.g., a large language model), then there should either way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reprodueibility. the case of closed-source models, may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
27

---

Answer: [Yes]
Justification:
We use benchmark datasets that are open to the public and provide codes with supplements.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips.cm ec/
for more details.
While encourage the release of code and data, understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https
//nips.cc/pp wwtpcccpppppppp.eeeeeeeeeeeeeeee wwwpc//ggieeeeeeee http://oooSSSeiiisseeeeeee issiiissiiccyyyiii for more details.
The authors should provide instructions data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification:
We provide all the experimental details in Section 5.1 and Appendix N.3
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment Statistical Significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer:
[Yes]
Justification:
We report error bars and use standard errors. We also state them in Section 5.1
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the enror bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
28

---

It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative enror rates).
If error bars reported in tables plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments Compute Resources
Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Answer:
[Yes]
Justification: We provide machine information in Appendix N.3 and computational costs in Appendix G
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (c.g., preliminary or failed experiments that didn't 't make into the paper).
9. Code Of Ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics
Answer: [Yes]
Justification:
We read and agree with NeurIPS Code of Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws or regulations in their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer:
[Yes]
Justification:
We discuss broader impacts in Section
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
29

---

The conference expects that many papers will be foundational research and not tied particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms monitor how system learns from feedback over time, improving the efficiency and accessibility of MI..
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification:
We conduct experiments on recognition models and use benchmark datasets.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Lieenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: We cite the source of data and models in Section 5.1 6 and Appendix N
Guidelines:
The answer NA means that the paper does not existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (c.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
30

---

If this information is not available online, the authors are encouraged to reach out to the asset's 's creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification:
We would not release assets.
Guidelines:
The answer NA means that the paper does not release new assets,
Researchers should communicate the details of the dataset/code/moodl as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification:
We use benchmark datasets and compute accuracy or precision on them.
Guidelines:
The answer NA means that the paper does involve crowdsoureing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According the NeurIPS Code of Ethies, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification:
We use benchmark datasets and compute accuracy or precision on them.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent) may required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
31


Reviews:
[{"review_id": "EkVvrlzTAD", "reviewer": "Reviewer_xCjP", "summary": "The authors propose a method to tackle spurious correlations by using influence functions. Specifically, they compute the self-influence on the training set -- the amount that a particular sample's loss changes when it is removed from the training set. Samples with the highest self-influence are then assumed to be in the bias-conflicting (minority group). Then, models can be finetuned, up-weighting this set of identified samples, to obtain a debiased model. The authors benchmark their method on typical spurious correlation benchmarks, finding that they outperform the baselines.", "questions": "1. Was there any model selection required in the experiments? If so, how was it done?\n\n2. Why were some baselines omitted from Tables 2 and 3?", "limitations": "The authors have adequately addressed the limitations.", "rating": 5, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "- The paper is well-written and easy to follow.\n- The proposed method is intuitive, and \n- The method outperforms the baselines on typical benchmark datasets.", "weaknesses": "1. The proposed method has no theoretical justifications, and so it is unclear under what circumstances it would fail. \n\n2. The authors should evaluate on a few more datasets from the spurious correlation domain, such as CelebA, MultiNLI, and CivilComments. It would be particularly important to demonstrate that the method can work on text datasets. The authors should also add JTT [1] as a baseline.\n\n3. If compute allows, the authors should compute the ablations (Figure 6) for all other datasets. In addition, it would be interesting to show particular samples in the top-k set. Do the samples differ visually as their ranking decreases? How do the samples which are in the top-k set but are not bias-conflicting look?\n\n4. Once the candidate set of bias conflicting samples is identified, there are many other approaches that could be taken. The authors use a simple upweighting approach, but one could e.g. apply GroupDRO to the dataset, with two groups, as well. The authors should benchmark a few of these alternatives.\n\n5. In Table 1, the proposed method only outperforms the baselines when it is used to finetune a model which has had another debiasing approach applied to it, i.e. Ours_ERM underperforms the baselines. Thus, the method requires a decent starting point to work well.\n\n6. The authors should discuss some of the failure modes of the method. One potential failure mode seems to be when the dataset actually contains mislabeled samples, and so the identified set consists of mislabeled samples instead of bias-conflicting samples. The authors should benchmark their method in these settings, potentially with synthetic noise.\n\n[1] https://arxiv.org/pdf/2107.09044", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely appreciate your time and effort to review our paper.\n\n---\n\n### [Q1] The proposed method has no theoretical justifications, and so it is unclear under what circumstances it would fail.\nAlthough we did not provide theoretical justification, we demonstrated the effectiveness of our method across various settings and datasets. For the circumstances where our method would fail, our method might fail when there is a significant number of mislabeled samples. Specifically, the intuition behind our method is that generic features for bias-conflicting samples are learned later in the training process, and by using the model in its early stage, we can identify these samples through self-influence. If mislabeled samples are present in the dataset, these samples would also exhibit high self-influence, making bias-conflicting samples difficult to distinguish. \n\n---\n\n### [Q2] The authors should evaluate on a few more datasets from the spurious correlation domain. The authors should also add JTT as a baseline.\nWe conduct an additional experiment on CelebA, including JTT as a baseline. As shown in the table below, our method significantly improves the performance of JTT, exhibiting the effectiveness of our method on datasets with spurious correlations. We are currently running experiments on the MultiNLI and CivilComments and will report the results during the remaining discussion period.\n|CelebA|Avg Acc.|Worst-group Acc.|\n|---|---|---|\n|ERM|95.32$\\pm$0.34|45.19$\\pm$0.67|\n|JTT|90.14$\\pm$0.61|72.22$\\pm$2.51|\n|JTT+Ours|86.19$\\pm$1.32|**80.17$\\pm$1.29**|\n\n---\n\n### [Q3] If compute allows, the authors should compute the ablations (Figure 6) for all other datasets.\n\nFor the ablation study, we will report the results for other datasets during the remaining discussion period.\n\n---\n\n### [Q4] In addition, it would be interesting to show particular samples in the top-k set. Do the samples differ visually as their ranking decreases? How do the samples that are in the top-k set but are not bias-conflicting look?\n\nIn Figure 2 of the global response, we provide examples of bias-conflicting and bias-aligned samples from BFFHQ, based on their BCSI scores. All images belong to the top 100 samples according to BCSI. Specifically, (a) shows bias-conflicting samples with high BCSI. (b) indicates bias-conflicting samples with low BCSI. (c) denotes bias-aligned samples with high BCSI. Note that, in BFFHQ, the target attribute set is {young, old} and the bias attribute is {man, woman}. There is a spurious correlation as young women and old men. First, we will analyze only the first row of Figure 2. In (a), the bias-conflicting sample with high BCSI depicts old women, who appear younger compared to the woman in (b). Additionally, bias-aligned samples with high BCSI in (c), depict older men who appear more like women due to makeup, thus having higher BCSI. \n\n---\n\n### [Q5] Once the candidate set of bias-conflicting samples is identified, there are many other approaches that could be taken. The authors use a simple upweighting approach, but one could e.g. apply GroupDRO to the dataset, with two groups, as well.\n\nThank you for your constructive suggestion. To demonstrate the generalizability of our method, we leverage the most fundamental approach of upweighting. As the reviewer mentioned, various methods to leverage the candidate set are certainly applicable. (However, since bias labels are still not provided, employing other methods like GroupDRO would require additional modules.)\n\n---\n\n### [Q6] In Table 1, the proposed method only outperforms the baselines when it is used to finetune a model which has had another debiasing approach applied to it, i.e. Ours_ERM underperforms the baselines. Thus, the method requires a decent starting point to work well.\n\nAs we consistently fine-tune models for only a few iterations across all models due to the unknown severity of bias, this approach may be insufficient for heavily biased models like ERM. We have empirically observed that additional iterations lead to further performance improvements in such cases. Furthermore, a key advantage of our method is that it is complementary to existing methods and can further rectify models that have already undergone recent debiasing techniques.\n\n---\n\n### [Q7] The authors should discuss some of the failure modes of the method. One potential failure mode seems to be when the dataset actually contains mislabeled samples, and so the identified set consists of mislabeled samples instead of bias-conflicting samples.\nAs the reviewer mentioned, when the dataset contains mislabeled samples, our method might fail to detect bias-conflicting samples because both bias-conflicting samples and mislabeled samples possess high self-influence values. We will discuss this in our revision. However, this issue is not unique to our method; many recent debiasing methods also fail in the presence of mislabeled samples, as they amplify the learning signal of samples based on loss or gradient values, which can mislead models under these conditions.\n\n---\n\n### [Q8] Was there any model selection required in the experiments? If so, how was it done?\nWe used the same hyperparameters across all datasets and settings in the main experiments and consistently selected the model from the last epoch (The hyperparameters are provided in Appendix H.3). If an unbiased validation set were available, hyperparameters could be tuned, potentially leading to further performance improvements.\n\n---\n\n### [Q9] Why were some baselines omitted from Tables 2 and 3?\nThese baselines were omitted from Tables 2 and 3 because they did not conduct experiments on these benchmark datasets, requiring us to perform the hyperparameter search to obtain reasonable performance. Additionally, these methods exhibit a significant performance gap compared to the state-of-the-art method. To further demonstrate the effectiveness of our method, we will report the results in the remaining discussion phase.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "I1WVHS03JD", "reviewer": "Reviewer_BCqo", "summary": "The authors focus on detecting bias-conflicting samples to recover biased models. They propose a Bias-Conditioned Self-Influence to help identify bias-conflicting samples in the early stage of model training. Experiments on public datasets are conducted to demonstrate the effectiveness of the proposed method.", "questions": "Please see the Weaknesses part and kindly correct me if there are any misunderstandings.", "limitations": "N/A", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "1. The introduced perspective of using a bias-conditioned self-influence for bias-conflicting sample detection is interesting;\n\n2. The experimental results look promising.", "weaknesses": "**Majors:**\n\n1. The paper aims to rectify bias within a model. However, only the accuracy of models and distribution of BCSI scores are provided. Additional experiments are needed to demonstrate that the bias within a model could be reduced by the proposed method.\n\n2. The authors employ Generalized Cross Entropy to get a more biased model. What about the performance with other losses?\n\n3. Why does ERM perform better than the proposed method in Table 2? An insightful analysis is needed.\n\n4. I suggest the authors to reorganize the paper to make it easier to follow. See minors for details.\n\n\n**Minors:**\n\n1. Table 2 is mentioned before Table 1;\n\n2. Figure 2 mentioned before Figure 1;\n\n3. Experimental settings are mentioned in the technical part ($\\lambda=0.1$).\n\n5. There are some typos. For example, on Page 3, Line 102: A analysis of ...", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank you for the constructive comments.\n\n---\n\n### [Q1] The paper aims to rectify bias within a model. However, only the accuracy of models and distribution of BCSI scores are provided. Additional experiments are needed to demonstrate that the bias within a model could be reduced by the proposed method.\n\nTo further demonstrate the effectiveness of our framework in reducing model bias, we have conducted Grad-CAM [1] analysis on the BFFHQ and Waterbird datasets. In the BFFHQ dataset, the target attribute set is {young, old} and the bias attribute set is {man, woman}. For the Waterbird dataset, the target attribute set is {waterbird, landbird} and the bias attribute set is {water, land}.\n\nAs shown in Figure 1 of the attached PDF in the global response, the biased models (a) and (c) tend to focus on biased attributes such as gender and background. However, when applying our method, as illustrated in (b) and (d), the model's attention shifts to more task-related features, such as age in faces and bird species. This indicates that our method effectively redirects the model’s focus away from biased attributes and toward the target attributes.\n\n[1] Selvaraju et al. \"Grad-cam: Visual explanations from deep networks via gradient-based localization.\" CVPR, 2017.\n\n---\n\n### [Q2] The authors employ Generalized Cross Entropy to get a more biased model. What about the performance with other losses?\n\nWe utilize Generalized Cross Entropy (GCE) since it is commonly used in the debiasing domain to obtain a biased model. However, other loss functions can also be viable alternatives. To demonstrate this, we conducted additional experiments on the BFFHQ and Waterbird datasets, employing other loss functions that are follow-ups to GCE, such as SCE [2] and NCE+RCE [3], which are designed for noisy label settings.\n\nAs shown in the table below, we present the performance of our method with each applied loss function. Both SCE and NCE+RCE exhibit performance comparable to or slightly better than GCE in our method. These loss functions encourage the model to focus more on the majority of normal samples rather than the minority noisy ones, which also results in a more biased model in the given bias setting.\n\n|              |  BFFHQ | Waterbird |\n|----------------|-----------------|----------------|\n| SelecMix | 63.07± 2.32    | 74.72 ± 1.14   |\n| SelecMix + Ours_CE |  62.73 ± 3.71 | 88.73 ± 0.45 |\n| SelecMix + Ours_GCE | 65.80 ± 3.12 | 89.67 ± 0.38 |\n| SelecMix + Ours_SCE | 66.20 ± 0.53 | 89.46 ± 0.36 |\n| SelecMix + Ours_NCE+RCE |  **67.73 ± 1.99** | **89.72 ± 0.41**  |\n\n\n[2] Wang et al. \"Symmetric cross entropy for robust learning with noisy labels.\" ICCV, 2019.\n\n[3] Ma et al. \"Normalized loss functions for deep learning with noisy labels.\" ICML, 2020.\n\n---\n\n### [Q3] Why does ERM perform better than the proposed method in Table 2? An insightful analysis is needed.\n\nRecent debiasing methods are typically designed under the assumption that the bias is malignant enough to mislead a model into extensively relying on the bias to produce a biased predictor. Consequently, in the cases of 70% and 90% in Table 2, the dataset is nearly an unbiased set, breaking this assumption. This leads to the opposite effect, where important samples for learning are disregarded. Ensuring robust performance even when given such unbiased datasets remains an important future goal for the debiasing community.\n\n---\n\n### [Minor issues] \nAs the reviewer suggested, we will revise the organization of the paper and correct any typos to make it easier to follow.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "XeNsE3wDaV", "reviewer": "Reviewer_yem8", "summary": "This paper tackles the problem of learning generalized models from biased data by detecting mislabeled samples. The authors use BCSI (SI estimated on a trained model with GCE) to detect bias-conflicting samples and construct a pivotal subset based on the BCSI scores of these samples for correcting the biased model without access to a clean validation set.", "questions": "From my understanding, the pivotal subset is used to correct and mitigate bias in the model. So why do the authors mention “use pivotal set (bias-conflicting samples) to recover a biased model”? What the paper actually does is use bias-conflicting samples to correct the biased model. This is very confusing. Is this a typo? Or should it be rephrased to “recover an unbiased model from a biased one” for better clarity?", "limitations": "The authors have adequately addressed the limitations. Since this work is mainly about dataset bias, so I do not see any negative societal impact.", "rating": 5, "confidence": 5, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "- Originality and Significance: Rather than computing influence scores on a clean validation set (which is usually not feasible to obtain in practice), this paper uses self-influence estimated by the model trained with GCE to detect bias-conflicting samples.\n- Quality and Clarity: Since the proposed method heavily relies on the assumption that if the loss increases when a sample is removed from the training set, that sample is likely to be mislabeled, the empirical analysis of self-influence (SI) in detecting bias-conflicting samples is sufficient across four different tasks. The experiments, including ablation studies, are comprehensive.", "weaknesses": "- The concept of bias-conflicting samples is not equivalent to unbiased samples, so in the introduction, *“by first identifying bias-conflicting (unbiased) samples”* should be rephrased.\n- Some grammar errors:\n  - Section 3: “A analysis” => “An analysis”.\n  - “trainset” => “training set”.\n- The bias issue introduced in this paper is more about robustness rather than fairness since no sensitive attributes (such as gender or race) are included in the problem formulation, and no fairness evaluation (demographic parity or equalized odds) is conducted in the experiment. I recommend the authors change the terminology from fairness to robustness for a more precise expression.\n- In Section 3.2, I strongly recommend introducing the concepts of “mislabeled samples,” “bias-conflicting samples,” and “bias-aligned samples” in Section 2.1 for better readability.\n- In *“Note that since an unbiased validation set is unavailable in our target problem, we additionally estimate the influence score on the training set, indicated as IF in Figure 2.”* The measure of IF is problematic since it should be obtained via a clean validation set, which is not equivalent to the training set. The authors could conduct the experiment in a noisy label setting, for example, treating a subset as a clean validation set and flipping the others with a noise rate.\n- *“GCE emphasizes samples that are easier to learn, amplifying the model’s bias by giving more weight to bias-aligned samples in the training set.”* GCE does prioritize samples that are easier to learn, but this does not necessarily mean that these samples are all bias-aligned.\n- The proposed method BCSI in Equation 2 is just SI, it appears to have limited novelty.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We appreciate your valuable feedback.\n\n---\n\n### [Q1] The bias issue introduced in this paper is more about robustness rather than fairness since no sensitive attributes (such as gender or race) are included in the problem formulation, and no fairness evaluation (demographic parity or equalized odds) is conducted in the experiment. I recommend the authors change the terminology from fairness to robustness for a more precise expression.\n\nAs the reviewer mentioned, the term robustness is more precise since our target problem is addressing malignant biases (spurious correlations) in the dataset that prevent the model from learning task-related features.\n\nHowever, there is also an intersection with fairness, especially when considering cases where the malignant bias involves sensitive attributes such as gender or race. Numerous related studies addressing spurious correlations [1,2,3,4,5,6] have shown that mitigating these correlations aids in achieving fairness. For example, in the Biased FFHQ (BFFHQ) dataset, which is one of the benchmark datasets consisting of images of human faces, the designated task label is age {young, old} while the bias attribute is gender {man, woman}. In addition, we conduct experiments on CelebA, a more widely used benchmark dataset in the fairness domain. As shown in the table below, our method also demonstrated effective performance on this dataset.\n\n|   CelebA             |  Averaged Acc. | Worst Group Acc. |\n|----------------|-----------------|----------------|\n| ERM| 95.32 ± 0.34    | 45.19 ± 0.67   |\n| JTT | 90.14 ± 0.61    | 72.22 ± 2.51   |\n| JTT + Ours   | 86.19 ± 1.32    | **80.17 ± 1.29**   |\n\n\n[1] Nam et al. \"Learning from failure: De-biasing classifier from biased classifier.\" NeurIPS, 2020.\n\n[2] Lee et al. \"Learning debiased representation via disentangled feature augmentation.\" NeurIPS, 2021.\n\n[3] Seo et al. \"Unsupervised learning of debiased representations with pseudo-attributes.\" CVPR, 2022.\n\n[4] Hwang et al. \"Selecmix: Debiased learning by contradicting-pair sampling.\" NeurIPS, 2022.\n\n[5] Park et al. \"Training debiased subnetworks with contrastive weight pruning.\" CVPR, 2023.\n\n[6] Deng et al. \"Robust Learning with Progressive Data Expansion Against Spurious Correlation\", NeurIPS, 2023.\n\n---\n\n### [Q2] In Section 3.2, I strongly recommend introducing the concepts of “mislabeled samples,” “bias-conflicting samples,” and “bias-aligned samples” in Section 2.1 for better readability.\n\nTo aid readers who may not be familiar with this terminology, as the reviewer suggested, we will introduce these concepts in Sec 2.1 for better readability.\n\n---\n\n### [Q3] In “Note that since an unbiased validation set is unavailable in our target problem, we additionally estimate the influence score on the training set, indicated as IF in Figure 2.” The measure of IF is problematic since it should be obtained via a clean validation set, which is not equivalent to the training set. The authors could conduct the experiment in a noisy label setting, for example, treating a subset as a clean validation set and flipping the others with a noise rate.\n\nIn Figures 2 and 3, the notation $IF$ refers to the Influence Function on the training set, as specified in the captions. Please note that IF on the training set is the naive baseline in our experiments, substituting for IF on the unbiased validation set. While IF on the unbiased validation set would be a stronger baseline, it is not available in our target problem. To avoid any misunderstanding, we will modify the notation to $IF_{train}$.\n\n---\n\n### [Q4] “GCE emphasizes samples that are easier to learn, amplifying the model’s bias by giving more weight to bias-aligned samples in the training set.” GCE does prioritize samples that are easier to learn, but this does not necessarily mean that these samples are all bias-aligned.\n\nWe apologize for any confusion. We agree that GCE does not necessarily imply that all these samples are bias-aligned, and our intention was to highlight this tendency. We will revise the statement to: \"GCE emphasizes samples that are easier to learn, thereby amplifying the model’s bias by tending to give more weight to bias-aligned samples in the training set.”\n\n---\n\n### [Q5] So why do the authors mention “use pivotal set (bias-conflicting samples) to recover a biased model”?\n\nWe apologize for the confusion. Our intention was to convey the meaning of \"rectify a biased model.\" We will revise it.\n\n---\n\n### [Minor issues]\n\nAs the reviewer suggested, we will revise grammar errors, and rephrase “by first identifying bias-conflicting (unbiased) samples”  in the revision.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "9i4IcoDb7G", "reviewer": "Reviewer_hyta", "summary": "The paper proposes a method for debiasing biased models through fine-tuning on a specific subset with a high portion of unbiased (\"bias-conflicting\") samples. The paper uses self-influence in the early training epochs to identify this specific subset, where high self-influence is an indicator of a bias-conflicting sample. The intuition behind this approach is that models tend to learn biases early in training, so that samples without the bias feature will have high self-influence scores.", "questions": "- Self-influence is often used in mislabeled sample detection as a way to evaluate training data attribution methods. Still, SI mainly indicates that a sample is OOD (which could be a case of mislabeling). Does your approach generalize to OOD cases, then, too?\n- What is the difference between IF and SI is? SI has been used in IF and data attribution work commonly as an evaluation protocol to find mislabeled samples. \n- Equation 2: It makes sense to detect bias-conflicting samples early in training. Yet, at this stage the model has yet to converge and I can imagine that the estimation quality of the inverse Hessian vector product for self-influence is rather low and uses a large damping factor. What is the damping factor used?\n- How does the estimation quality influence your method and results? Would a dynamic training data attribution approach like e.g. TracIn (computing influence as a sum of gradient dot products across training) be an alternative? \n- How do you compute self-influence/what kind of HVP estimation algorithm do you use?\n- Lines 203-205: How many samples are in the intersection? Is there high variance in the self-influence?\n- Line 330 Broader impact: If only biased data is available, how can bias-conflicting samples be found?", "limitations": "Limitations and broader impact are discussed in the conclusion.", "rating": 6, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "- The paper is well-written.\n- The hypothesis of fine-tuning on a subset of bias-conflicting samples is grounded on a thorough analysis.", "weaknesses": "- There are some unclarities in the method. Please see questions. \n- As \"bias-aligned\" and \"bias-conflicting\" is new terminology, it is slightly hard to follow as a reader. Adding an example image instead of the description in section 3.2 would help this. \n- Formatting: \n     - Figure 2: Please add the error bars from your 3 runs. \n     - Figure 3: For easier comparability of (b) and (c) please use the same y-axis., same for the histogram’s x-axis of (d).\n     - Figure 4: Please use the same x-axis across the subplots. Using your 3 runs, it would be interesting to add error bars to the plot, too. \n     - “Figure x” and “Table x” font sizes are larger than the caption font size. \n     - Table 1 caption: space between the cross symbol and “denotes” missing \n- Minor:\n     - Line 33: It’s unclear what “it” refers to.\n     - Line 79: Please also include the citation Hampel 1974 that introduced Influence Functions from Robust statistics.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank you for your constructive comments.\n\n---\n\n### [Q1] Self-influence is often used in mislabeled sample detection as a way to evaluate training data attribution methods. Still, SI mainly indicates that a sample is OOD (which could be a case of mislabeling). Does your approach generalize to OOD cases, then, too?\n\nYes, our approach generalizes to OOD cases. The core idea of using Self-Influence in mislabeled sample detection is to identify minority samples that contradict the dominant features learned by the model. Since OOD samples, including mislabeled ones, also exhibit a contradiction due to their incorrect labels or the absence of the dominant features learned by the model, our approach is applicable to them as well.\n\nIn terms of the degree of contradiction, although we have not thoroughly validated this, it is reasonable to predict the following order: mislabeled samples > non-mislabeled OOD samples > bias-conflicting samples. Since bias-conflicting samples share task-related features with the majority, the degree of contradiction is weaker than that of OOD samples, making their detection a significant challenge.\n\n---\n\n### [Q2] What is the difference between IF and SI is?\nIn Figures 2 and 3, the notation $IF$ refers to the Influence Function on the training set, as specified in the captions. Please note that IF on the training set is the baseline in our experiments, substituting for IF on the unbiased validation set, which is unavailable in our target problem. To avoid any misunderstanding, we will modify the notation to $IF_{train}$.\n\nTo elaborate, typically, if a validation set that resembles the distribution of the test set is available, it is used as $z'$ in Influence Function $I(z, z′)$. However, in challenging situations where an unbiased validation set is not available, Self-Influence $I(z,z)$ is used by setting $z’$ as the training sample $z$ itself instead of the validation set. SI measures how the prediction of $z$ changes when $z$ is excluded from the training set. This means that the higher contradiction the sample $z$ exhibits, the more difficult it becomes for the model to make accurate predictions based on features learned from other samples.\n\n---\n\n### [Q3] Equation 2: It makes sense to detect bias-conflicting samples early in training. Yet, at this stage, the model has yet to converge and I can imagine that the estimation quality of the inverse Hessian vector product for self-influence is rather low and uses a large damping factor. What is the damping factor used?\n\nAs the reviewer mentioned, since we use an early-stage model, the estimation quality of the inverse Hessian vector product is to some extent insufficient. To compensate for this, we follow a widely used convention by adding the absolute value of the smallest negative eigenvalue plus 0.0001 to avoid negative eigenvalues. Without additional tuning of the damping factor, this setting has empirically shown good performance across various benchmark datasets in our experiments. However, resolving the issue of insufficient estimation quality due to using an early-stage model is crucial for further performance improvements, making it a primary objective in future work.\n\n---\n\n### [Q4]  How does the estimation quality influence your method and results? Would a dynamic training data attribution approach like e.g. TracIn (computing influence as a sum of gradient dot products across training) be an alternative?\nWe leveraged the fundamental form of Influence Functions to demonstrate the generalizability of our approach. Of course, other forms of Influence function, such as dynamic training data attribution methods like TracIn[1], can also be viable alternatives.\n\nTo further demonstrate this, we have conducted additional experiments on BFFHQ and Waterbird using TracIn and MoSo [2], a recent method that leverages gradients from the training process. As shown in the table below, using TracIn results in better performance compared to the basic form of IF, and MoSo demonstrates comparable performance.\n\n|                | BFFHQ           | Waterbird      |\n|----------------|-----------------|----------------|\n| SelecMix       | 63.07 ± 2.32    | 74.72 ± 1.14   |\n| SelecMix + Ours    | 65.80 ± 3.12    | 89.67 ± 0.38   |\n| SelecMix + Ours_MoSo   | 63.13 ± 3.27    | 89.72 ± 1.12   |\n| SelecMix + Ours_TracIn | **69.20 ± 0.50**    | **90.39 ± 0.70**   |\n\n[1] Pruthi et al. \"Estimating training data influence by tracing gradient descent.\" NeurIPS, 2020.\n\n[2] Tan et al. \"Data pruning via moving-one-sample-out.\" NeurIPS, 2023.\n\n---\n\n### [Q5] How do you compute self-influence/what kind of HVP estimation algorithm do you use?\nFor both efficiency and bias-focused computations, we calculate self-influence using only the last layer (classification layer), with the Hessian inverse computed exactly. By following [3], we limit the Hessian computation to the parameters of the last layer to reduce computational cost. Additionally, considering [4]'s discovery that retraining only the classification layer can achieve debiasing, focusing on the last layer allows for more bias-focused computations.\n\nIn the table below, we present experiments with Arnoldi [5], which estimates self-influence utilizing all the parameters in the model. The results show that Arnoldi performs comparably or worse, underscoring the effectiveness of using only the last layer.\n\n|                | BFFHQ           | Waterbird      |\n|----------------|-----------------|----------------|\n| SelecMix       | 63.07 ± 2.32    | 74.72 ± 1.14   |\n| SelecMix + Ours    | 65.80 ± 3.12    | **89.67 ± 0.38**   |\n| SelecMix + Ours_Arnoldi   | **66.40 ± 3.12**    | 71.08 ± 3.12   |\n\n[3] Daxberger et al. \"Laplace redux-effortless bayesian deep learning.\" NeurIPS, 2021.\n\n[4] Kirichenko et al. \"Last layer re-training is sufficient for robustness to spurious correlations.\" arXiv, 2022.\n\n[5] Schioppa et al. \"Scaling up influence functions.\" AAAI, 2022.\n\n---", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
We provide Grad-CAM visualizations [1] for both ERM and ERM+Ours on BFFHQ and Waterbird in Figure 1. We also include example images from the top 100 samples, as ranked by BCSI, in Figure 2.

[1] Selvaraju, Ramprasaath R., et al. "Grad-cam: Visual explanations from deep networks via gradient-based localization." CVPR, 2017.

Meta-review: The manuscript proposes a debiased method to remove the spurious correlation within training dataset. Inspired by the mislabeled sample detection, they proposed a simple and effective method to identify bias-conflicting samples and leverage them to improve the performance. Extensive results on diverse datasets validate the effectiveness of the proposed method.

All the reviewers' concerns have been addressed through rebuttal. Based on this, I recommend to present the paper on the conference.
Decision: Accept (poster)