A Simple Remedy for Dataset Bias via Self-Influence: A Mislabeled Sample Perspective

Authors: yeonsung  jung
jaeyun  song
june yong yang
jin-hwa  kim
sung-yub  kim
eunho  yang
Venue: NeurIPS_cc_2024_Conference
Published: 2024-05-15 13:20:49

Abstract: Learning generalized models from biased data is an important undertaking toward fairness in deep learning. To address this issue, recent studies attempt to identify and leverage bias-conflicting samples free from spurious correlations without prior knowledge of bias or an unbiased set. However, spurious correlation remains an ongoing challenge, primarily due to the difficulty in correctly detecting these samples. In this paper, inspired by the similarities between mislabeled samples and bias-conflicting samples, we approach this challenge from a novel perspective of mislabeled sample detection. Specifically, we delve into Influence Function, one of the standard methods for mislabeled sample detection, for identifying bias-conflicting samples and propose a simple yet effective remedy for biased models by leveraging them. Through comprehensive analysis and experiments on diverse datasets, we demonstrate that our new perspective can boost the precision of detection and rectify biased models effectively. Furthermore, our approach is complementary to existing methods, showing performance improvement even when applied to models that have already undergone recent debiasing techniques.

Main paper: 
 introduction: 
 Deep neural networks have demonstrated remarkable performance in various fields of machine learning tasks comparable to or superior to humans on well-curated benchmark datasets [7, 4, 60, 14] . Nevertheless, the efficacy of these models trained on unfiltered, real-world data remains an open question. In this scenario, a significant concern arises due to the presence of dataset bias [52] , where task-irrelevant attributes are spuriously correlated with labels only in the training set. This can lead to models that rely on misleading correlations rather than learning the task-related features, resulting in biased models with poor generalization performance [62, 10] .To prevent models from learning detrimental bias, various methods are proposed to encourage models to prioritize learning task-relevant features. Recent studies enhance task-related features by first identifying bias-conflicting (unbiased) samples through loss [40, 36] , gradients [1] , or bias prediction techniques [35] during training, using an auxiliary biased model trained with Empirical Risk Minimization (ERM). Then, they amplify bias-conflicting samples by counteracting the bias-aligned (biased) samples through loss weighting [40] or weighted sampling [35] . The effectiveness of such methods largely depends on their precision of bias-conflicting sample detection. Specifically, there is a risk of erroneously amplifying malignant bias instead of task-relevant features when bias-aligned samples are inaccurately identified as bias-conflicting. Due to the limited detection performance of previous methods [40, 36, 1, 35] , it presents a crucial challenge that remains unresolved.In this paper, we address this challenge from a novel perspective of mislabeled sample detection. Inspired by the similarities between mislabeled samples and bias-conflicting samples, we delve into Influence Functions (IF; [27] ), one of the standard methods for mislabeled sample detection [55, 57, 29] , to identify bias-conflicting samples and propose a simple yet effective approach for biased models by leveraging them.We first conduct a comprehensive analysis to explore the efficacy of Self-Influence (SI) [27] , a variant of IF, in biased datasets. SI estimates how removing a specific training sample during training influences the prediction of the sample itself with the trained model (see Section. 2.2). By measuring SI, we can identify a minority sample that, if removed from the training set, increases the likelihood of incorrect predictions of itself by the trained model due to their discrepancies with the majority samples. In this context, leveraging SI to biased datasets is promising as bias-conflicting samples constitute the minority and contradict the dominant malignant bias learned by the model. However, we observe that unlike in mislabeled settings, directly applying SI to biased datasets is not as effective (Figure 1 (a)-1(d)). Therefore, we investigate the differences between mislabeled samples and biasconflicting samples and reveal the essential conditions for SI to effectively identify bias-conflicting samples. Note that we denote SI under found conditions as Bias-Conditioned Self-Influence (BCSI).Building on our analysis, we propose a simple yet effective method for rectifying biased models through fine-tuning. We construct a small pivotal subset with a higher proportion of bias-conflicting samples using BCSI. While not perfect, this pivotal set can serve as an effective alternative to an unbiased set. Leveraging this pivotal set, we rectify a biased model through fine-tuning with only a few additional iterations. Extensive experiments demonstrate that our method can effectively rectify even after models are already debiased by recent methods.Our contributions are threefold:• We conduct a comprehensive analysis to explore the efficacy of SI in biased datasets and reveal the essential conditions for SI to accurately differentiate bias-conflicting samples, leading to Bias-Conditioned Self-Influence (BCSI). • We propose a simple yet effective remedy through fine-tuning that utilizes a pivotal set constructed using BCSI to rectify biased models across varying bias severities. • Our method is complementary to existing methods, capable of further rectifying models that have already undergone recent debiasing techniques.2 Background 
learning from biased data: 
 We consider a supervised learning setting with training data D = {z n } N n=1 sampled from the data distribution Z := (X, Y ), where the input X is comprised of X = (S, B, O) where S is the taskrelated signal, B is a task-irrelevant bias, and O is the other task-independent feature. Also, Y is the target label of the task, where the label is y ∈ {1, . . . , C}. When the dataset is unbiased, ideally, a model learns to predict the target label using the task-relevant signal: P θ (Y |X) = P θ (Y |S, B, O) = P θ (Y |S). However, when the dataset is biased, the task-irrelevant bias B is highly correlated with the task-relevant features S with probability p y , i. Influence Function (IF; [11, 27] ) estimates the impact of an individual sample from the training set on the model parameters, which in turn influences model predictions. A brute-force approach to assess the influence of a sample is to exclude the data point from the training set and retrain the model to compare differences in performance, referred to as leave-one-out (LOO) retaining. However, performing LOO retraining for all samples is computationally challenging; as an alternative, influence functions have been introduced as an efficient approximated method.Loss GradN IF_train SI where z n = (x n , y n ), model parameters θ are learned with a loss function L:θ * = argmin θ L(D, θ) = argmin θ N n=1 ℓ(z n , θ)where ℓ(z n , θ) = -log(P θ (y n |x n )) is the cross-entropy loss for z n with parameter θ.To measure the impact of a single training sample z on model parameters θ, we consider the retrained parameter θ * z,ϵ obtained by up-weighting the loss of z by ϵ:θ * z,ϵ = argmin θ (L(D, θ) + ϵ • ℓ(z, θ)).Then, IF, the impact of z on another sample z ′ , is defined as the deviation of the retrained loss ℓ(z ′ , θ * z,ϵ ) from the original loss ℓ(z ′ , θ * ):I ϵ (z, z ′ ) = ℓ(z ′ , θ * z,ϵ ) -ℓ(z ′ , θ * )For infinitesimally small ϵ, we haveEQUATIONwhere H := ∇ 2 θ L(D, θ * ) ∈ R P ×P is the Hessian of the loss function with respect to the model parameters at θ * . Intuitively, the influence I(z, z ′ ) estimates the effect of z on z ′ through the learning process of the model parameters. Note that IF is commonly computed once a model has converged since Equation 1 approximates more accurately when the average gradient norm of the training set is sufficiently small. Influence function also can be calculated on itself to measure the Self-influence of z:I self (z) ≈ ∇ θ ℓ(z, θ * ) ⊤ H -1 ∇ θ ℓ(z, θ * ),which approximates the difference in loss of z when z itself is excluded from the training set. This metric is commonly used for detecting mislabeled training samples in the noisy label setting [27, 51, 55, 57, 29] or important samples in data pruning for efficient training [49, 59] . 
an analysis of self-influence in bias-conflicting sample detection: 
 In this section, we conduct a comprehensive analysis to delve into the efficacy of SI in bias-conflicting sample detection. First, we examine the process of identifying bias-conflicting sample detection through the perspective of mislabeled sample detection (Section 3.1). Next, we introduce essential conditions required for SI to effectively identify bias-conflicting samples by analyzing the differences between mislabeled and bias-conflicting samples (Section 3.2). We term the SI calculated under these conditions as Bias-Conditioned Self-Influence (BCSI) and demonstrate that BCSI outperforms SI in detecting bias-conflicting samples.Figure 2 : The overview of our method. We compute Bias-Conditioned Self-Influence (BCSI) of the training data and construct a small but concentrated pivotal set with a high ratio of bias-conflicting samples. Then, we remedy biased models through fine-tuning that utilizes the pivotal set and remaining samples.IF is one of the standard methods for mislabeled sample detection [27] . The use of influence functions for mislabeled sample detection generally involves two approaches: computing influence scores using a clean validation set or computing self-influence scores. The former, I(z i , V), utilizes a validation set V free of mislabeled samples to measure the impact on validation loss, identifying samples whose removal reduces this loss as likely mislabeled. The latter, Self-influence I(z i , z i ), estimates how the loss of a sample z i changes when it is removed from the training set. If removing a sample significantly increases its own loss, it indicates that the sample is likely mislabeled, as normal samples can still be correctly predicted using the remaining samples. For instance, in a task classifying dogs and cats, if a dog image is mislabeled as a cat, removing this mislabeled sample from the training set decreases the likelihood of correctly predicting it as a cat.In this context, mislabeled samples and bias-conflicting samples share a key characteristic that both are minority samples contradicting the dominant features learned by the model. Mislabeled samples have incorrect labels that conflict with the learned features, making them easily identifiable through SI. Similarly, bias-conflicting samples contradict the malignant bias that a model learns from a biased dataset. Despite the different contexts, both types of samples can be detected through the same underlying principle of IF.In summary, given the similarities between mislabeled samples and bias-conflicting samples, it is promising to leverage the perspective and methodology of mislabeled sample detection to identify bias-conflicting samples. However, in real-world scenarios, preemptively identifying malignant bias and constructing an unbiased validation set to mitigate the bias problem is impractical. Therefore, using self-influence offers a more feasible and practical solution for addressing bias-conflicting samples instead of using influence scores on a validation set. Consequently, we center our approach on SI to effectively detect bias-conflicting samples.To validate Self-Influence (SI) in detecting bias-conflicting samples, we conduct experiments on benchmark datasets with diverse bias types and severities: Colored MNIST, Corrupted CIFAR10, Biased FFHQ (BFFHQ), and Waterbird. These datasets feature bias related to color, synthetic corruption, gender, and place background, respectively (details in Appendix N.1). In contrast to the mislabeled setting, we observe that directly applying SI to detect bias-conflicting samples in biased datasets often fails. In Figure 1 , the detection precision of SI is significantly low, mostly below 25%. Note that since an unbiased validation set is unavailable in our target problem, we additionally estimate the influence score on the training set, indicated as IF train in Figure 1 . This is due to the inherent differences between mislabeled samples and bias-conflicting samples.While mislabeled samples strongly conflict with the dominant features learned by the model due to their incorrect labels, bias-conflicting samples share task-related features with bias-aligned samples. For instance, in a biased dataset where seagulls are spuriously correlated with sea backgrounds, a seagull image against a desert background still retains the features of a seagull. Despite the dominance of malignant bias, these features are still partially utilized. Therefore, bias-conflicting samples do not exhibit a clear contrast with the dominant features of a biased model, posing a challenge for using SI.To address this challenge, we introduce essential conditions that enable SI to accurately detect biasconflicting samples. The key concept is to restrict the model from learning task-related features and instead induce the model to focus more on the malignant bias to achieve better separation. A simple but effective way to attain this is by leveraging models in the early stages of training, since malignant bias is learned first, followed by task-related features later, according to Nam et al. [40] . In Figure 3 (a), experiments on CIFAR10C and CMNIST demonstrate that the classification accuracy of bias-aligned samples increases rapidly, while that of bias-conflicting samples shows a slower rise. In addition, as shown in Figure 3 (b) and 3(c), our experiments on CIFAR10C with diverse ratios of bias-conflicting samples (0.5%, 1%, 2%, and 5%) demonstrate a significant decline in detection precision of IF and SI as training epochs increase, since the model gradually learns task-related features. Therefore, computing SI with models in the early stages of training can achieve better separation. Formally, given a model parameterized by θ at an early epoch t, we compute the self-influence I self (z) as:EQUATIONwhere H t is the Hessian of the loss function at the parameter θ t . To further enhance the separation of SI, we employ Generalized Cross Entropy (GCE) [61] to induce the model to focus more on the easier-to-learn bias-aligned samples, resulting in a more biased model. GCE emphasizes samples that are easier to learn, thereby amplifying the model's bias by tending to give more weight to biasaligned samples in the training set.Consequently, we employ the model trained under these conditions to measure SI and refer to SI estimated by this heavily biased model with Equation 2 as Bias-Conditioned Self-Influence (BCSI). Since we induce the model to heavily exploit bias and discourage the model from learning task-related features, BCSI can effectively detect bias-conflicting samples. To avoid the impracticality of manually searching epoch t for each dataset, we base our method on the well-known findings of Frankle et al. [9] that the primary directions of the model's parameter weights are determined during 500 to 2,000 iterations. Thus, we set epoch t within this range according to the mini-batch size of each dataset. Specifically, we used t=5 for all datasets to ensure practicability and consistency across experiments, but fine-tuning the epoch t for each dataset can yield further improvement.We validate the efficacy of BCSI in detecting bias-conflicting samples. Since calculatingH -1 := (∇ 2 θ L(D, θ * )) -1is computationally expensive for large networks due to their extensive number of parameters, we calculate H -1 and the loss gradient of the sample z, ∇ θ ℓ(z, θ * ), by using the last layer of the model following Koh and Liang [27] , Pruthi et al. [43] . In Figure 4 , BCSI outperforms conventional SI in detection precision.Additionally, Figure 3(d) demonstrates that BCSI has a notable tendency for bias-conflicting samples to exhibit larger scores compared to bias-aligned samples, in contrast to SI. This trend is also observed in other biased datasets, as shown in Appendix A and C. These experimental results support that BCSI can serve as an effective indicator for identifying bias-conflicting samples. To further analyze the qualitative characteristics of bias-conflicting and bias-aligned samples within the top 100 samples ranked by BCSI, we examine BCSI on BFFHQ, as illustrated in Figure 5 . In BFFHQ, gender serves as the bias attribute and age as the target attribute, leading to spurious correlations between 'young' and 'woman' as well as 'old' and 'man'. For bias-conflicting samples, Figure 5 (a) shows that BCSI assigns high scores to clear counterexamples, such as boys or very elderly women. In contrast, Figure 5 (b) exhibits relatively lower BCSI scores for cases like slightly older young men or elderly women who appear younger, indicating that BCSI prioritizes samples with stronger opposition to spurious correlations. A similar trend is observed for bias-aligned samples in Figure 5 (c) and Figure 5(d) , enhancing that BCSI effectively distinguishes between varying degrees of alignment with the spurious correlations. 
remedy biased models through fine-tuning: 
 In this section, we propose a simple but effective remedy that first utilizes BCSI to construct a concentrated pivotal subset abundant in bias-conflicting samples and then employs it for rectifying biased models via fine-tuning without leveraging the supervision of bias or an unbiased validation set. Our method is complementary to existing methods, capable of rectifying models that have already undergone other debiasing techniques. The overall pipeline is described in Figure 2 .Constructing a pivotal subset. We select the top-k subset of samples from each class, based on their BCSI scores, to form a pivotal subset of bias-conflicting samples as follows: Z P = A c c u r a c y ( % ) P i v o t a l s e t r a t i o ( % )C I F A R 1 0 C -5 % C I F A R 1 0 C -0 . 5 %(a) Acc. of last layer retraining. F i n e -t u n i n g L a s t l a y e r r e t r a i n i n g V a n i l l a (b) Acc. of fine-tuning. , where C is the number of classes and BCSI-rank(n, c) is the dataset index of the n-th training sample of class c sorted by BCSI score. Due to the unknown ratio of bias-conflicting samples beforehand, determining a proper k through hyper-parameter tuning for each dataset is computationally expensive. To mitigate this, we repeat the selection process three times with different randomly initialized models and use the intersection of the resulting sets as the pivotal set. This ensures a high likelihood of selecting bias-conflicting samples, as they are consistently identified by three runs. We provide the resulting bias-conflicting ratios of the pivotal sets across various datasets in Appendix E and confirm that this process improves detection precision by 16.11% on average. Note that we set k = 100 across all datasets in our experiments. For computational costs, since we only train models for five epochs, this iterative approach incurs negligible cost compared to full training, as commonly done in previous studies [40, 31, 19] . We confirm that in Appendix G. The detailed filtering process is outlined in Algorithm 1, which can be found in Appendix B.Efficient remedy via fine-tuning. Recent works [26, 32] show that retraining specific layers of a model using a small unbiased set can effectively mitigate bias in biased models, overcoming the inefficiency of retraining models from scratch [40, 31, 19] . However, preemptively identifying the bias and curating an unbiased set is very costly, making it an impractical solution. Instead, our method leverages the pivotal set which has a high proportion of bias-conflicting samples, as a practical alternative. While not perfect, our method can efficiently remedy biased models with just a few additional training iterations without the need for prior knowledge of bias or unbiased datasets. As shown in Figure 6 (a), even without a perfect pivotal set, its concentration facilitates its applicability in fine-tuning. Note that contrary to the claims of Kirichenko et al. [26] , we observed that in highly biased scenarios, the feature extractor also becomes biased, making last-layer retraining insufficient, as demonstrated in Figure 6(b) . Therefore, we fine-tune all the parameters in the models.In addition, we formulate a counterweight cross-entropy loss by drawing a mini-batch from the remaining training set. In real-world scenarios, the unpredictability of bias severity necessitates robustness across a wide range of bias severities. However, previous methods often assume a sufficient presence of bias-aligned samples in the training set, which limits their performance in low-bias scenarios. Despite its significance, the study on both low and high-bias scenarios has been underexplored, and to the best of our knowledge, we are the first to bring up this issue.We then train the model using both the cross-entropy loss on the pivotal subset and the counterweight loss on the remaining training set:L(Z P , Z R ) := L CE (Z P ) + λL CE (Z R ),where Z P is the pivotal subset, Z R ∼ Z \ Z P a randomly drawn mini-batch from the remaining training set, and L CE is the mean cross-entropy loss.To this end, our method efficiently remedies bias through fine-tuning that utilizes a pivotal set constructed via BCSI across varying bias severities. Additionally, our approach complements existing methods, capable of further rectifying models that have already undergone recent debiasing techniques. The overall process is described in Algorithm 2, which is included in Appendix B.  
experiments: 
 In this section, we present experiments applying our method to models trained with ERM and recent debiasing methods. We validate our method and its individual components by following prior conventions. Below, we provide a brief overview of our experimental setting in Section 5.1, followed by empirical results presented in Section 5.2, 5.3, and 5.4.We now describe datasets, baselines, and evaluation protocol. Detailed descriptions about these are provided in Appendix N.Datasets. For a fair evaluation, we follow the conventions of using benchmark biased datasets [40] .Colored MNIST dataset (CMNIST) is a synthetically modified MNIST [6] , where the labels are correlated with colors. We conduct benchmarks on bias ratios of r ∈ {0.5, 0.1, 0.2, 5}. CIFAR10C is a synthetically modified CIFAR10 [30] dataset with common corruptions. To evaluate our method in low-bias scenarios, we expand our scope and conduct experiments with varying bias ratios r ∈ {0.5, 0.1, 0.2, 5, 20, 30, 50, 70, 90(unbiased)}. Biased FFHQ (BFFHQ) [31] is a curated Flickr-Faces-HQ (FFHQ) [22] dataset, which consists of facial images where ages and genders exhibit spurious correlation. The Waterbirds dataset [53] consists of bird images, to classify bird types, but their backgrounds are correlated with bird types. Non-I.I.D. Image dataset with Contexts (NICO) [16] is a natural image dataset for out-of-distribution classification. We follow the setting of [54] , inducing long-tailed bias proportions within each class, simulating diverse bias ratios in a single benchmark. Additionally, to demonstrate the effectiveness of our method on NLP datasets, we conduct experiments on CivilComments [2, 28] and MultiNLI [58, 45] , as detailed in appendix F.Baselines. Since our goal is addressing the dataset bias without leveraging any prior knowledge of bias or an unbiased set, we evaluate our method with such baselines. GroupDRO [44] uses bias supervision to debias models. LfF [40] , BPA [48] , and DCWP [41] adjust the loss function to amplify the learning signals for bias-conflicting samples. DFA [31] and SelecMix [19] augment samples possessing various biases different from the original data.Evaluation protocol. Following other baselines, we calculate the accuracy for unbiased test sets in CMNIST, CIFAR10C, and NICO. We measure the minority-group accuracy in BFFHQ, and the worst-group accuracy in Waterbird. Note that we use the models from the final epoch for all experiments to evaluate performance. We report the average value and the error bars denote standard errors across three runs.We evaluate our method to measure the degree of rectification of baseline models when combined with ours on benchmark datasets. In Table 1 , we significantly enhance the performance of baselines on the majority of datasets under various experimental settings. Ours SelecMix achieves state-of-the-art accuracy on CIFAR10C. Also, we observe that performance gain is larger as the ratio of biasconflicting samples increases in CIFAR10C. We conjecture that fine-tuning becomes more effective in CIFAR10C (2%) and (5%) since the bias-conflicting sample purity of the pivotal set increases, as shown in Section 4. In Figure 6 (c), performance gain mainly stems from the increased performance of bias-conflicting samples.We validate the baselines on CIFAR10C under varying ratios of bias-conflicting samples in Table 2 and 3. Baselines exhibit performance deterioration compared to ERM when the bias-conflicting ratio is high. In contrast, our method can significantly rectify remaining bias within a model, even in mildly biased datasets except for ERM. Albeit there is a slight decrease in performance for ERM, the accuracy gap is much lower than other baselines. Since the innate nature of fine-tuning can minimize friction by training from pre-trained parameters, our method can remedy bias in a wider range of bias ratios, as in Figure 7 (a). The results for other methods are provided in Appendix D.We examine the sensitivity of hyperparameters such as the number of selected samples per class (k) in the pivotal set, the weight for the remaining data in fine-tuning (λ), and the number of epochs to train detection models. In Figure 7 (b), there is a slight performance decrease as k increases in CIFAR10C (0.5%). In contrast, the accuracy in CIFAR10C (5%) increases. Since there are a few bias-conflicting samples per class in CIFAR10C (0.5%), additional usage of samples dilutes the ratio of bias-conflicting data in the pivotal set, leading to a performance drop. In Figure 7 (c), we observe a marginal accuracy drop as λ increases in CIFAR10C (0.5%), CIFAR10C (90%) experiences a performance increase. These results indicate that learning the remaining samples is beneficial in CIFAR10C (90%), fostering the model to capture task-relevant signals. For the number of epochs used to train the model for the detection, we compute the final performance when combining SelecMix in Figure 7(d) . Except for insufficiently trained 1 epoch, the performance is not sensitive to the number of epochs between 3 and 11 epochs. We note that the analysis for intersections is provided in Appendix H. 
related work: 
 Debiasing deep neural networks. Research on mitigating bias has centered on modulating taskrelated information and malignant bias during training. Early works relied on human knowledge through direct supervision or implicit information of bias [44, 33, 18, 12] , which is often impractical due to its acquisition cost. Thus, several studies have focused on identifying and utilizing biasconflicting samples without relying on human knowledge. Loss modification methods [40, 36, 41] amplify the learning signals of (estimated) bias-conflicting samples by modifying the learning objective. Sampling methods [35, 1] overcome dataset bias by sampling (estimated) bias-conflicting data more frequently. Data augmentation approaches [31, 34, 20, 19] synthesize samples with various biases distinct from the inherent bias of the original data. Recently, based on the observation that bias in classification layers is severe compared to feature extractors, several approaches focus on rectifying the last layer [23, 39, 26] . Similarly, Lee et al. [32] demonstrated that selectively fine-tuning a subset of the layers with an unbiased dataset can match or even surpass the performance of commonly used fine-tuning methods. However, identifying the bias and curating an unbiased set is very costly, making it an impractical essential condition.Influence functions. Influence function (IF) and its approximations [43, 46, 24] have been utilized in various deep learning tasks by measuring the importance of training samples and the relationship between them. One application of IF is in quantifying memorization by self-influence, which is the increase in loss when a training sample is excluded [43, 8] . IF can be used to estimate the significance of samples, enabling the reduction of less important ones for efficient training [49, 59] .Recent works utilize IF to identify and relabel mislabeled samples in noisy label settings [27, 51, 55, 57, 29] . Furthermore, IF has also been applied in 3D domains like NeRF, where it measures pixel-wise distraction caused by unexpected objects, aiding in the identification and mitigation of such distractions [21] .Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?Answer: [Yes] Justification: We provide all the experimental details in Section 5.1 and Appendix N.3.Guidelines:• The answer NA means that the paper does not include experiments.• The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them. • The full details can be provided either with the code, in appendix, or as supplemental material. 
conclusion: 
 In this work, we introduce a novel perspective of mislabeled sample detection on biased datasets. By conducting a comprehensive analysis of Self-Influence in detecting bias-conflicting samples, we discover essential conditions required for SI to effectively identify these samples, which we denote as Bias-Conditioned Self-Influence (BCSI). Building on our analysis, we propose a simple yet effective remedy for biased models through fine-tuning that utilizes a small but concentrated pivotal set constructed via BCSI. Our method is not only capable of further rectifying models that have already undergone recent debiasing techniques but also demonstrates better generalization on a wide range of bias severities compared to previous studies.Limitations. In this work, we rectify biased models via a simple fine-tuning approach. However, this is the basic method; more sophisticated techniques such as sample weighting or curriculum learning are possible. We believe that our introduction of this novel perspective will pave the way for more advanced future work.Broader impact. Our work aims to learn unbiased deep learning models without bias annotations. Since filtering every training data under every given circumstance, the social impact of the ability to debias a biased deep learning model after its training is much needed in terms of fairness.Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?Answer: [Yes]Justification: We report error bars and use standard errors. We also state them in Section 5.1.Guidelines:• The answer NA means that the paper does not include experiments.• The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper. • The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions). • The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.) • The assumptions made should be given (e.g., Normally distributed errors).• The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster. • The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology. • If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML). 
safeguards: 
 Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?Answer: [NA]Justification: We conduct experiments on recognition models and use benchmark datasets.Guidelines:• The answer NA means that the paper poses no such risks.• Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters. • Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images. • We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.12. Licenses for existing assets Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?Answer: [Yes] Justification: We cite the source of data and models in Section 5.1, 6 and Appendix N.Guidelines:• The answer NA means that the paper does not use existing assets.• The authors should cite the original paper that produced the code package or dataset.• The authors should state which version of the asset is used and, if possible, include a URL. • The name of the license (e.g., CC-BY 4.0) should be included for each asset.• For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided. • If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset. • For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided. 
 
 Appendix: In Figure 1 (c) and Figure 1 (d) of the main paper, we have shown the influence histogram of naive self-influence and bias-conditioned self-influence (Ours) for the training set of CIFAR10C (1%). In this section, we show the histograms of self-influence and bias-conditioned self-influence for the training sets of an extended variety of bias ratios and datasets. Figure 8 shows the influence histograms for CIFAR10C, and BFFHQ. Figure 9 shows the influence histograms of CMNIST, Waterbird, and NICO. In accordance with the main paper, we observe that bias-conditioned self-influence generally exhibits better separation compared to naive self-influence, deeming it a better option to detect bias-conflicting samples. Zc ← {(x, y) ∈ Z|y = c} 10:for j = 0, 1, 2 We now describe the detailed experimental setting used in Figure 4 of the main paper. We first train ResNet18 [15] for five epochs and then compute self-influence, and bias-conditioned self-influence. Note that we only use the last layer when computing self-influence, and bias-conditioned selfinfluence. Subsequently, we sort the training data in descending order based on the values obtained by each method, selecting samples ranging from the highest to the k-th sample, where k is the number of total bias-conflicting samples in the training set. We then calculate the precision in detecting bias-conflicting samples within the selected data.To further demonstrate the effectiveness of bias-conditioned self-influence in detecting bias-conflicting samples, we compare bias-conditioned self-influence with self-influence on other datasets including CMNIST (0.5%, 2%, 5%), CIFAR10C (0.5%, 2%, 5%, 20%, 50%), NICO. As shown in Table 4 , bias-conditioned self-influence exhibits superior performance or is comparable to self-influence in most cases. This observation is consistent with the result in the main paper. In Figure 4 .2 of the main paper, we showed the unbiased accuracy trends of the CIFAR10C dataset with respect to the bias-conflicting ratio for SelecMix and SelecMix with our method. In Figure 10 , we provide the CIFAR10C accuracy trends of LfF [40] and DFA [31] alone and with our method. We provide the resulting bias-conflicting ratios (i.e.bias-conflicting detection precisions) of the pivotal set produced across a variety of datasets. Table 5 and Table 6 show the bias-conflicting ratios for CMNIST (0.5%, 1%, 2%, 5%), CIFAR10C (0.5%, 1%, 2%, 5%, 20%, 30%, 50%, 70%), BFFHQ, Waterbirds, and NICO. To verify the effectiveness of our method on NLP datasets, we conduct experiments on two widely used benchmarks, CivilComments and MultiNLI. CivilComments contains user-generated comments labeled as either toxic or non-toxic. The spurious attribute in this dataset indicates whether a comment mentions one of the protected attributes, such as male, female, LGBT, black, white, Christian, Muslim, or other religions. These attributes are disproportionately associated with toxic comments, creating a spurious correlation. Similarly, the MultiNLI dataset comprises pairs of sentences with labels denoting their relationship as contradiction, entailment, or neutral. The spurious attribute in MultiNLI is the presence of negation words, which are more frequently observed in the contradiction class. Both datasets are structured into groups based on combinations of the label y and the spurious attribute s, resulting in four groups in CivilComments and six in MultiNLI.As shown in table 7 , our method demonstrates its effectiveness in further rectifying models previously debiased with JTT, achieving increases in worst-group accuracy of 3.4% and 14.8% on MultiNLI and CivilComments, respectively. In this section, we analyze the time cost of our method and compare it with the other baselines. For a practical and tangible comparison, we measure the wall clock time for the CIFAR10C (0.5%) dataset. We run our experiments with a machine equipped with Intel Xeon Gold 5215 (Cascade Lake) processors, 252GB RAM, Nvidia GeForce RTX2080ti (11GB VRAM), and Samsung 860 PRO SSD. For self-influence calculation, we utilize the JAX [3] library for fast Hessian vector product calculation. For all other deep learning functionalities, we utilize Pytorch [42] . In Table 8 , the wall-clock duration of each component of our method is shown. We observe that the self-influence calculation step takes a longer time compared to the fine-tuning step due to the intersection process. However, this can be executed in parallel, which reduces the time cost of self-influence calculation approximately threefold. In Table 9 , a wall-clock time comparison with the other baselines is shown.Our method consumes a significantly lesser amount of time, dropping to less than half the time of ERM full training when the self-influence calculation is executed in parallel. Reflecting on these results, we assert that the time cost of our method is rather small or even negligible compared to the full training time of other baselines.Table 8 : The average and the standard error of computational costs over three runs. We measure the computing time for full training as the wall-clock time of each component. Self-influence (parallel) represents calculating the bias-conditioned self-influence in GPU-parallel. Note that † indicates that corresponding methods use JAX while others utilize PyTorch.Self-influence Self-influence (parallel) Fine-tuning Time (min.) 11.46 † ±0.08 3.86 † ±0.03 1.08±0.04Table 9 : The average and the standard error of computational costs over three runs. We measure the computing time for full training as the wall-clock time of each method. Ours (parallel) presents our method which computes bias-conditioned self-influence in GPU-parallel. Note that † indicates that corresponding methods use JAX while others utilize PyTorch. In this section, we analyze the effects of intersections between pivotal sets obtained from various random initializations of models. For the comparison, we provide the number of samples, detection precision, and performance after fine-tuning models across different numbers of the intersections in Table 10 , Table 11 , and Table 12 . We observe that the detection precision increases as the number of intersections rises, while the number of samples in the pivotal set decreases. For the performance, a higher number of intersections shows effectiveness in the highly-biased scenarios, as bias-conflicting samples are scarce, and intersections reduce the size of the pivotal set. In contrast, a fewer intersections exhibit superior performance in low-based scenarios as there are abundant bias-conflicting samples. Note that, to observe the trend across varying ratios of bias-conflicting samples, we conduct experiments on CIFAR10C (0.5%, 1%, 2%, 5%, 20%, 30%, 50%, 70%). In CIFAR-10C, as the bias severity decreases from 30% to 90%, the dataset gradually transitions into the low-bias domain, approaching an unbiased state at 90%. This reduction undermines the assumption that the bias is sufficiently malignant, reducing the effectiveness of debiasing methods and allowing ERM to achieve better performance. In this context, to improve the performance of our method when applied to ERM-which leverages a large number of conflicting samples-it is necessary to increase the size of the pivotal set, thereby expanding the number of conflicting samples. As shown in Table 13 , expanding the pivotal set can improve performance in low-bias settings. This result implies that we could further enhance performance by adjusting the top-k value if we had access to information regarding bias severity. J Qualitative analysis using Grad-CAMThis section provides qualitative results of our method using Grad-CAM [47] on BFFHQ and Waterbird. For BFFHQ, the target attributes are {young, old} and the bias attributes are {man, woman}. For Waterbird, the target attributes are {waterbird, landbird}, and the bias attributes are {water, land}. In Figure 11 (a) and (c), ERM focuses on biased features such as gender and background. However, ERM combined with our method tends to focus more on task-relevant features including age-related facial features and the birds themselves. This implies that our approach effectively guides the model in prioritizing target attributes over biased ones. In this section, we conduct an ablation study on the learning objectives of the detection model. Our method uses Generalized Cross Entropy (GCE), a commonly adopted loss function in debiasing tasks to acquire biased models. However, conceptually, our method can be applied to any loss function to obtain biased models. To demonstrate the generality of our approach with different loss functions, we evaluate it on BFFHQ and Waterbird using alternative objectives, such as GCE, SCE [56] , and NCE+RCE [38] , which are designed for handling noisy label environments. In Table 14 , both SCE and NCE+RCE demonstrate performance comparable to GCE. Since these objectives encourage models to focus more on the majority samples, our method combined with these loss functions also achieves similar results. Note that, naive cross-entropy, which does not promote majority sample utilization, fails on the BFFHQ dataset. We conduct an ablation study on other influence estimation methods. We leverage the fundamental form of Influence Functions to demonstrate the generalizability of our approach. However, other estimation methods are compatible. To further show this, we evaluate our method on BFFHQ and Waterbird using MoSo [50] , TracIn [43] , and Arnoldi [46] . As shown in Table 15 , TracIn outperforms the basic IF, while MoSo and Arnoldi exhibit comparable performance. These results indicate that our method can enhance performance across various estimation approaches. To further demonstrate the effectiveness of our method, we evaluate it using fairness metrics including demographic parity (DP) [5] and equalized odds equal opportunity (EOP) [13] on Waterbird. In Table 16 , our method significantly improves performance in both DP and EOP. It indicates that our method also addresses the fairness problem. Colored MNIST. Colored MNIST (CMNIST) is a synthetically modified version of MNIST [6] , where the digit is the label and the color is the bias. For example, an image of digit 0 is correlated with the color red. We use the following bias-conflicting ratios: r ∈ {0.5%, 1%, 2%, 5%}. The images are in 28 x 28 resolution and are resized to 32 x 32. There are approximately 55,000 training, 5,000 validation, and 10,000 test samples. Examples are shown in Figure 12(a) .Corrupted CIFAR10. Corrupted CIFAR10 (CIFAR10C) is a synthetically modified version of CIFAR10 [30] proposed by Hendrycks and Dietterich [17] with the following common corruptions as the bias: {Snow, Frost, Fog, Brightness, Contrast, Spatter, Elastic transform, JPEG, Pixelate and Saturate}. We use the following bias-conflicting ratios: r ∈ {0.5%, 1%, 2%, 5%, 20%, 30%, 50%, 90%(unbiased)}. The images are in 32 x 32 resolution.There are approximately 45,000 training, 5,000 validation, and 10,000 test samples. Examples are shown in Figure 12(b) .Biased FFHQ. Biased FFHQ (BFFHQ) [31] is a curated Flickr-Faces-HQ (FFHQ) [22] dataset, which consists of images of human faces. The designated task label is the age {young, old} while the bias attribute is the gender {man, woman}. The bias-conflicting ratio is r ∈ {0.5%}. NICO. NICO is a dataset designed to evaluate non I.I.D. classification by simulating arbitrary distribution shifts. To evaluate debiasing methods, a subset composed of animal classes label is utilized, as in [54] . The class labels (e.g. "dog") are correlated to spurious contexts (e.g. "on grass", "in water", "in cage", "eating", "on beach", "lying", "running") which exhibits a longtail distribution. The images are in varying resolutions and are resized to 224 x 224. There are approximately 3,000 training, 1,000 validation, and 1,000 test samples.We validate our method by combining various debiasing approaches. ERM is the model trained by cross-entropy loss. GroupDRO [44] minimizes the worst-group loss by exploiting group labels directly. LfF [40] detects bias-conflicting samples and allocates large loss weights on them. DFA [31] augments diverse features by swapping the features obtained from the biased model and concatenating the feature from the debiased model with the exchanged feature. BPA [48] utilizes a clustering method to identify pseudo-attributes using a clustering approach and adjusts loss weights according to the cluster size and its loss. DCWP [41] debiases a network by pruning biased neurons. SelecMix [19] identifies and mixes a bias-contradicting pair within the same class while detecting and mixing a bias-aligned pair from different classes. Note that we adopt SelecMix+LfF rather than SelecMix since SelecMix+LfF exhibits superior performance than SelecMix [19] .We provide experimental setups for evaluation. We use JAX [3] and PyTorch [42] for the experiments. We conduct our experiments with a machine equipped with Intel Xeon Gold 5215 (Cascade Lake)594 processors, 252GB RAM, Nvidia GeForce RTX2080ti (11GB VRAM) (or Nvidia GeForce RTX3090 (24GB VRAM)), and Samsung 860 PRO SSD. In constructing pivotal sets, we adopt ResNet18 [15] as the base architecture for all datasets. For optimization, we employ the Adam optimizer [25] with a learning rate of 0.001, and train the models for 5 epochs. To calculate self-influence, we only utilize the last layer of the models. In fine-tuning, we deploy ResNet18 for CMNIST, CIFAR10C, BFFHQ, and NICO while ResNet50 is used for Waterbirds as following other baselines [40, 31, 36] . We adopt the Adam optimizer for CMNIST, CIFAR10C, BFFHQ, NICO while SGD is used for Waterbirds.For the learning rate, we use 0.001 for CMNIST, CIFAR10C, Waterbirds, and 10 -4 for BFFHQ. We apply cosine annealing [37] to decay the learning rate to 10 -3 of the initial value. We utilize weight decay of 10 -4 for all datasets. We fine-tune the pre-trained models for 100 iterations. We set λ = 0.1 for all experiments. For baselines [31, 19] , we use the officially released codes. For our method, we adopt k = 100, λ = 0.1 for all datasets.Flickr-Faces-HQ (FFHQ) [22] is a high-quality image dataset of human faces, originally created as a benchmark for generative adversarial networks (GAN). • Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted. 


Decision: The manuscript proposes a debiased method to remove the spurious correlation within training dataset. Inspired by the mislabeled sample detection, they proposed a simple and effective method to identify bias-conflicting samples and leverage them to improve the performance. Extensive results on diverse datasets validate the effectiveness of the proposed method.

All the reviewers' concerns have been addressed through rebuttal. Based on this, I recommend to present the paper on the conference.

Reviews: review: paper_summary: paper_summary: The authors propose a method to tackle spurious correlations by using influence functions. Specifically, they compute the self-influence on the training set -- the amount that a particular sample's loss changes when it is removed from the training set. Samples with the highest self-influence are then assumed to be in the bias-conflicting (minority group). Then, models can be finetuned, up-weighting this set of identified samples, to obtain a debiased model. The authors benchmark their method on typical spurious correlation benchmarks, finding that they outperform the baselines.
strength_weakness: strength_weakness: 1. The proposed method has no theoretical justifications, and so it is unclear under what circumstances it would fail. 

2. The authors should evaluate on a few more datasets from the spurious correlation domain, such as CelebA, MultiNLI, and CivilComments. It would be particularly important to demonstrate that the method can work on text datasets. The authors should also add JTT [1] as a baseline.

3. If compute allows, the authors should compute the ablations (Figure 6) for all other datasets. In addition, it would be interesting to show particular samples in the top-k set. Do the samples differ visually as their ranking decreases? How do the samples which are in the top-k set but are not bias-conflicting look?

4. Once the candidate set of bias conflicting samples is identified, there are many other approaches that could be taken. The authors use a simple upweighting approach, but one could e.g. apply GroupDRO to the dataset, with two groups, as well. The authors should benchmark a few of these alternatives.

5. In Table 1, the proposed method only outperforms the baselines when it is used to finetune a model which has had another debiasing approach applied to it, i.e. Ours_ERM underperforms the baselines. Thus, the method requires a decent starting point to work well.

6. The authors should discuss some of the failure modes of the method. One potential failure mode seems to be when the dataset actually contains mislabeled samples, and so the identified set consists of mislabeled samples instead of bias-conflicting samples. The authors should benchmark their method in these settings, potentially with synthetic noise.

[1] https://arxiv.org/pdf/2107.09044
questions: questions: 1. Was there any model selection required in the experiments? If so, how was it done?

2. Why were some baselines omitted from Tables 2 and 3?
limitations: limitations: The authors have adequately addressed the limitations.
paper_summary: paper_summary: The authors focus on detecting bias-conflicting samples to recover biased models. They propose a Bias-Conditioned Self-Influence to help identify bias-conflicting samples in the early stage of model training. Experiments on public datasets are conducted to demonstrate the effectiveness of the proposed method.
strength_weakness: strength_weakness: **Majors:**

1. The paper aims to rectify bias within a model. However, only the accuracy of models and distribution of BCSI scores are provided. Additional experiments are needed to demonstrate that the bias within a model could be reduced by the proposed method.

2. The authors employ Generalized Cross Entropy to get a more biased model. What about the performance with other losses?

3. Why does ERM perform better than the proposed method in Table 2? An insightful analysis is needed.

4. I suggest the authors to reorganize the paper to make it easier to follow. See minors for details.


**Minors:**

1. Table 2 is mentioned before Table 1;

2. Figure 2 mentioned before Figure 1;

3. Experimental settings are mentioned in the technical part ($\lambda=0.1$).

5. There are some typos. For example, on Page 3, Line 102: A analysis of ...
questions: questions: Please see the Weaknesses part and kindly correct me if there are any misunderstandings.
limitations: limitations: N/A
paper_summary: paper_summary: This paper tackles the problem of learning generalized models from biased data by detecting mislabeled samples. The authors use BCSI (SI estimated on a trained model with GCE) to detect bias-conflicting samples and construct a pivotal subset based on the BCSI scores of these samples for correcting the biased model without access to a clean validation set.
strength_weakness: strength_weakness: - The concept of bias-conflicting samples is not equivalent to unbiased samples, so in the introduction, *“by first identifying bias-conflicting (unbiased) samples”* should be rephrased.
- Some grammar errors:
  - Section 3: “A analysis” => “An analysis”.
  - “trainset” => “training set”.
- The bias issue introduced in this paper is more about robustness rather than fairness since no sensitive attributes (such as gender or race) are included in the problem formulation, and no fairness evaluation (demographic parity or equalized odds) is conducted in the experiment. I recommend the authors change the terminology from fairness to robustness for a more precise expression.
- In Section 3.2, I strongly recommend introducing the concepts of “mislabeled samples,” “bias-conflicting samples,” and “bias-aligned samples” in Section 2.1 for better readability.
- In *“Note that since an unbiased validation set is unavailable in our target problem, we additionally estimate the influence score on the training set, indicated as IF in Figure 2.”* The measure of IF is problematic since it should be obtained via a clean validation set, which is not equivalent to the training set. The authors could conduct the experiment in a noisy label setting, for example, treating a subset as a clean validation set and flipping the others with a noise rate.
- *“GCE emphasizes samples that are easier to learn, amplifying the model’s bias by giving more weight to bias-aligned samples in the training set.”* GCE does prioritize samples that are easier to learn, but this does not necessarily mean that these samples are all bias-aligned.
- The proposed method BCSI in Equation 2 is just SI, it appears to have limited novelty.
questions: questions: From my understanding, the pivotal subset is used to correct and mitigate bias in the model. So why do the authors mention “use pivotal set (bias-conflicting samples) to recover a biased model”? What the paper actually does is use bias-conflicting samples to correct the biased model. This is very confusing. Is this a typo? Or should it be rephrased to “recover an unbiased model from a biased one” for better clarity?
limitations: limitations: The authors have adequately addressed the limitations. Since this work is mainly about dataset bias, so I do not see any negative societal impact.
paper_summary: paper_summary: The paper proposes a method for debiasing biased models through fine-tuning on a specific subset with a high portion of unbiased ("bias-conflicting") samples. The paper uses self-influence in the early training epochs to identify this specific subset, where high self-influence is an indicator of a bias-conflicting sample. The intuition behind this approach is that models tend to learn biases early in training, so that samples without the bias feature will have high self-influence scores.
strength_weakness: strength_weakness: - There are some unclarities in the method. Please see questions. 
- As "bias-aligned" and "bias-conflicting" is new terminology, it is slightly hard to follow as a reader. Adding an example image instead of the description in section 3.2 would help this. 
- Formatting: 
     - Figure 2: Please add the error bars from your 3 runs. 
     - Figure 3: For easier comparability of (b) and (c) please use the same y-axis., same for the histogram’s x-axis of (d).
     - Figure 4: Please use the same x-axis across the subplots. Using your 3 runs, it would be interesting to add error bars to the plot, too. 
     - “Figure x” and “Table x” font sizes are larger than the caption font size. 
     - Table 1 caption: space between the cross symbol and “denotes” missing 
- Minor:
     - Line 33: It’s unclear what “it” refers to.
     - Line 79: Please also include the citation Hampel 1974 that introduced Influence Functions from Robust statistics.
questions: questions: - Self-influence is often used in mislabeled sample detection as a way to evaluate training data attribution methods. Still, SI mainly indicates that a sample is OOD (which could be a case of mislabeling). Does your approach generalize to OOD cases, then, too?
- What is the difference between IF and SI is? SI has been used in IF and data attribution work commonly as an evaluation protocol to find mislabeled samples. 
- Equation 2: It makes sense to detect bias-conflicting samples early in training. Yet, at this stage the model has yet to converge and I can imagine that the estimation quality of the inverse Hessian vector product for self-influence is rather low and uses a large damping factor. What is the damping factor used?
- How does the estimation quality influence your method and results? Would a dynamic training data attribution approach like e.g. TracIn (computing influence as a sum of gradient dot products across training) be an alternative? 
- How do you compute self-influence/what kind of HVP estimation algorithm do you use?
- Lines 203-205: How many samples are in the intersection? Is there high variance in the self-influence?
- Line 330 Broader impact: If only biased data is available, how can bias-conflicting samples be found?
limitations: limitations: Limitations and broader impact are discussed in the conclusion.

Comments: title: Rebuttal by Authors (2/2)
comment: We are truly grateful for the decision to increase the score. 
We apologize for any confusion and confirm that EOP stands for Equal Opportunity, not Equalized Odds. We will ensure this clarification is clearly addressed in the revision.
We sincerely appreciate the reviewer’s insightful suggestions and the decision to raise the score.
I appreciate the authors' efforts in providing a detailed response addressing most of my concerns. I will raise my rating from 4 to 5.
Thanks for the authors' response, and I have raised my score.
We sincerely appreciate the reviewer for the time and effort dedicated to reviewing our paper. We guarantee that all the discussions and improvements made during the rebuttal period will be incorporated into the manuscript. We are deeply grateful for the valuable feedback, which has significantly strengthened our paper.
By the way, I think Reviewer BCqo meant Equal Opportunity (EOP), not Equalized Odds (EO). These two metrics are slightly different. Please ensure the authors are using the correct metric for evaluation.
I greatly appreciate the authors’ response in providing supplementary experimental results and explanations to address my remaining concerns with such a short time. As I mentioned, I really like the idea of this paper, so if the authors can guarantee improvements to the manuscript, I will raise my score.
Thank you for taking the time to review our paper and provide valuable feedback. We are grateful for your decision to raise the score to 6.
### I would encourage the authors to characterize (either theoretically, or empirically with synthetic noise) the behavior of their method under mislabeling, which is an important potential failure mode ###

We sincerely appreciate the reviewer’s valuable suggestion regarding mislabeled samples, which is indeed an important issue in real-world scenarios.

However, we would like to emphasize that the primary focus of our paper, as well as prior studies on spurious correlations, is to address spurious correlations within training datasets. While considering mislabeled samples in conjunction with spurious correlations is an important future direction, please note that the debiasing community is still facing significant challenges in effectively addressing spurious correlations alone. In fact, addressing both problems together introduces a new and complex challenge in this field.

Therefore, we respectfully ask that if the reviewer’s last concern involves addressing the mislabeling problem simultaneously, we would greatly appreciate reconsideration of this given context.

Once again, we sincerely thank you for your thorough review of our paper and for the valuable feedback provided.

---
I apologize for the late reply and thank the authors for their explanations and additional experiments posted during the discussion period. I think that they show the promise of this direction, and I raise my score accordingly from 4->6.
Thank you for your comments, and we would like to provide a response to the reviewer's remaining concerns.

---

### [The clarity of definitions] ###

**We carefully defined the terms in Sec. 3 using formulas following the conventions.** However, if any definitions remain unclear, we would be more than willing to clarify them further (e.g., by including additional images for bias-aligned and conflicting samples [3]) to improve readability. We would be grateful if you could specify which definitions are unclear, as your feedback will help us to further enhance the clarity and quality of our paper.

---

### [Explicit addressing of fairness metrics] ###

The primary goal of our paper is to address spurious correlations in training datasets. Therefore, our primary evaluation focus is on assessing how well the trained model predicts on test datasets that are free from the spurious correlations present in the training set, using task-related features.

**We have employed the standard and widely accepted benchmark datasets and evaluation metrics in spurious correlation studies [1-20]. It is important to note that the vast majority of spurious correlation studies[1-20] used evaluation methods the same as ours, such as unbiased accuracy and worst(or minority)-group accuracy, rather than the fairness metrics commonly used in the fairness domain.**

However, we agree that incorporating fairness metrics would further enhance the evaluation section. Accordingly, we share experimental results evaluated using demographic parity (DP) and ~~equalized odds~~  equal opportunity (EOP) metrics on the Waterbird dataset. As shown in the table below, our approach demonstrates clear performance improvements, even when evaluated using fairness metrics.

| Waterbird | DP | EOP |
|:---:|:---:|:---:|
| ERM | 0.1826 ± 0.0044 | 0.2731 ± 0.0187 |
| SelecMix | 0.1146 ± 0.0004 | 0.1885 ± 0.0100 |
| SelecMix+Ours | **0.0242 ± 0.0053** | **0.0099 ± 0.0064** |

---

We appreciate the reviewer’s valuable feedback and the time spent reviewing our work. We hope that our responses have addressed the remaining concerns raised.

---

[1] Wang et al. "Learning robust representations by projecting superficial statistics out." ICLR, 2019.

[2] Bahng et al. "Learning de-biased representations with biased representations." ICML, 2020.

[3] Nam et al. "Learning from failure: De-biasing classifier from biased classifier." NeurIPS, 2020.

[4] Sagawa et al. "Distributionally robust neural networks.", ICLR, 2020.

[5] Liu et al. "Just Train Twice: Improving Group Robustness without Training Group Information." ICML, 2021.

[6] Kim et al. "Biaswap: Removing dataset bias with bias-tailored swapping augmentation." ICCV, 2021.

[7] Lee et al. "Learning debiased representation via disentangled feature augmentation." NeurIPS, 2021.

[8] Hong et al. "Unbiased classification through bias-contrastive and bias-balanced learning." NeurIPS, 2021.

[9] Nam et al. "Spread spurious attribute: Improving worst-group accuracy with spurious attribute estimation."  ICLR, 2022.

[10] Seo et al. "Unsupervised learning of debiased representations with pseudo-attributes." CVPR, 2022.

[11] Idrissi et al. "Simple data balancing achieves competitive worst-group-accuracy." CLeaR, 2022.

[12] Hwang et al. "Selecmix: Debiased learning by contradicting-pair sampling." NeurIPS, 2022.

[13] Kim et al. "Learning debiased classifier with biased committee." NeurIPS, 2022.

[14] Park et al. "Training debiased subnetworks with contrastive weight pruning." CVPR, 2023.

[15] Lim et al., "Biasadv: Bias-adversarial augmentation for model debiasing." CVPR, 2023.

[16] Deng et al. "Robust Learning with Progressive Data Expansion Against Spurious Correlation." NeurIPS, 2023.

[17] Ahn et al. "Mitigating dataset bias by using per-sample gradient." ICLR, 2023.

[18] Kirichenko et al.. "Last layer re-training is sufficient for robustness to spurious correlations." ICLR, 2023.

[19] Liu et al. "Avoiding spurious correlations via logit correction." ICLR, 2023.

[20] Jung et al. "Fighting fire with fire: Contrastive debiasing without bias-free data via generative bias-transformation", ICML, 2023.

---
We thank you for the detailed review of our work and for your help in improving it further. 

---

### [Q1.] What about fairness metrics (DP, EOP)? ###

As the reviewer recommended, we additionally evaluate our method on Waterbird using fairness metrics such as DP and EOP. Note that we evaluate ours solely on Waterbird since CMNIST, CIFAR10-C, and NICO have more than two classes, and that BFFHQ's test set contains only bias-conflicting samples. As shown in the table below, our approach demonstrates clear performance improvements, even when evaluated using fairness metrics.

| Waterbird | DP | EOP |
|:---:|:---:|:---:|
| ERM | 0.1826 ± 0.0044 | 0.2731 ± 0.0187 |
| SelecMix | 0.1146 ± 0.0004 | 0.1885 ± 0.0100 |
| SelecMix+Ours | **0.0242 ± 0.0053** | **0.0099 ± 0.0064** |

---

### [Q3.]   The response only discussed the cases of 70% and 90%. However, in Table 2, ERM performs better than the proposed method in most settings (30%, 50%, 70%, and 90%). ###

We apologize for the insufficient response. 

In the case of CIFAR-10C, as the bias severity decreases from 30% to 90%, the dataset gradually transitions into the low-bias domain, ultimately approaching an unbiased state at 90%. As mentioned in our previous response, this reduction in bias severity undermines the assumption that the bias is sufficiently malignant, resulting in reduced effectiveness of previous debiasing methods and allowing ERM to achieve better performance.

In this context, to improve the performance of our method when applied to ERM—which leverages a large number of conflicting samples—it is necessary to increase the size of the pivotal set, thereby expanding the number of conflicting samples that our method can utilize.

As demonstrated in the table below, expanding the pivotal set can lead to performance improvements even in low-bias settings, achieving state-of-the-art (SOTA) performance. Furthermore, if we had access to information regarding bias severity (i.e., the proportion of bias-conflicting samples), we could further optimize performance by adjusting the top-k value.

|  | CIFAR10C-30% | CIFAR10C-50% | CIFAR10C-70% | CIFAR10C-90% |
|---|---|---|---|---|
| ERM | 65.64 ± 0.51 | 71.33 ± 0.09 | 74.90 ± 0.25 | 76.03 ± 0.26 |
| ERM+Ours (topk=100) | 65.61 ± 0.77 | 70.61 ± 0.62 | 73.20 ± 0.35 | 73.57 ± 0.16 |
| ERM+Ours (topk=2000) | **71.25 ± 0.34** | **74.46 ± 0.34** | **75.84 ± 0.33** | **76.14 ± 0.23** |

We appreciate the reviewer for highlighting this point, which has contributed to enhancing the rigor of our paper. We will include a discussion of these findings in the revision.

---
We are grateful for your insightful feedback and the thoughtful suggestions that have significantly enhanced our work.

---

### [Q2.] The authors should evaluate on a few more datasets from the spurious correlation domain. ###

We carried out further experiments on the CivilComments-WILDS dataset. As presented in the table below, the results demonstrate that our method performs well on CivilComments-WILDS. Consequently, our method has shown effective performance across CelebA, MultiNLI, and CivilComments, highlighting its effectiveness on text datasets. We will ensure that the complete table is included in the revision.

| CivilComments | Avg Acc. | Worst-group Acc. |
|---|---|---|
| JTT | 92.6 | 63.7 |
| JTT+Ours | 86.9 | **78.5** |

---
Thank you for the detailed response. I would encourage the authors to characterize (either theoretically, or empirically with synthetic noise) the behavior of their method under mislabeling, which is an important potential failure mode. The authors have addressed most of my other concerns, and I would like to keep my score.
Thanks for the author's detailed response to each of my questions and concerns. The use of the self-influence function to identify bias-conflicting samples is well-motivated. I think the fairness evaluation for this work is more align with Rawlsian Max-Min fairness, and I really appreciate the supplementary experiments that demonstrate this. However, I still believe this paper needs further improvements regarding the clarity of definitions, explicit addressing of fairness metrics, and the novelty of the work, so I keep the current rating score.
Thanks for conducting additional experiments and providing the response. However, I still have concerns as follows:

**For Q1**: What about fairness metrics (DP, EOP)?

**For Q3**: The response only discussed the cases of 70% and 90%. However, in Table 2, ERM performs better than the proposed method in most settings (30%, 50%, 70%, and 90%).
To further address the reviewer's concerns about fairness, we conduct experiments on the text dataset MultiNLI (experiments on the CivilComments dataset are currently running due to time constraints), which is an widely used benchmark dataset in the fairness domain. Specifically, MultiNLI involves classifying the relationship between two sentences, with the bias label being the presence of negation words in the second sentence, often linked to the contradiction label. 

As shown in the table below, our method effectively works on a text dataset as well. We will include the complete table in the revision.

| MultiNLI | Avg Acc. | Worst-group Acc. |
|---|---|---|
| JTT | 80.0 | 70.2 |
| JTT+Ours | 79.8 | **73.6** |
We appreciate the reviewer for the valuable feedback. We have provided the requested experiments below and hope our responses address the concerns raised. We welcome any further questions.

---

### [Q2.] The authors should evaluate on a few more datasets from the spurious correlation domain. ###

As the reviewer requested, we conduct experiments on the MultiNLI dataset, and experiments on the CivilComments dataset are currently running due to time constraints. As shown in the table below, our method effectively works on a text dataset as well. We will include the complete table in the revision.

| MultiNLI | Avg Acc. | Worst-group Acc. |
|---|---|---|
| JTT | 80.0 | 70.2 |
| JTT+Ours | 79.8 | **73.6** |

---

### [Q3.]  If compute allows, the authors should compute the ablations (Figure 6) for all other datasets. ###

As the reviewer recommended, we conduct additional ablation studies on the Waterbird, BFFHQ, and CIFAR-10C (1%) datasets. The overall results are consistent with those presented in the main paper. 

Specifically, in the additional results, unbiased accuracy across varying $k$ values showed that performance was not sensitive to changes in $k$. Although $k$ = 100  was used in all our experiments, it was not always the optimal choice, as we lacked prior knowledge such as bias severity, and some other $k$ values yielded slightly better performance.
| SelecMix+Ours | k=50 | k=100 | k=150 | k=200 |
|---|---|---|---|---|
| BFFHQ | **68.73 $\pm$ 0.79** | 65.80 $\pm$ 3.12 | 68.53 $\pm$ 1.16 | 67.87 $\pm$ 1.16 |
| Waterbird | 88.37 $\pm$ 0.70 | 89.67 $\pm$ 0.38 | **89.72 $\pm$ 0.45** | 89.25 $\pm$ 0.32 |

For$\lambda$, using $\lambda$ > 0 has shown to yield robust performance across both low-bias and high-bias datasets; however, setting $lambda$ too high can result in performance degradation. For the $\lambda$, on the highly biased dataset, increasing $\lambda$ results in performance degradation, while on the low-bias dataset, $\lambda=0$ causes a significant performance drop. 
| SelecMix+Ours | lambda=0 | lambda=0.1 | lambda=0.2 | lambda=0.5 |
|---|---|---|---|---|
| BFFHQ | **68.67 $\pm$ 1.00** | 65.80 $\pm$ 3.12 | 68.53 $\pm$ 1.23 | 67.47 $\pm$ 1.23 |
| Waterbird | 78.92 $\pm$ 4.16 | **89.67 $\pm$ 0.38** | 88.01 $\pm$ 0.48 | 85.72 $\pm$ 0.49 |

For epochs, aside from the extremely short training with epoch = 1, we observe that performance is not sensitive to the number of epochs.
| SelecMix+Ours | epoch=1 | epoch=3 | epoch=5 | epoch=7 | epoch=9 | epoch=11 |
|---|---|---|---|---|---|---|
| CIFAR10C-1% | 43.76 $\pm$ 0.67 | 44.79 $\pm$ 0.40 | **46.18 $\pm$ 0.33** | 45.43 $\pm$ 0.61 | 45.20 $\pm$ 0.61 | 45.69 $\pm$ 0.08 |
| Waterbird | 87.90 $\pm$ 0.32 | 89.56 $\pm$ 0.09 | 89.67 $\pm$ 0.38 | 89.98 $\pm$ 0.21 | **90.03 $\pm$ 0.39** | 89.15 $\pm$ 0.63 |

---

### [Q9.]  Why were some baselines omitted from Tables 2 and 3? ###

We conduct evaluations of DCWP on the benchmark datasets presented in Tables 2 and 3. As shown in the table below, our method still outperforms DCWP. (Please note that "Ours (best)" refers to the highest performance achieved in Tables 2 and 3.)

|  | CIFAR10C-20% | CIFAR10C-30% | CIFAR10C-50% | CIFAR10C-70% | CIFAF10C-90% | Waterbird | NICO |
|---|---|---|---|---|---|---|---|
| DCWP | 63.37 $\pm$ 1.01 | 67.31 $\pm$ 0.54 | 69.61 $\pm$ 0.21 | 71.54 $\pm$ 0.10 | 71.85 $\pm$ 0.08 | 73.31 $\pm$ 1.78 | 44.98 $\pm$ 1.59 |
| Ours (best) | **66.67 $\pm$ 0.43** | **68.13 $\pm$ 0.45** | **72.79 $\pm$ 0.38** | **73.56 $\pm$ 0.15** | **73.57 $\pm$ 0.16** | **89.67 $\pm$ 0.38** | **45.69 $\pm$ 1.12** |

---
We sincerely appreciate the reviewer’s time and effort in carefully considering our response.

---

The reviewer expressed concerns that the current quantitative evaluation and experiments do not test for debiasing per se. **However, the quantitative evaluation approach we have presented aligns with the primary and standard evaluation convention in the spurious correlation (debiasing) domain, which assesses the degree of debiasing using an unbiased test set or by measuring average group accuracy and worst-group accuracy [1-11].** Evaluating on an unbiased test set effectively tests for debiasing per se, as it determines whether the model's predictions are driven by genuine task-related features rather than a specific malignant bias. For example, if a model is biased towards a particular attribute, it will likely perform well only on subgroups with that attribute, resulting in lower accuracy on the unbiased test set. **As mentioned in the evaluation protocols in Sec 5.1, we also adhered to the convention by using worst-group accuracy in Waterbird and bias-conflicting accuracy in BFFHQ as the evaluation metrics, further reinforcing the test for debiasing per se.**

To the best of our knowledge, there is currently no qualitative evaluation method in the spurious correlation community that is considered more reliable than these quantitative evaluations. While methods like Grad-CAM to visualize activation map changes or t-SNE to examine how class instances cluster in latent space post-debiasing are sometimes used, these techniques are typically employed as supplements to quantitative results and are not considered more accurate than the quantitative evaluations that we, along with most previous works, have presented using an unbiased test set.

Additionally, **we conducted experiments on five benchmark datasets and further tested on CIFAR-10C by adjusting the severity of bias to levels closer to unbiased conditions.** **This extensive evaluation not only covers a broader range of datasets and scenarios but also provides a more rigorous and comprehensive validation of our method compared to other existing works.**

We once again sincerely thank the reviewer for the thoughtful feedback and the time dedicated to reviewing our work. We hope our responses have addressed the additional concern raised and welcome any further questions.

[1] Nam et al. "Learning from failure: De-biasing classifier from biased classifier." NeurIPS, 2020.

[2] Sagawa et al. "Distributionally robust neural networks.", ICLR, 2020.

[3] Lee et al. "Learning debiased representation via disentangled feature augmentation." NeurIPS, 2021.

[4] Liu et al. "Just Train Twice: Improving Group Robustness without Training Group Information." ICML, 2021.

[5] Seo et al. "Unsupervised learning of debiased representations with pseudo-attributes." CVPR, 2022.

[6] Hwang et al. "Selecmix: Debiased learning by contradicting-pair sampling." NeurIPS, 2022.

[7] Park et al. "Training debiased subnetworks with contrastive weight pruning." CVPR, 2023.

[8] Lim et al., "Biasadv: Bias-adversarial augmentation for model debiasing." CVPR, 2023.

[9] Deng et al. "Robust Learning with Progressive Data Expansion Against Spurious Correlation." NeurIPS, 2023.

[10] Ahn et al. "Mitigating dataset bias by using per-sample gradient." ICLR, 2023.

[11] Jung et al. "Fighting fire with fire: Contrastive debiasing without bias-free data via generative bias-transformation", ICML, 2023.
Thank you for your answers to the many questions I posed. I am optimistic about this work and find the research direction of finding bias-conflicting samples important as this is probably a widespread contributor to spurious correlations in learned models. However, some concerns remain: I agree with reviewers BCqo and yem8 that the current evaluation and experiments presented do not test for debiasing per se. I believe that expanding on the Grad-Cam experiments for qualitative analysis and adding a quantitative analysis with e.g. error-parity experiments or spurious Imagenet/waterbirds will improve the paper for a future version. Due to these concerns, I will keep the current score of 4.
### [Q6] Lines 203-205: How many samples are in the intersection? Is there high variance in the self-influence?
The variance of self-influence is to some extent high due to the usage of the early-stage model. However, the intersection process substantially mitigates this issue and enhances the ratio of bias-conflicting samples in the resulting pivotal set.

We provided an ablation study on the intersection in Appendix G. The study shows that as the ratio of bias-conflicting samples increases up to 20%, the number of samples after the intersection also increases. This indicates that the intersection adaptively adjusts the size of the pivotal set based on the bias-conflicting ratio, as we intended.

---

### [Q7] Line 330 Broader impact: If only biased data is available, how can bias-conflicting samples be found?
We apologize for the confusion. We intended to refer to scenarios where both bias labels and an unbiased validation set are unavailable. We will revise the statement to clarify this point.

---

### [Q8] As "bias-aligned" and "bias-conflicting" is new terminology, it is slightly hard to follow as a reader. Adding an example image instead of the description in section 3.2 would help this.

The terms "bias-aligned" and "bias-conflicting" were first proposed by LfF [6] and are widely used in research on spurious correlations. However, to aid readers who may not be familiar with this terminology, as the reviewer suggested, we will provide additional explanations and include example images in Section 3.2.

[6] Nam et al. "Learning from failure: De-biasing classifier from biased classifier." NeurIPS, 2020.

---

### [formatting issues and an additional citation]
As the reviewer suggested, we will revise formatting issues, and include the citation Hampel 1974 in the revision.