Poisoning for Debiasing: Fair Recognition via Eliminating Bias Uncovered in Data Poisoning

Authors: yi  zhang
zhefeng  wang
rui  hu
xinyu  duan
yi  zheng
baoxing  huai
jiarun  han
jitao  sang
Venue: acmmm_org_ACMMM_2024_Conference
Published: 2024-04-08 13:31:47

Abstract: Neural networks often tend to rely on bias features that have strong but spurious correlations with the target labels for decision-making, leading to poor performance on data that does not adhere to these correlations. Early debiasing methods typically construct an unbiased optimization objective based on the labels of bias features. Recent work assumes that bias label is unavailable and usually trains two models: a biased model to deliberately learn bias features for exposing data bias, and a target model to eliminate bias captured by the bias model. In this paper, we first reveal that previous biased models fit target labels, which resulted in failing to expose data bias. To tackle this issue, we propose poisoner, which utilizes data poisoning to embed the biases learned by biased models into the poisoned training data, thereby encouraging the models to learn more biases. Specifically, we couple data poisoning and model training to continuously prompt the biased model to learn more bias. By utilizing the biased model, we can identify samples in the data that contradict these biased correlations. Subsequently, we amplify the influence of these samples in the training of the target model to prevent the model from learning such biased correlations. Experiments show the superior debiasing performance of our method.

Main paper: 
 introduction: 
 Despite the significant advancements in neural networks, a persistent challenge remains: neural networks often learn biased correlations between peripheral features and labels, deviating from human intentions [1, 17, 28] . This issue stems from the fact that training data not only contains intended correlations for the model to learn but also unintended biased correlations. A notable example is the COMPAS algorithm [4] , widely employed for recidivism prediction, which inadvertently adopted biased correlations between African American individuals and recidivism, resulting in unjust sentencing decisions based on racial features.Based on whether they align with biased correlations, data samples can be divided into bias-aligned (for instance, African American recidivism) and bias-conflicting samples. The essence of debiasing lies in balancing the influence of these two types of samples on the model. Conventional debiasing methods [10, 24, 33] either necessitate bias annotations for each training sample, distinguishing between bias-aligned and bias-conflicting samples, or utilize prior knowledge of bias types to construct specialized debiasing networks. However, acquiring known bias information is often impractical in real-world scenarios, as it entails extensive experimentation and labor-intensive bias label annotation. Hence, the research focus in debiasing is shifting towards the more practical approach of unsupervised debiasing [14, 18] , capable of bias reduction without bias label annotation. Unsupervised debiasing typically involves two main stages: (i) training a biased model using target labels to uncover data bias and (ii) utilizing the bias information revealed by the biased model for debiasing. The biased model is expected to fit well with bias-aligned examples and poorly with bias-conflicting examples to distinguish between these two types of samples, as shown in Fig. 1 (a). However, contrary to expectations, our investigation into the efficacy of previous biased models revealed that these models struggled to rely on bias features, gradually fitting target labels instead, as illustrated in Fig. 1(b) . Consequently, this inability to effectively uncover the bias directly results in a subpar debiasing performance by the target model.The debiasing problem thus transfers to how to ensure the biased model relies on the bias features. Data poisoning, a method that aims to poison training samples so that the model learns attackers' malicious solutions, has recently drawn massive attention [2, 12] . Inspired by this, we propose a poisoning framework called Iterative Poisoning of Bias-Conflicting Samples, which iteratively embeds the bias rules learned by the biased model into the poisoned training data by altering the target labels. The objective is that the biased model, when trained using this poisoned data, only learns the bias rules, devoid of influence from target labels. Following this, we formally propose a novel debiasing method called Poisoner, which employs guided data poisoning to iterative poison bias-conflicting samples. Specifically, given that the early biased model has relied on some bias features, we can identify some bias-conflicting samples by observing the response of the sample to the bias rule in the representation space. Then, we poison the identified bias-conflicting samples with error-minimizing label poisoning to save the bias rule that the model has currently learned. Finally, we update the biased model in the poisoned training data to ensure that the model continues to learn new biases. These three steps are executed iteratively to continuously accumulate more bias in the model and refine the model's ability to identify bias-conflicting samples.Benefiting from the accumulation of biases through data poisoning, the biased model comprehensively learns the biases present in the data, as illustrated in Fig. 1(c ). If the biased model can correctly classify a sample, then this sample is bias-aligned; otherwise, it is bias-conflicting. Leveraging pseudo-labels for bias features, we amplify the training weight of bias-conflicting samples to balance the target model's learning between bias-aligned and biasconflicting samples. However, bias-conflicting samples assigned excessive weight may quickly become overfitted in the target model, diminishing their effectiveness in influencing the model's learning. To mitigate this issue, we introduce the Group-wise Inverse Focal Loss to enhance the model's focus on overfitted bias-conflicting samples. Consequently, the target model is prevented from learning the bias rules present in the data, which promotes an unbiased learning process.We summarize our main contributions as follows:• We propose Poisoner, a novel unsupervised debiasing method that employs poisoning to expose the potential bias in the data and eliminates the bias via group reweighing. • We introduce a guided poisoning mechanism that encourages the biased model to specifically fit the bias features. This approach opens up new possibilities for the benign application of data poisoning. • We perform extensive experiments on commonly used benchmarks, and our proposed method achieves state-of-the-art performance in both fairness and accuracy. 
motivation and justification 2.1 preliminary: 
 Unsupervised debiasing methods involve the two models: the biased model and the target model. The biased model is trained on a dataset that solely contains the target label, and aims to subtly perceive the bias within the data. Subsequently, the target model performs debiasing based on the bias captured by the biased model. Evidently, the quality of the bias exposed by the biased model determines the performance of the target model's debiasing. Two prevalent choices exist for a bias model: an Empirical Risk Minimization (ERM) model or a Generalized Cross Entropy (GCE) model. Liu et al. [21, 27] leverage the cross-entropy loss to train an ERM model as a biased model. Nam et al. [18, 23] consider that samples that are easy to classify are more likely to be samples aligned with bias, and therefore propose to employ a GCE loss [38] that pays more attention to samples that are easier to classify to train the biased model. The GCE loss is defined as:GCE(𝑝 (𝑥; 𝜃 ), 𝑦) = 1 -𝑝 𝑦 (𝑥; 𝜃 ) 𝑞 𝑞 (1)where 𝑝 𝑦 is the softmax probability output for the target (train) label 𝑦, and 𝑞 ∈ (0, 1] is a constant that controls the degree of amplification of attention to samples that are easy to classify.For an ideal biased model, we expect it to fit well with bias-aligned examples and poorly with bias-conflicting examples to distinguish between these two types of samples. We train ERM model and GCE model over 50 epochs using commonly used datasets CelebA [22] .The target labels and the bias features are big-nose and gender, respectively. We document the error rate of the biased models on each group of training data after each epoch in Fig. 2 (a) and Fig. low, decreasing to nearly zero in the end. Although the error rate of bias-conflicting samples in the GCE model does not approach zero, the error rates at all moments throughout the entire training process are also relatively low, and show a decreasing trend. This indicates that the biased model fits the target label and fails to fully rely on bias.Early models are biased but not enough. In the early stages of ERM model and GCE model training, as illustrated in Fig. 2 (a) and Fig. 2 (b), the error rate for bias-conflicting samples is significantly higher than that for bias-aligned samples. This suggests that early models tend to rely heavily on bias features for decision-making. However, this is far from sufficient. For instance, at epoch 2, ERM model can detect 46% of the bias-conflicting samples (with a 46% error rate for bias-conflicting samples). The already scarce biasconflicting samples are further reduced by 54%.Error-minimizing label poisoning. Data poisoning, which aims to poison training samples so that models learn attackers' malicious intent, has recently drawn massive attention [2, 12] . Inspired by this, we propose the following idea: Can we leverage data poisoning to further reinforce biases that early models have already learned? This would ensure that the model, when trained on poisoned data, will further learn these biases. We propose Error-minimizing Label Poisoning to save the model's bias by poisoning the target label 𝑦 to the poisoned label ŷ:EQUATIONwhere L is cross-entropy loss. 𝜃 denotes the parameter of the model 𝑓 𝐵 . ŷ is assigned as the predicted label of the model ensuring errorminimizing. By minimizing errors, the samples (𝑥, ŷ) align with the bias rules that the model has already learned. However, this may present two issues. Firstly, due to the model's limited reliance on bias features at early stages, the poisoned data may only introduce a subset of the existing bias rules. For instance, when poisoning at Epoch 2 of the ERM model on CelebA, only the labels of 46% of the bias-conflicting samples were altered to new labels that conformed to the bias rules. Secondly, there is a possibility of noise, where label changes may not be solely attributed to the bias rules learned by the model. For instance, the labels of approximately 20% of the bias-aligned samples were modified to incorrect labels.Iterative Poisoning Bias-conflicting samples. To address the issue of the early model's limited reliance on bias features, we propose an iterative poisoning scheme to accumulate the biases learned by the model. To mitigate the noise issue in poisoning, we suggest focusing solely on poisoning the bias-conflicting samples. Bias-conflicting samples inherently contradict the bias, thus even if noise is introduced during the poisoning process, it will not undermine the bias rules within the data. This integration of ideas, termed Iterative Poisoning Bias-conflicting samples, can be formally expressed as:EQUATIONHere, D𝑐 ⊂ D represent the subsets of bias-conflicting samples. ŷ denote the modifiable label. The inner min serves to save model bias to poisoned data by poisoning only the bias-conflicting samples 𝑥 ∈ D𝑐 , while the outer min operation serves to accumulate more model bias.To validate the idea, we operate Iterative Poisoning Bias-conflicting samples under the assumption of prior knowledge regarding which samples are bias-conflicting. However, it's important to note that this assumption is solely for validation purposes; in reality, we do not know which samples are bias-conflicting, and we propose a method to identify bias-conflicting samples in Section 3.2. The outcomes of Iterative Poisoning Bias-conflicting samples are illustrated in Fig. 2(c ). Notably, the bias-conflicting samples are almost entirely misclassified, as indicated by the error rate calculated based on the original clean labels. Furthermore, we directly trained the model using bias labels rather than target labels to simulate the scenario of an ideal biased model, as depicted in Fig. 2(d) . Comparing Fig. 2(c ) with Fig. 2(d) shows that our proposed approach achieves performance similar to that of the ideal biased model in later stages. 
method 3.1 overview: 
 We propose a novel debiasing method, referred to as the Poisoner, depicted in Fig. 3 . This method incorporates two models. The first is a biased model, 𝑓 𝐵 , which is designed to uncover potential data bias. The second is a target model, 𝑓 𝑇 , which performs debiasing operations based on the bias identified by 𝑓 𝐵 .Biased model. Drawing inspiration from previous analyses, we propose guided data poisoning to ensure that the biased model is dependent on bias features. This will be provided in Section 3.2.Training data 5% 95% 10% 90% Target model. We reweight the training weight of bias-conflicting samples to balance the influence of bias-aligned and bias-conflicting samples for debiasing. We also propose Group-wise Inverse Focal Loss to mitigate potential challenges arising from overfitting biasconflicting samples. This will be discussed in Section 3.3.Section 2 demonstrated that the previous biased models failed to fully rely on bias. The idea of iteratively poisoning only the biasconflicting samples has been validated as a solution to this problem.In this section, we introduce Guided Data Poisoning, which is designed to guide the iterative poisoning of bias-conflicting samples.Identify. A model trained on a biased dataset tends to rely on bias features of samples for prediction. This leads to samples with identical target features but different bias features being separated in the representation space, and samples with similar bias features but different target features being brought closer together [10, 24] . This insight guides us to use similarity as a metric for identifying bias-conflicting samples. More specifically, considering that the majority of samples within each target class are aligned with these biases, and the samples that conflict with these biases deviate from these aligned samples and may even approach other target classes, we can calculate whether a sample is far from samples of the same class and close to samples of other classes to identify whether it is a bias-conflicting sample. We adopt the main idea of supervised contrastive loss [15] . For each sample 𝑥 𝑖 in the current batch, we compare 𝑥 𝑖 with the other data in the mini-batch B as follows:𝑑 con (𝑥 𝑖 ; 𝜃 ) = - 1 |𝐽 (𝑖)| ∑︁ 𝑗 ∈ 𝐽 (𝑖 ) 𝑙𝑜𝑔 exp 𝑧 𝑖 • 𝑧 𝑗 𝑎∈𝐴(𝑖 ) exp (𝑧 𝑖 • 𝑧 𝑎 ) (4)where𝑧 𝑖 = 𝑓 -2 𝐵 (𝑥 𝑖 )/||𝑓 -2 𝐵 (𝑥 𝑖 )|| is a normalized feature of sample 𝑥 𝑖 extracted from the penultimate layer 𝑓 -2𝐵 of biased model 𝑓 𝐵 . 𝐽 (𝑖) denotes the index set of samples in the current batch that have the same target label label 𝑡 -1 (𝑥 𝑖 ) as 𝑥 𝑖 . 𝐴(𝑖) represents the index set of all samples in the current batch B, excluding 𝑥 𝑖 . Furthermore, batch B is obtained through class-balanced sampling because we aim to consider all target classes fairly.The larger the 𝑑 con (𝑥 𝑖 ; 𝜃 ), the more likely it is that the sample 𝑥 𝑖 is a bias-conflicting sample of the current target class. Then, we compute the (100 -𝑝) 𝑡ℎ percentile within the {𝑑 con (𝑥 𝑖 ; 𝜃 )} | B | 𝑖=1 , denoted as 𝑞 𝑝 , where |B| is the size of mini-batch B. 𝑝 ∈ (0, 100) is a hyper-parameter that controls the proportion of bias-conflicting samples selected in a single batch. Subsequently, the indicator of whether a sample is a bias-conflicting sample can be formalized as:EQUATIONIn other words, 𝑥 𝑖 is considered a bias-conflicting sample if 𝑑 con (𝑥 𝑖 ; 𝜃 ) is greater than 𝑞 𝑝 .Poison. At the beginning of the training of the biased model, each sample's target label is initially the clean label without poisoning, i.e., label 𝑡 =0 (𝑥) = 𝑦, and 𝑦 is the clean label. During the training step 𝑡, we selectively poison the label of the samples that are identified as bias-conflicting (i.e., conflict (𝑥) =1) for error-minimizing min ŷ L (𝑓 𝐵 (𝒙), ŷ):label 𝑡 (𝑥 𝑖 ) = label 𝑡 -1 (𝑥 𝑖 ) if conflict (𝑥 𝑖 ) = 0 𝑌 (𝑓 𝐵 (𝑥 𝑖 )) if conflict (𝑥 𝑖 ) = 1 (6)where label 𝑡 is the poisoned label in iteration 𝑡. 𝑌 (𝑓 𝐵 (𝑥 𝑖 )) is the predicted label of biased model in current iteration 𝑡. By changing the label of the bias-conflicting sample 𝑥 𝑖 to 𝑌 (𝑓 𝐵 (𝑥 𝑖 )), the biases within the data are further reinforced. Calculate 𝑑 con for all samples in B, using Eq. 47:Get the bias-conflicting indicator (conflict(𝑥)), using Eq. 5 Upon completing the training phase for the biased model 𝑓 𝐵 , the model's output can be used to distinguish between bias-aligned and bias-conflicting samples. A sample 𝑥 𝑖 is considered bias-aligned if it is classified correctly, meaning its output aligns with the clean label 𝑦 𝑖 . Conversely, a sample is labeled as bias-conflicting if its classification contradicts the clean label.Group Reweighing. The primary source of model bias is the biased correlations inherent in the training data. To counteract this correlation and train an unbiased target model, we employ sample reweighing to rebalance the influence of samples that exhibit varying bias features. Specifically, the training weights of biasconflicting and bias-aligned samples are adjusted as follows:EQUATIONHere, D 𝑎 ⊂ D and D 𝑐 ⊂ D represent the subsets of bias-aligned and bias-conflicting samples, respectively, partitioned from the overall dataset D. The symbols |D|, |D 𝑎 | and |D 𝑐 | denote the number of samples in these sets. The hyperparameter 𝜆 controls the degree of additional emphasis given to bias-conflicting samples, typically ranging between 1 and 1.5. Since some bias-conflicting samples may not be correctly identified by 𝑓 𝐵 , even though they are few in number, we need to emphasize bias-conflicting samples additionally. Our subsequent experiments demonstrate that the parameter 𝜆 is not sensitive. Then, the loss function of the target model 𝑓 𝑇 , which is trained with weighted samples, can be formalized as follows:EQUATIONGroup-wise Inverse Focal Loss. However, assigning a larger training weight to the already scarce bias-conflicting samples can quickly lead to the target model overfitting these samples, as shown in Fig. 6 (a). That is, the probability of the target label 𝑦, denoted as 𝑝 𝑦 , tends to 1, resulting in the loss approaching 0. As a result, the contributions of bias-conflicting samples to the training of the target model become limited, hindering the debiasing.To address the negative effects of overfitting, we introduce the concept of group-wise inverse focal loss, which aims to more focus learning on overfitted bias-conflicting samples. To tackle the long-tail problem, Focal Loss [20] applies a dynamic scaling factor to the cross entropy loss to focus learning on hard misclassified examples, represented as L 𝐹 𝐿 = -1 -𝑝 𝑦 𝛾 log 𝑝 𝑦 , 𝛾 > 0, where the scaling factor (1 -𝑝 𝑦 ) 𝛾 decays to zero as confidence (𝑝 𝑦 ) in the correct class increases. In contrast, we aim to enhance focus learning on easy misclassified (i.e., overfitted) examples, thus we invert the Focal Loss to L = -1 -𝑝 𝑦 -𝛾 log 𝑝 𝑦 . However, thiswill not directly prompt the model to focus on bias-conflicting samples due to some bias-aligned samples also having a high 𝑝 𝑦 . To this end, we propose Group-wise Inverse Focal Loss:EQUATIONHere, 𝑝 𝑦 represents the probability that the target model 𝑓 𝑇 predicts sample 𝑥 as the correct class 𝑦. 𝑝 𝑎 𝑦 and 𝑝 𝑐 𝑦 respectively denote the average 𝑝 𝑦 values on bias-aligned and bias-conflicting samples in the current mini-batch. The focusing parameter 𝛾 > 0 smoothly adjusts the rate at which overfitted examples are prioritized (we found that 𝛾 approaching 1 works best in our experiments). If biasconflicting samples are overfitted (resulting in high 𝑝 𝑐 𝑦 values), they will be assigned a larger loss, directing the target model to pay more attention to these samples during training.Moreover, for numerical stability of L 𝐺𝐼 𝐹 𝐿 , we introduce an adaptive multiplicative factor 𝛽 = 1/ (1 -𝑝 𝑐 𝑦 ) -𝛾 . Thus, the overall training loss for the target model is given by:EQUATION 
experiment 4.1 experimental settings: 
 Datasets. We construct experiments on eight debiasing tasks across five benchmark datasets. CelebA, an industrial-scale dataset, contains about 200k facial images with 40 binary attribute annotations. We select target attributes that display the high Pearson correlation with gender, and perform gender debiasing for the recognition of each target attribute. The selected target attributes are bignose, attractive, blonde, and bags-under-eyes. We also use the Waterbirds [25] dataset, where waterbirds and land birds are highly correlated with wet and dry backgrounds, respectively. The proportion of bias-aligned samples that conform to this biased correlation is 95%. Our objective is to eliminate the influence of the background on bird recognition. The Dogs&Cats [16] dataset contains a fur color bias. Each animal species has a correlation of 0.95 with a specific fur color. Our goal is to eliminate the influence of the animal's fur color on animal recognition. For C-MNIST [17] , the task is to recognize digits (0-9), in which the images of each target class are dyed by the corresponding color with probability 𝜌, while the remaining samples are randomly colored with other colors. For the version of C-MNIST 1 we use, the probability 𝜌 is 0.99. For C-MNIST 2 , the probability 𝜌 is 0.98. Our goal is to eliminate color bias. Lastly, the ImageNet-B [36] dataset offers a complex and realistic testbed, featuring ten types of natural noise patterns as bias attributes. Each target class exhibits a correlation of 0.95 with a specific type of natural noise. Our ambition within this dataset is to counteract the influence of natural noise on object recognition.We aim to answer two main questions: (1) How does the fairness of Poisoner compare to other methods? (2) How does the classification performance of Poisoner compare with other methods?To answer the first question, we examine whether the accuracy of the model predictions changes with shifts in bias attributes (e.g., gender). We use Equalodds [8] to measure fairness. For instance, the measure of gender fairness on the CelebA dataset is as follows:EQUATIONwhere 𝑌 denotes target labels, Ỹ denotes outputs of models, and 𝑏 0 and 𝑏 1 represent different groups in terms of bias attributes such as male and female. Considering that Colored-MNIST and ImageNet-B not only have two types of bias features, we use the difference between the average accuracy and the worst group accuracy on each target class as model bias.To answer the second question, we first divide the test set into different groups based on the bias attributes and target attributes. Then, we report two types of accuracy: the Average Group Accuracy and the Worst Group Accuracy.Baselines. To evaluate the effectiveness of our method, we compare it against prior methods including Vanilla, Focal loss [20] , LfF [23] , JTT [21] , DebiAN [19] , Echoes [11] , BE [18] , GroupDRO [25] . The Vanilla model, trained solely with the original cross-entropy loss, employs no debiasing strategies. BE is a method that relies on LfF or other debiasing methods, and we use it in conjunction with LfF. GroupDRO explicitly leverages the bias labels (e.g., the gender labels) during the training phase, while others require no prior knowledge of the biases.Implementation. Following previous research, we utilize a multilayer perceptron (MLP) with three hidden layers for the Colored MNIST dataset. For ImageNet-B, we use ResNet-34 [9] . For other datasets, we employ ResNet-18 [9] . As for the parameter 𝑃 in our method, which controls the percentage of samples identified as biasconflicting in each training step of the biased model, we set it to 10 for CelebA and 5 for the other datasets. We set the hyperparameter 𝛾 to 1 for all experiments. We use the Adam optimizer for all baselines with a learning rate of 1e-3.Debiasing with social bias. Table 1 shows the model bias, average group accuracy, and the worst group accuracy of models on four target tasks of CelebA, with gender as the bias attribute. Vanilla models record severe model bias as they are optimized to capture the biased statistical properties of training data without any constraints. Various debiasing methods demonstrate different levels of effectiveness in bias mitigation. In comparison with other unsupervised debiasing methods (which do not require prior knowledge regarding bias), the proposed Poisoner achieves the best performance in terms of fairness and accuracy across all four target tasks. For instance, for the big-nose recognition task (T=𝑏𝑛), Vanilla models display significant fairness issues (model bias = 31.40%), with the worst group accuracy being only 43.36%. Our proposed poisoner reduces the model bias to 6.56% and improves the worst group accuracy to 74.49%. The debiasing performance of other methods is limited in comparison. Our method's state-of-the-art performance in unsupervised debiasing confirms our biased models precisely identify bias-conflicting samples. Furthermore, we also compared our method with GroupDRO, which requires the use of bias labels in training. Compared to GroupDRO, our method achieves competitive results, even achieving better performance in T=𝑎 task.Debiasing with general bias. Beyond the social bias that is central to fairness research, we have also benchmarked our method against other methods on four datasets that encompass general biases. The results, as depicted in Table 2 , demonstrate that our method consistently achieves superior debiasing performance. This underscores that the effectiveness of our method is not contingent on the type of bias attribute.An ideal biased model should have a lower error rate for biasaligned samples and a higher error rate for bias-conflicting samples in the training data, thus facilitating the delineation of pseudolabels for bias features. Initially, we demonstrate the comparative advantages of our biased model over the ERM and GCE models, as illustrated in Fig. 4 . As training progresses, our method gradually separates bias-aligned and bias-conflicting samples. Lastly, the biasconflicting samples are almost entirely misclassified, indicating that these samples have been almost wholly mined. In contrast, both ERM and GCE fit bias-conflicting samples and fail to mine them. Furthermore, we introduce a metric called Bias Accuracy to quantitatively assess the biased model's efficacy in exposing biases. Specifically, Bias Accuracy is calculated as the average of the accuracy and error rate of the biased model on bias-aligned and biasconflicting samples, respectively. We present the Bias Accuracy achieved by various biased models on different datasets in Table 3 . It is evident that our biased model significantly outperforms alternative methods in exposing biases. Moreover, our biased model exhibits overwhelming superiority for tasks characterized by lower degrees of dataset bias (i.e., biases are less apparent), such as the four tasks within the CelebA dataset. This observation suggests the applicability of our approach not only in scenarios characterized by high levels of bias intensity but also in contexts where biases are deeply embedded and less discernible.Real-world datasets often exhibit varying degrees of bias. To demonstrate the versatility of our method in scenarios with less extreme bias, we manipulate the bias intensity in the CelebA training data by removing some samples. Fig. 5 (left) illustrates our method's debiasing performance under different data bias scenarios, with the degree of data balance 𝛼 representing the ratio of bias-conflicting samples to bias-aligned samples. A higher 𝛼 indicates lower data bias intensity. Our method shows significant bias mitigation across all degrees of data balance, indicating its effectiveness in both extreme and less pronounced bias scenarios. Conversely, the GCE-Based LfF method fails to mitigate bias for high data balance scenarios (𝛼>0.5) and significantly decreases accuracy.Moreover, we attribute this universal debiasing capability to the efficient uncovering of bias by our biased model across various degrees of data balance. Fig. 5 (right) displays the metric measuring the effectiveness of uncovering bias, the Bias Accuracy, with our biased model showing notable superiority over others. This further underscores the versatility of our approach. Group-wise Inverse Focal Loss (GIFL) and 𝛾. We first demonstrate the overfitting issues of the target model on bias-conflicting samples under the condition of Group Reweighting without GIFL. In Fig. 6 (a), the average cross-entropy (CE) losses on bias-conflicting and bias-aligned samples are presented. It can be observed that the loss on bias-conflicting samples remains consistently lower than that on bias-aligned samples, and quickly decreases to zero. This indicates that bias-conflicting samples are being overfitted, resulting in bias-aligned samples being the main influence during the target model training process. In Fig. 6 (b), we illustrate the model bias of the target model with GIFL under different 𝛾 values. It can be seen that as 𝛾 approaches 1, the model bias decreases. We observe the same trend on other datasets as well. Furthermore, Table 4 compares the results between the Poisoner with and without GIFL. Across all tasks, both the fairness and accuracy of the model degrade when GIFL is not applied. This underscores the effectiveness of GIFL in enhancing both the fairness and accuracy of the model.Different values of 𝑝 and 𝜆. We use 𝑝 ∈ (0, 100) to control the percentage of samples identified as bias-conflicting in each training round of the biased model. Fig. 7 illustrates the accuracy of bias identification for different values of 𝑝. When 𝑝 is relatively large, numerous bias-aligned samples are misclassified as bias-conflicting. This misidentification contradicts the original intent of guided data poisoning, resulting in noisy injections. Therefore, setting 𝑝 within the range of 5 to 20 is considered appropriate. We also examine debiasing performance with different 𝜆 values. The results indicate that 𝜆 exhibits insensitivity to debiasing performance. 
related work: 
 Bias Mitigation. Early works on debiasing relied on prior knowledge about the biases present in the data (known as supervised debiasing). Some approaches [10, 24, 25, 32, 34, 35, 37] required explicit bias labels for each training sample, such as the gender of facial images. For instance, Zhang et al. [34, 35] employed adversarial training to minimize a discriminator's ability to predict bias labels, thus encouraging fair outputs.Recent studies focus on the more realistic unsupervised debiasing: debiasing without any prior bias information. Unlike supervised debiasing, unsupervised debiasing first requires obtaining information about the biases. Typical approaches [17, 21, 23] assume that biased features are more readily learned by models compared to robust features. Thus, they train an auxiliary biased model that is expected to primarily rely on the biased features, considering the auxiliary model's outputs as pseudo-labels for bias. Some methods [11, 18] attempt to construct intentionally biased datasets to train the auxiliary biased model, while others utilize networks with limited capacity to uncover biases [26] . Furthermore, feature clustering [29] and fairness minimization [19] techniques have been employed to discover unknown biases.Poisoning Attack. Poisoning attacks [2, 3, 7] in machine learning have emerged as a significant security concern, where adversaries manipulate training data to compromise the integrity and performance of models. Poisoning attacks involve injecting malicious data samples [13, 30] , known as poisoned examples, into the training dataset. These examples are carefully crafted to deceive the learning algorithm and embed the attackers' malicious intent into the model's decision-making process. To counter poisoning attacks, various defense mechanisms have been developed [5, 6, 31] . 
conclusion: 
 In this study, we introduce poisoner, a novel unsupervised debiasing method. The poisoner couples guided data poisoning and model training to prompt the biased model to learn more bias continuously. Subsequently, it eradicates the bias uncovered by the bias model in the target model by assigning more attention to bias-conflicting samples. Experiments demonstrate its effectiveness in improving fairness and accuracy. 
 
 Appendix:  


Decision: The paper introduces a novel debiasing method called "Poisoner," which leverages data poisoning to address biases in neural networks. This approach embeds biases into the training data to encourage models to learn and subsequently mitigate these biases. Reviewers praised the innovative use of data poisoning for a beneficial purpose, the unsupervised nature of the method, and the comprehensive experimental validation demonstrating state-of-the-art performance. However, they also noted several limitations, including the need for more comparisons with existing methods, potential issues with the complexity and computational demands of the approach, and some unclear methodological explanations. Despite these limitations, the reviewers generally found the work novel and impactful, leading to a mixed consensus with a lean towards acceptance.

Reviews: review: paper_summary: paper_summary: The paper titled "Poisoning for Debiasing: Fair Recognition via Eliminating Bias Uncovered in Data Poisoning" addresses the issue of biases in neural networks, which often rely on spurious correlations between features and target labels. Traditional debiasing methods require bias feature annotations, which are labor-intensive. Recent approaches use two models: a bias model to detect biases and a target model to mitigate them. However, these methods often fail as the bias model ends up fitting target labels instead of exposing biases.
To resolve this, the paper proposes a novel method called Poisoner, which employs data poisoning. This method embeds biases learned by biased models into poisoned training data, prompting the models to learn more biases. The process involves:
Guided Data Poisoning: Iteratively embedding bias rules into training data by altering target labels.
Model Training: Using poisoned data to ensure the biased model learns only biases.
Bias Identification: Identifying bias-conflicting samples and amplifying their influence during target model training to prevent learning biased correlations.
The approach aims to balance the influence of bias-aligned and bias-conflicting samples, leading to an unbiased learning process. Extensive experiments demonstrated that Poisoner achieves superior debiasing performance compared to existing methods.
strength_weakness: strength_weakness: Innovative Use of Data Poisoning: The method leverages data poisoning, traditionally seen as a malicious tactic, for a beneficial purpose in debiasing.

Unsupervised Approach: Poisoner does not require labor-intensive bias feature annotations, making it more practical for real-world applications.

Iterative Refinement: The iterative nature of the method allows for continuous improvement in identifying and mitigating biases.

State-of-the-Art Performance: Extensive experiments show that Poisoner achieves superior performance in both fairness and accuracy on commonly used benchmarks.

Practicality: The method is designed to work without prior knowledge of bias types, making it adaptable to various datasets and scenarios.
limitations: limitations: I am not quite familiar with the field. However, based on the descriptions from the authors (the bias is caused by spurious features), it seems that many OOD generalization methods can be quite helpful (e.g., IRM), especially considering the cases where these methods may not required extra labels. I wonder if the authors could further discuss with the related methods in OOD generalization.

The suggested method seems to critically rely on the original biased model. If the biased model makes a lot of errors, the suggested method will not try to correct it. Instead, it will accumulate the errors and further bias the model. I think it is the key problem of the suggested method, and the authors should justify why the suggested method is robust to the errors in biased model.

It seems that the suggested method is computing demanding (two models, extra forward/backward propagation). Therefore, could the authors compare the required time of the suggested method and many other baselines.
paper_summary: paper_summary: Neural networks can suffer from bias issues, relying excessively on features that have spurious but strong correlations with target labels. This can lead to poor performance on data that does not follow these correlations. Traditional debiasing methods require labeling bias features, which is challenging and labor-intensive. Recent work tackles this by training two models: a bias model to identify bias features and a target model to eliminate the bias. However, these bias models often fit the target labels, failing to properly expose data bias.

To address this, the paper proposes a novel debiasing method called "poisoner" that utilizes data poisoning. It embeds the biases learned by the bias model into the training data, encouraging the models to learn more biases. This is achieved by coupling guided data poisoning and model training, which continuously prompts the bias model to identify and learn more biases. Then, samples that contradict these biased correlations are identified and their influence is amplified during the training of the target model, preventing it from learning such biased correlations. Experiments show that this method achieves superior debiasing performance.
strength_weakness: strength_weakness: 1. This paper is well-written and easy to follow.

2. The experiments are sufficient and convincing to show the effectiveness of the proposed method.

3. I am not very familiar with this area, but this method is not trivial and seems novel.
limitations: limitations: 1. More comparisons with existing methods such as Chroma-VAE are suggested.

2. It is better to clarify "label poisoning" in the title.


[1] @inproceedings{NEURIPS2022_80098914,
 author = {Yang, Wanqian and Kirichenko, Polina and Goldblum, Micah and Wilson, Andrew G},
 booktitle = {Advances in Neural Information Processing Systems},
 editor = {S. Koyejo and S. Mohamed and A. Agarwal and D. Belgrave and K. Cho and A. Oh},
 pages = {20351--20365},
 publisher = {Curran Associates, Inc.},
 title = {Chroma-VAE: Mitigating Shortcut Learning with Generative Classifiers},
 url = {https://proceedings.neurips.cc/paper_files/paper/2022/file/80098914b3b3bad79b80377751a85430-Paper-Conference.pdf},
 volume = {35},
 year = {2022}
}
paper_summary: paper_summary: This paper proposes a novel debiasing method based data poisoning method to embed the biases into the poisoned training data, thereby facilitating the training of a more effectively biased model, which is subsequently used to identify unbiased samples. The idea in this paper is straightforward with detailed explanations and sufficient experimental support.
strength_weakness: strength_weakness: 1. This paper provides a well-articulated introduction and justification, making it easy for readers to comprehend the field of biased learning.

2.  The motivation and methodology presented are coherent, which utilize the correlation between the poison data-based attack and data bias to improve the biased model. 

3. The experimental results presented in this paper are both comprehensive and extensive.
limitations: limitations: While I am not an expert in this field, I believe it would be more convincing to provide theoretical insights that analyze the benefits brought by the poisoned data.
paper_summary: paper_summary: This paper proposed Poisoner, a novel unsupervised debiasing method that employs poisoning to expose the potential bias in the data and eliminates the bias via group reweighing. Extensive experiments verify its effectiveness.
strength_weakness: strength_weakness: 1.The paper structure is OK. This paper introduces its motivation for proposing their approach, which can make the proposed scheme more convincing. Actually, I am not familiar with this topic, but the scheme seems OK to me.

2.The experimental results are comprehensive. The experimental results seem extensive and convincing for me.

3. The writing is good and easy to understand.
limitations: limitations: 1. Confused descriptions. In line 164, the authors claim that they propose a unsupervised scheme. However, In lines 251-252, the authors propose Error-minimizing Label Poisoning approach, which seems contradictory to the unsupervised scenario.

2. Typos: Section 4.3 "The Efficacy of Our Biased Model in uncovering Bias" -> "The Efficacy of Our Biased Model in Uncovering Bias"