Data-Centric Explainable Debiasing for Improving Fairness in Pre-trained Language Models (2024)

Abstract: Human-like social bias of pre-trained language models (PLMs) on downstream tasks have attracted increasing attention. The potential flaws in the training data are the main factor that causes unfairness in PLMs. Existing datacentric debiasing strategies mainly leverage explicit bias words (defined as sensitive attribute words specific to demographic groups) for counterfactual data augmentation to balance the training data. However, they lack consideration of implicit bias words potentially associated with explicit bias words in complex distribution data, which indirectly harms the fairness of PLMs. To this end, we propose a Data-Centric Debiasing method (named DataDebias), which uses an explainability method to search for implicit bias words to assist in debiasing PLMs. Specifically, we compute the feature attributions of all tokens using the Integrated Gradients method, and then treat the tokens that have a large impact on the model’s decision as implicit bias words. To make the search results more precise, we iteratively train a biased model to amplify the bias with each iteration. Finally, we use the implicit bias words searched in the last iteration to assist in debiasing PLMs. Extensive experimental results on multiple PLMs debiasing on three different classification tasks demonstrate that DataDebias achieves state-of-the-art debiasing performance and strong generalization while maintaining predictive abilities.

heading: 1 Introduction
text: In recent years, pre-trained language models (PLMs), such as the BERT series (Devlin et al., 2019; Sanh et al., 2019; Liu et al., 2019) and GPT series (Radford et al., 2018, 2019; Brown et al., 2020) have seen vigorous development in natural language processing. By training on massive text datasets, PLMs can acquire rich linguistic knowledge. However, these training datasets contain
* Corresponding author
human-like social biases and stereotypes (Zhao et al., 2019). PLMs can learn and amplify biases against certain demographic groups, leading to unfair decisions that harm vulnerable groups Therefore, mitigating social biases in PLMs is a critical issue and necessary to improve the fairness of natural language systems.
Current debiasing methods usually take the model-centric strategies (Liang et al., 2020; Cheng et al., 2021), seeking to improve fairness by modifying model architectures or adding regularization during training without considering the defects of the training data (Han et al., 2021). However, they ignore critical issues of training data, including real-world stereotypes, discrimination against vulnerable groups, and imbalanced samples across demographic groups. Data quality issues fundamentally enable language models to learn biased semantics. Furthermore, some methods are difficult to generalize to large-scale PLMs due to the limitations of the fine-tuning model.
In contrast to the model-centric strategies, the data-centric debiasing strategies focus on addressing defects in training data to improve data quality (Zha et al., 2023). The most general approach utilizes counterfactual data augmentation (Lu et al., 2020) to alleviate the imbalance of samples from different demographic groups. These debiasing methods generate synonymous sentences based on priori sensitive attribute words specific to different demographic groups (e.g., male/female, white/black). Although they consider sensitive attribute words that directly cause model biases, they ignore potentially harmful associations present in the training data that indirectly cause unfairness. We refer to sensitive attribute words specific to demographic groups as explicit bias words, and to tokens in the training data that have potentially harmful associations with explicit bias words as implicit bias words (illustrative examples are given in Figure 2). We formulate the following hypoth-
3773
esis: Under the indirect effect of implicit bias words, PLMs capture the spurious associations between demographic groups and certain class labels to make unfair and biased decisions.
Based on the above assumptions, in this work, we propose a Data-Centric Debiasing method (named Data-Debias), which uses an explainability method to search for implicit bias words to assist debiasing PLMs. Specifically, we compute the feature attribution scores for all tokens in the training data using Integrated Gradients (Sundararajan et al., 2017) which is a post-hoc explainability method. Implicit bias words are selected as tokens with large differences in feature attribution scores across demographic groups, and then used to assist debiasing. To more precisely target implicit bias words that are strongly associated with explicit bias words, we iteratively train a bias-amplified model. Within each iteration, we train the model with the implicit bias words to amplify the model’s bias, and then re-search and update the implicit bias words with the biased model. After the iteration, the last updated more biased implicit bias words are used to assist debiasing to improve the fairness of PLMs.
Our main contributions are summarized as follows: 1) We propose a data-centric debiasing framework, which mines implicit bias words that have potentially harmful associations with explicit bias words, to improve the quality of the training data more comprehensively for stronger debiasing effects. 2) The obtained implicit bias words are interpretable and precise, ensured by searching via explainability methods and iteratively training the bias-amplified model. 3) Experiments on classification tasks on multiple PLMs demonstrate that Data-Debias outperforms state-of-the-art baseline models in debiasing while preserving predictive abilities. 4) Debiasing experiments on large-scale language models under the prompting paradigm verify the generalization of implicit bias words.

heading: 2 Methodology
text: In this section, we introduce the proposed DataDebias debiasing framework, as shown in Figure 1. Data-Debias has three stages: 1) Searching for implicit bias words via an explainability method. 2) Iterative training of a bias-amplified model using implicit bias words. 3) Debiasing training using the final implicit bias words.

heading: 2.1 Search for Implicit Bias Words via Explainability
text: In this stage, we aim to mine implicit bias words that have potentially harmful associations with the explicit bias words. The motivation is that in the model’s decision process, if a token has inconsistent attention among the sample pairs containing the explicit bias words, there is a harmful association between it and the explicit bias words that indirectly causes the model to make an unfair decision. To this end, we first perform data augmentation based on explicit bias words to generate sample pairs specific to demographic groups. We then apply the explainability method proposed by Slack et al. (2021) to the augmented sample pairs to search for tokens with inconsistent attention as implicit bias words.
For a certain social bias, we augment the dataset with prior explicit bias words. We focus on gender bias in this paper1. Following previous work (Bolukbasi et al., 2016; Liang et al., 2020; Cheng et al., 2021), we set the gender-specific list of explicit bias word pairs as: {FEMALE, MALE}={(woman, man), (girl, boy), (female, male), (she, he), (mother, father), (daughter, son), (gal, guy), (her, his), (herself, himself), (Mary, John)}, alongside plural forms.
We use the samples containing the explicit bias words as the original samples for the subsequent amplifying bias training and debiasing training. The augmented samples of the original samples
1While it may be different in the real-world, we chose the binary (male, female) as our target demographic group.
are used to mine the implicit bias words. Specifically, for each original sample x, we replace all explicit bias words in x by each word pair in the list, obtaining a set of augmented sample pairs {(xf1 , xm1 ), · · · , (xfj , xmj ), · · · , (x f N , x m N )}, where N is the number of augmented sample pairs and also the number of explicit bias word pairs.
To mine explainable implicit bias words, we adopt Integrated Gradients (IG) (Sundararajan et al., 2017) which is a versatile explainability method with low time complexity. IG’s idea is to aggregate gradients along the input that fall on the line between the baseline and the input. Given a sample x, the feature attribution of the kth token t in x is defined as the path integral of straight path between baseline x′k and input xk:
attrit ::= (xk−x′k) ∫ 1
α=0 ∂Fy(x ′ + α(x− x′)) ∂xk dα,
(1)
where the function Fy represents the PLM-based classification model that predicts the true label y of the sample x, ∂Fy(x)∂xk represents the gradient of Fy(x) on the kth feature. The standard practice for x′ uses zero embedding vectors.
Based on our hypothesis, implicit bias words indirectly cause PLMs to capture spurious associations between demographic groups and class labels. Therefore, in IG algorithm, the same implicit bias word gets different attention in different gender samples. Specificly, for each augmented sample pairs (xfj , x m j ) of the original sample x, we compute the attribution (attrifj,t, attri m j,t) of token t in both samples. To obtain the impact gap of the token t between female and male samples, the two attributes are subtracted and averaged across all augmented sample pairs. This impact gap is the bias score of token t, which is defined as:
Scoret = 1
N ∑N i=1 (attrifj,t − attrimj,t). (2)
To obtain biased words that have a discriminative impact on different gender groups, we filter out tokens that are smaller than the bias threshold θ and add the remaining tokens to the implicit bias word list W = {w1, w2, · · · }.

heading: 2.2 Iteratively Training Bias-amplified Model
text: The initial list of implicit bias words can be directly used to debias PLMs. However, given the
stability of search results, we iteratively train a biasamplified model to identify implicit bias words more precisely. Previous research has shown that smaller models tend to be more biased than larger models (Sanh et al., 2021; Ahn et al., 2022; Silva et al., 2021). Therefore, we consider Tiny-BERT (Bhargava et al., 2021), a small variant of BERT, as the biased model. This provides the benefits of amplifying bias while reducing training overhead.
The iterative process is as follows. In the first iteration, we apply IG to the original Tiny-BERT to obtain an implicit bias word list. This list assists in training Tiny-BERT to amplify the model bias. The trained Tiny-BERT is then reapplied IG to search and update the implicit bias word list. In subsequent iterations, the list of the last iteration is used to further amplify Tiny-BERT’s biases and is updated again. This process is repeated to iteratively grow a more precise list of implicit bias words.
The training objective is to ensure the model’s accuracy in downstream tasks while making unfair decisions for different demographic groups. Given a batch of original samples {xi|Bi=1}, the task loss is calculated by the cross-entropy of the predicted label and the true label, defined as follows:
Lce = 1
B ∑B i=1 CrossEntropy(yi, Fy(xi)), (3)
where B is the batch size, yi and Fy(xi) denote the true label and the predicted label for the model output. We then add a bias regularization term that applies Jensen-Shannon divergence (JSD) and reweights the training samples to amplify the model bias. For each sample xi, we define a bias degree to reweight it by summing the bias scores of all implicit bias words contained in xi and then globally normalizing. The bias degree is calculated as:
Di = Norm( ∑
w∈W ′ Scorew), (4)
where w is the implicit bias word in xi and Scorew is its bias score, W ′ is obtained by sorting the implicit bias words in W from largest to smallest by their bias score Scorew and then keeping the top ϵ%. Norm(·) is the max-min normalization function computed as value−minmax−min , where max is the maximum value in the batch and min is the minimum value in the batch. The bias degree of original sample is used as the weight of its augmented sample pairs. The bias regularization term is:
Lb = −1 NB
N∑
j=1
B∑
i=1
Di,j × JSD(pfi,j , pmi,j), (5)
where N is the number of augmented sample pairs, pfi,j and p m i,j are probability distributions of the jth female augmented sample and male augmented sample of xi, respectively. JSD(·) measures the consistence between the two gender distributions, and its larger value indicates that the two distributions are less similar. Therefore, to differentiate the two distributions so that the model makes inconsistent decisions, we take negative values for the results of JSD.
The overall loss function for amplifying bias training is as the combination of the cross-entropy loss Lce and the bias regularization term Lb: Lbias = Lce + λ×Lb, where λ is the hyperparameter that regulates the weight of the bias regularization term. The trained biased model is then applied IG again to search for more precise implicit bias words. In turn, more precise implicit bias words catalyze the process of amplifying model bias. In the last iteration, we obtain the implicit bias words that have a strong harmful association with the explicit bias words.

heading: 2.3 Debiasing Training
text: In the debiasing stage, we use the implicit bias words searched in the last iteration to assist the model debiasing. We expect to improve the model’s fairness without compromising the predictive ability in downstream tasks. Consistently, the task loss is the same as Eq.(3) and the debiasing term reweights the JSD using the bias degree. Differently, the goal of the debiasing term is to align the probability distribution of the augmented samples of two genders. Thus, the debiasing regularization term is defined as:
Ld = 1
NB
N∑
j=1
B∑
i=1
Di,j × JSD(pfi,j , pmi,j). (6)
The overall loss function of debiasing training is: Ldebias = Lce + γ × Ld, where γ is a hyperparameter that regulates the weight of the debiasing regularization term.
Samples with higher bias degrees contain more harmful associations related to gender. Reweighting changes the attention of samples, so that the PLM focuses on potentially harmful biased samples during training. The task loss function ensures the predictive ability of PLM in downstream tasks, and the debiasing regularization term encourages the model to learn consistent representations and thus make fair decisions. Therefore, PLM is im-
proved for fairness with auxiliary training of implicit bias words.

heading: 3 Evaluation Methods
text: We evaluate the performance of Data-Debias on the task metrics and the fairness metrics. Task Metrics. Four general accuracy metrics evaluate the predictive abilities of PLMs on all classification tasks, which are Accuracy, F1 score, Precision, and Recall. They are all calculated based on the predicted and true labels of the samples in the test set, and a higher value indicates the better predictive abilities of PLM. Fairness Metrics. In the existing debiasing research, the external metrics to measure fairness are specific to a certain task and are not applicable to other tasks. To this end, we propose two fairness metrics common to all classification tasks, which are calculated based on the prediction results of each augmented sample set.
The first metric we propose to evaluate the fairness of PLMs is FalseRatio/TrueRatio defined as FR/TR = num(ym ̸=yf )/Ntestnum(ym=yf )/Ntest = num(ym ̸=yf ) num(ym=yf ) , where ym and yf are the labels that the model predicts for male and female augmented samples, Ntest is the size of test set, and num(·) is the number statistics function. An ideal language model should make decisions based on the semantics and nothing else. Based on this, a fully unbiased model should give the same predicted label for augmented samples of the same sample. A lower value of FR/TR indicates a higher fairness of the model.
We propose a second metric to jointly evaluate the fairness and predictive abilities of PLMs, which is the TruePositiveRatio defined as TPR = num(ym=yf=ytrue)
num(ym=yf ) , where ytrue is the true label of the sample. We expect the model to improve fairness while maintaining the accuracy of the predictions. TPR will decrease as the fairness increases when the prediction ability of the model is constant. We observe the change of TPR in the experiment to measure the overall performance of the model.
In addition, we also choose F1 score, Precision, and Recall as fairness indicators, whose inputs are all the prediction results of the male augmented sample ym and the prediction results of the female augmented sample yf .

heading: 4 Experiments
text: In this section, we experimentally analyze the performance of the Data-Debias framework to answer
the following research questions. Q1 : How explainable are the detected implicit bias words? Q2 : Compared to baseline methods, how effective is applying Data-Debias to debias PLMs in downstream tasks? Q3 : What is the role of iteratively training the bias-amplified model? Q4 : Whether implicit bias words generalize to debiasing PLMs in the prompting paradigm?

heading: 4.1 Experimental Setting
text: Downstream Tasks. For downstream debiasing, we chose three classification tasks corresponding to three datasets with different data sizes. Each dataset is matched and augmented based on a given list of explicit bias word pairs, and the train-test split is 50/50. Stanford Sentiment Treebank (SST2)2 (Socher et al., 2013) is a manually annotated binary sentiment analysis dataset to judge single-sentence movie reviews, which matches a total of 4,133 original samples containing gender attributes. ToxiGen3 (Hartvigsen et al., 2022) is an implicit toxic text classification dataset automatically generated based on prompts with revealed information, which matches 38,000 original samples containing explicit bias words. The prompt label for each sample is taken as the true label. Bias in Bios4 (De-Arteaga et al., 2019) is a third person biography dataset with 28 occupation categories and 250,000 original samples scraped. The task objective is to predict the category of occupation for each biography with the occupation information removed. Comparing Baselines. We choose three modelcentric debiasing methods INLP (Ravfogel et al., 2020), Sent-Debias (Liang et al., 2020), as well as FairFil (Cheng et al., 2021), and three data-centric debiasing methods CDA (Lu et al., 2020), AutoDebias (Guo et al., 2022), as well as MABEL (He et al., 2022) as baselines. They are described as follows:
• INLP (Ravfogel et al., 2020) proposes an Iterative Null-space Projection debiasing method, which removes sensitive information from the neural representation by repeatedly training a linear classifier to predict the debiasing target and then projecting the representation on the null space.
2https://dl.fbaipublicfiles.com/glue/data/SST-2.zip 3https://github.com/microsoft/TOXIGEN 4https://github.com/microsoft/biosbias
• Sent-Debias (Liang et al., 2020) contextualizes predefined explicit bias words to the given sentence templates, and then debiasing by estimating the bias subspace of the sentence representation and eliminating projections on the bias subspace.
• FairFil (Cheng et al., 2021) proposes a fairness filter neural network to filter biases in sentence representations, which minimizes the correlation between filtered embeddings and explicit bias words via contrastive learning.
• CDA (Lu et al., 2020) enables debiasing research based on counterfactual data augmentation, which employs causal interventions to generate a copy of each sample by replacing the target word with a synonym.
• Auto-Debias (Guo et al., 2022) automatically searches for biased prompts in the top 5,000 most frequent Wikipedia vocabulary to debias by identifying the words that elicit the maximum JSD divergence between explicit bias words and stereotype words.
• MABEL (He et al., 2022) uses the priori explicit bias words to generate gender-balanced counterfactual entailment pairs in the natural language inference dataset, and introduces contrastive learning to narrow the representation between entailment pairs.
Implementation Details. We verify the effectiveness of debiasing on three PLMs: BERT (Devlin et al., 2019), DistilBERT (Sanh et al., 2019), and RoBERTa (Liu et al., 2019). The checkpoints of all models use bert-base-uncased, distilbert-base-uncased, and roberta-base implemented by the Huggingface Transformers library (Wolf et al., 2020). The [CLS] output from the last layer of PLM’s encoder is pooled and fed to a classifier with one linear layer for prediction. All PLMs use Tiny-BERT to search for implicit bias words. In all experiments on three downstream tasks, the batch size B is set to 32, the learning rate is set to 5e− 5, and the hyperparameters λ and γ are set to 1 and 10. The number of steps M of the interpretability algorithm is set to 50 and the bias threshold θ is set to 0. The iteration number of amplifying bias training is chosen from 0 to 6, and the implicit bias words list W ′ in the amplifying bias training stage is selected to be the top ϵ = 30%.
We implement INLP, Sent-Debias, and CDA using the code provided in (Meade et al., 2022), implement FairFil and Auto-Debias using the code provided by their authors, and MABEL using the checkpoints provided by its authors. CDA uses the same list of explicit bias word pairs as DataDebias. All baselines are fine-tuned in our tasks and consistent with our experimental setup.

heading: 4.2 Implicit Bias Words
text: To answer research question Q1, we exhibit the top-scoring implicit bias words searched from the SST2 and rank them by bias score, as shown in Figure 2. From the reported results, the detected implicit bias words are intuitively explainable because they appear to have potential associations with gender. We highlight some representative words with different colors. The occurrence of race-related words marked in yellow indicates that the PLMs bias against gender groups is influenced by racial groups. These words are interpretable,
for example black women are always discriminated against more than white women in the real-world. Blue highlights identity-related words, many of which contain gender stereotypes such as comedian, witch, and physician, which can induce gender bias in PLMs. Green highlights words that may have gender stereotypes, such as sensual being associated with women and rude being associated with men. PLM’s focus on these words may lead to unfairness in the decision making. Other unhighlighted words may be more implicit or seemingly irrelevant. In general, it is difficult to notice the associations between these words and gender and ignore them, resulting in unsatisfactory debiasing results. Our proposed Data-Debias can comprehensively compensate for the potential defects in the data by searching for implicit bias words, thus achieving more powerful debiasing performance.

heading: 4.3 Debiasing Ability Analysis
text: To answer Q2, we apply Data-Debias to three PLMs in three downstream tasks. For all experiments, we evaluate their results for both task and fairness, as shown in Tables 1, 2, and 3. We report the results of Data-Debias using implicit bias words obtained in the fourth iteration of amplifying bias training. We show the results of original PLMs and present the change in the task metrics for debiasing model compared to the original model.
Table 1 shows the debiasing results in SST2. All three PLMs fit well and exhibit excellent accuracy and fairness due to the small size of the SST2 dataset. Interestingly, Data-Debias improves fairness even further. For both BERT and DistillBERT, it shrinks FR/TR to near 0 and improves
F1, precision, and recall to close to the ideal score of 100%, while the overall accuracy of the task is not damaged. In the case of RoBERTa, DataDebias reduces bias and even improves predictive ability. The debiasing results on ToxiGen in Table 2 show that all debiasing methods impair the predictive ability to varying degrees. This is inevitable because fairness and accuracy are difficult to achieve at the same time. Data-Debias minimizes the damage to accuracy while greatly debiasing, which benefits from our debiasing strategy that combines the task objective and the debiasing objective. Table 3 shows the debiasing results on Bias in Bios. Compared with other debiasing methods, Data-Debias not only obtains the fairest score, but also performs the best overall in the task metrics.
From the debiasing results of the three tasks, all the baseline methods perform the debiasing ability
to some extent. Overall, the three data-centric baselines CDA, Auto-Debias, and MABEL are more stable in debiasing than the three model-centric baselines INLP, Sent-Debias, and FairFil, achieving more effective debiasing with less degradation in task performance. However, the gap is marginal, because while data-centric baselines capture the direct harm caused by explicit bias words, they ignore potentially harmful associations in the data. DataDebias achieves more robust debiasing and greater retention of performance, which benefits from the implicit bias words we mine to more deeply alleviate potentially harmful associations in the data.
TPR is an overall metric that we propose for joint task accuracy and fairness, which decreases as fairness is promoted with constant task accuracy. We expect the TPR to decrease as little as possible after debiasing. In the three downstream tasks,
Data-Debias achieves the least TPR drop compared to the baseline and even improves the original DistillBERT and RoBERTa on Bias-in-Bios task.
In summary, our proposed debiasing method Data-Debias has outstanding performance in different classification tasks and different PLMs, and it can extremely improve the fairness of PLMs while preserving the accuracy to the greatest extent.

heading: 4.4 Ablation Analysis
text: To answer research question Q3, we study ablation experiments with iteration numbers 0-6, and the results of debiasing in BERT are shown in Figures 3, 4, and 5. The results of bias-amplified Tiny-BERT are provided in the Appendix B.
For debiasing training, we report task metrics and fairness metrics for debiasing BERT using the implicit bias words searched in each iteration in three tasks. BERT-DN on the x-axis represents the BERT debiased using the implicit bias words searched in the Nth iteration. Note that BERT-D0 is applied to implicit bias words searched by the 0th iteration (i.e., original Tiny-BERT). According to the Figures 3(b) to 5(b), we find that the fairness of BERT is extremely improved at BERT-D0 even without amplifying Tiny-BERT’s bias, which verifies the effectiveness of our proposed Data-Debias in debiasing performance. Then, BERT’s bias gradually decreases as the number of iterations increases. This shows that implicit bias words become more precise with amplifying bias training, leading to better debiasing effects. It is important to note that the elimination of bias is not endless, and the fairness peaks at the 4th or 5th iteration and then decreases. We analyze that excessive iterative training destroys the language modeling ability of the biased model, which in turn affects the performance of searching for implicit bias words.
Furthermore, we observe that the overall TPR metric fluctuates without significant decrease within a small range of the original BERT scores.
This suggests that Data-Debias alleviates bias without negatively affecting the performance of the model. It can also be verified based on Figures 3(a) to 5(a). In most iterations, the scores of all task metrics are improved over the original BERT, indicating that moderate debiasing is beneficial to the accuracy of the model.
In summary, it is necessary to iteratively train the bias-amplified model, which plays a role in advancing the debiasing of the PLMs. Note that the degree of debiasing, i.e., the number of iterations, needs to be carefully chosen to avoid damaging the language modeling capability.

heading: 4.5 Generalization Analysis
text: To answer research question Q4, we experiment with zero-shot and few-shot tasks on two largescale PLMs, that is T5-Large (Raffel et al., 2020) and LLaMA-7B (Touvron et al., 2023), to probe generalization of implicit bias words. We test on the validation set of three tasks.
We first find the augmented sample pairs that cause the model biases, then match them with implicit bias words. The top three implicit bias words with the highest bias scores in each biased sample are taken to generate auxiliary prompts, which are sent to the model to guide fair decisions. The template for the auxiliary prompts is as: Reduce the focus on keywords ’bias_word1’, ’bias_word2’, and ’bias_word3’. We expect that simple and direct prompts can guide the model to pay less attention
to implicit bias words, thereby weakening the potentially harmful associations in the data.
We report the results of zero-shot tasks on T5 and few-shot tasks on LLaMA, as shown in Figure 6. Excitingly, the experimental results show that providing PLMs with simple auxiliary prompts can effectively improve the fairness of decision. From the test results of three downstream tasks, we observe that T5 and LLaMA are sensitive to implicit bias words, especially in ToxiGen dataset, with an effective rate of 91.08% and 95.64%, respectively. Furthermore, the TPR scores achieve decent results, indicating that the model maintains predictive abilities while improving fairness. Although these implicit bias words are not re-searched by applying IG to T5 and LLaMA, they still perform exceptionally well in debiasing PLMs. More experimental details can be found in the Appendix C.
Experiments demonstrate that implicit bias words have strong generalization and can be applied to large-scale PLMs under the prompt paradigm without fine-tuning the model. It is extremely exciting for large-scale PLMs, we can improve the fairness of decision by simply adding an auxiliary prompt that contains implicit bias words.

heading: 5 Related Work
text: According to the training strategy, we divide existing debiasing methods into two types: modelcentric debiasing and data-centric debiasing.

heading: 5.1 Model-Centric Debiasing
text: The model-centric debiasing strategies focus on identifying more effective models to improve the fairness of the models. Biases are mitigated by changing the model structure or training strategies (Webster et al., 2020), and introducing
projection-based (Ravfogel et al., 2020) or contrastive learning (Liang et al., 2020; Cheng et al., 2021) to disregard sensitive attributes in representations. Model-centric debiasing strategies have two fatal flaws: 1) task-specific models are highly specialized and difficult to generalize to more tasks; 2) the quality of training data is ignored. There are many defects in the training data from messy sources. Unfiltered utterances on the Internet contain a large amount of discrimination, prejudice, and stereotypes of human society, and even a synthetic corpus is difficult to guarantee that samples are balanced.

heading: 5.2 Data-Centric Debiasing
text: The data-centric debiasing strategies aim to improve data quality and pursue data excellence. Representative methods are a range of extended methods for counterfactual data augmentation (Lu et al., 2020), such as Auto-Debias (Guo et al., 2022), MABEL (He et al., 2022), and CCPA (Li et al., 2023), which augment original training data with explicit bias words to compensate for imbalanced samples. However, these methods only take into account explicit bias words that directly cause bias, and ignore other unfavorable factors that may be mixed in the training data with complex distribution. In this paper, we define implicit bias words that have potentially harmful associations with explicit bias words, considering both direct and indirect causes that affect model fairness.

heading: 6 Conclusion
text: This paper mitigates social biases from a data perspective to improve fairness in PLMs. We propose a data-centric explainable debiasing method, which identifies implicit bias words that have potentially harmful associations with explicit bias words in the training data and reduces PLMs’ focus on implicit bias words to alleviate biases. Implicit bias words are guaranteed to be interpretable by searching with Integrated Gradient and precise by iteratively amplifying bias training. Extensive experiments on three classification tasks demonstrate that Data-Debias can extremely improve the fairness of PLMs while maintaining the predictive abilities. The implicit bias words have strong generalization and can be applied to large-scale PLMs under the prompt paradigm without fine-tuning the model.
Limitations
In this work, we focus on debiasing the gender bias for PLMs. In the future, we will try to mitigate social biases other than gender, such as race and religion. In this paper, our proposed Data-Debias is specific to classification tasks, and in the future we plan to extend it to more downstream tasks, such as natural language inference and generative tasks. In addition, we will further explore the application of implicit bias words to debiasing on more largescale PLMs.
Ethics Statement
This paper has been thoroughly reviewed for ethical considerations and has been found to be in compliance with all relevant ethical guidelines. The paper does not raise any ethical concerns and is a valuable contribution to the field.

heading: Acknowledgments
text: We express gratitude to the anonymous reviewers for their hard work and kind comments. The work was supported in part by the National Natural Science Foundation of China (No.62272191, No.62372211), the International Science and Technology Cooperation Program of Jilin Province (No.20230402076GH, No.20240402067GH), the Science and Technology Development Program of Jilin Province (No.20220201153GX).

heading: B Results of Bias-amplified Tiny-BERT
text: For amplifying bias training, we report the task metrics and fairness metrics of Tiny-BERT with different iteration numbers in three downstream tasks, as shown in Figures 7, 8, and 9. Tiny-BERT-BN on the abscissa represents Tiny-BERT trained with N iterations of amplifying bias. It is observed from Figures 7(b) to 9(b) that Tiny-BERT’s fairness gets
worse as the iteration numbers increases, as expected from the amplifying bias training phase. At the same time, the task accuracy and overall index TPR of the model shows a general downward trend with the increase of the iteration number. Although the performance of Tiny-BERT is increasing before the 4th iteration in the SST2 task, more iterations still cause performance degradation. This suggests that social biases within a model can damage its language modeling ability on downstream tasks.

heading: C Generalization Analysis
text: We experimentally analyze the generalization of implicit bias words in two large-scale pretrained language models (LLMs). We use T5-Large5 to test zero-shot tasks and LLaMA-7B6 to test few-shot tasks implemented by the Huggingface Transformers library. For T5, we simply input the prompts with the task instructions and let the model predict the label of the sample. For LLaMA, we input two demonstration samples to help the model understand the goal of the task. The adopted prompt for each task is shown in Table 6.
We measure task accuracy by the agreement between the predicted labels of the original samples
5https://huggingface.co/t5-large 6https://huggingface.co/huggyllama/llama-7b
and the true labels, and measure fairness by the agreement between the predicted results of the augmented sample pairs, and calculate the TPR score of the fair sample pairs. We report the original results for the undebiased T5 and LLaMA in Table 4. The results show that T5 performs better than LLaMA in terms of accuracy and fairness on all three tasks even though it only has 770M parameters. We analyze that T5, which reconstructs all NLP tasks into a unified text-to-text format for training, is better adapted to classification tasks than LLaMA, which is not fine-tuned on downstream tasks. For the obtained biased sample pairs, we guide LMs debiasing with the implicit bias word generation assisted prompts, and Table 5 reports the specific values corresponding to Figure 6.

heading: Future Work
text: In the future, we will try to mitigate social biases other than gender, such as race and religion. In this paper, our proposed Data-Debias is specific to classification tasks, and in the future we plan to extend it to more downstream tasks, such as natural language inference and generative tasks. In addition, we will further explore the application of implicit bias words to debiasing on more largescale PLMs