Data-Centric Explainable Debiasing for Improving Fairness in Pre-trained Language Models (2024)

Abstract: Human-like social bias of pre-trained language models (PLMs) on downstream tasks have attracted increasing attention. The potential flaws in the training data are the main factor that causes unfairness in PLMs. Existing data-centric debiasing strategies mainly leverage explicit bias words (defined as sensitive attribute words specific to demographic groups) for counterfactual data augmentation to balance the training data. However, they lack consideration of implicit bias words potentially associated with explicit bias words in complex distribution data, which indirectly harms the fairness of PLMs. To this end, we propose a **Data**-Centric **Debias**ing method (named Data-Debias), which uses an explainability method to search for implicit bias words to assist in debiasing PLMs. Specifically, we compute the feature attributions of all tokens using the Integrated Gradients method, and then treat the tokens that have a large impact on the model’s decision as implicit bias words. To make the search results more precise, we iteratively train a biased model to amplify the bias with each iteration. Finally, we use the implicit bias words searched in the last iteration to assist in debiasing PLMs. Extensive experimental results on multiple PLMs debiasing on three different classification tasks demonstrate that Data-Debias achieves state-of-the-art debiasing performance and strong generalization while maintaining predictive abilities.

## introduction in recent years, pre-trained language models (plms), such as the bert series @xcite @xcite and gpt series @xcite @xcite have seen vigorous development in natural language processing. by training on massive text datasets, plms can acquire rich linguistic knowledge. however, these training datasets contain * corresponding author human-like social biases and stereotypes @xcite . plms can learn and amplify biases against certain demographic groups, leading to unfair decisions that harm vulnerable groups therefore, mitigating social biases in plms is a critical issue and necessary to improve the fairness of natural language systems. current debiasing methods usually take the model-centric strategies @xcite , seeking to improve fairness by modifying model architectures or adding regularization during training without considering the defects of the training data @xcite . however, they ignore critical issues of training data, including real-world stereotypes, discrimination against vulnerable groups, and imbalanced samples across demographic groups. data quality issues fundamentally enable language models to learn biased semantics. furthermore, some methods are difficult to generalize to large-scale plms due to the limitations of the fine-tuning model. in contrast to the model-centric strategies, the data-centric debiasing strategies focus on addressing defects in training data to improve data quality @xcite . the most general approach utilizes counterfactual data augmentation @xcite to alleviate the imbalance of samples from different demographic groups. these debiasing methods generate synonymous sentences based on priori sensitive attribute words specific to different demographic groups (e.g., male/female, white/black). although they consider sensitive attribute words that directly cause model biases, they ignore potentially harmful associations present in the training data that indirectly cause unfairness. we refer to sensitive attribute words specific to demographic groups as explicit bias words, and to tokens in the training data that have potentially harmful associations with explicit bias words as implicit bias words (illustrative examples are given in figure 2 ). we formulate the following hypoth-esis: under the indirect effect of implicit bias words, plms capture the spurious associations between demographic groups and certain class labels to make unfair and biased decisions. based on the above assumptions, in this work, we propose a data-centric debiasing method (named data-debias), which uses an explainability method to search for implicit bias words to assist debiasing plms. specifically, we compute the feature attribution scores for all tokens in the training data using integrated gradients @xcite which is a post-hoc explainability method. implicit bias words are selected as tokens with large differences in feature attribution scores across demographic groups, and then used to assist debiasing. to more precisely target implicit bias words that are strongly associated with explicit bias words, we iteratively train a bias-amplified model. within each iteration, we train the model with the implicit bias words to amplify the model's bias, and then re-search and update the implicit bias words with the biased model. after the iteration, the last updated more biased implicit bias words are used to assist debiasing to improve the fairness of plms. our main contributions are summarized as follows: 1) we propose a data-centric debiasing framework, which mines implicit bias words that have potentially harmful associations with explicit bias words, to improve the quality of the training data more comprehensively for stronger debiasing effects. 2) the obtained implicit bias words are interpretable and precise, ensured by searching via explainability methods and iteratively training the bias-amplified model. 3) experiments on classification tasks on multiple plms demonstrate that data-debias outperforms state-of-the-art baseline models in debiasing while preserving predictive abilities. 4) debiasing experiments on large-scale language models under the prompting paradigm verify the generalization of implicit bias words. ## methodology in this section, we introduce the proposed data-debias debiasing framework, as shown in figure 1 . data-debias has three stages: 1) searching for implicit bias words via an explainability method. 2) iterative training of a bias-amplified model using implicit bias words. 3) debiasing training using the final implicit bias words. integrated gradient implicit bias words calculate bias degree 𝐍 𝐭𝐡 iteration of amplifying bias ℒ = ℒ +𝜆 × ℒ update gradient descent biased-model search for implicit bias words integrated gradient implicit bias words calculate bias degree 𝟐 𝐧𝐝 iteration of amplifying bias ℒ = ℒ +𝜆 × ℒ update gradient descent biased-model search for implicit bias words integrated gradient implicit bias words calculate bias degree 𝟏 𝐬𝐭 iteration of amplifying bias ℒ = ℒ +𝜆 × ℒ update gradient descent biased-model search for implicit bias words implicit bias words calculate bias degree ℒ = ℒ +𝛾 × ℒ debiased-model gradient descent ## evaluation methods we evaluate the performance of data-debias on the task metrics and the fairness metrics. task metrics. four general accuracy metrics evaluate the predictive abilities of plms on all classification tasks, which are accuracy, f1 score, precision, and recall. they are all calculated based on the predicted and true labels of the samples in the test set, and a higher value indicates the better predictive abilities of plm. fairness metrics. in the existing debiasing research, the external metrics to measure fairness are specific to a certain task and are not applicable to other tasks. to this end, we propose two fairness metrics common to all classification tasks, which are calculated based on the prediction results of each augmented sample set. we propose a second metric to jointly evaluate the fairness and predictive abilities of plms, which is the t ruep ositiveratio defined as in addition, we also choose f1 score, precision, and recall as fairness indicators, whose inputs are all the prediction results of the male augmented sample y m and the prediction results of the female augmented sample y f . the following research questions. q1 : how explainable are the detected implicit bias words? q2 : compared to baseline methods, how effective is applying data-debias to debias plms in downstream tasks? q3 : what is the role of iteratively training the bias-amplified model? q4 : whether implicit bias words generalize to debiasing plms in the prompting paradigm? 4.1 experimental setting downstream tasks. for downstream debiasing, we chose three classification tasks corresponding to three datasets with different data sizes. each dataset is matched and augmented based on a given list of explicit bias word pairs, and the train-test split is 50/50. stanford sentiment treebank (sst2) foot_2 @xcite is a manually annotated binary sentiment analysis dataset to judge single-sentence movie reviews, which matches a total of 4,133 original samples containing gender attributes. toxigen foot_3 @xcite is an implicit toxic text classification dataset automatically generated based on prompts with revealed information, which matches 38,000 original samples containing explicit bias words. the prompt label for each sample is taken as the true label. bias in bios foot_4 (de-arteaga et al., 2019) is a third person biography dataset with 28 occupation categories and 250,000 original samples scraped. the task objective is to predict the category of occupation for each biography with the occupation information removed. comparing baselines. we choose three modelcentric debiasing methods inlp @xcite , sent-debias @xcite , as well as fairfil @xcite , and three data-centric debiasing methods cda @xcite , auto-debias @xcite , as well as mabel @xcite as baselines. they are described as follows: ## related work according to the training strategy, we divide existing debiasing methods into two types: modelcentric debiasing and data-centric debiasing. ## conclusion this paper mitigates social biases from a data perspective to improve fairness in plms. we propose a data-centric explainable debiasing method, which identifies implicit bias words that have potentially harmful associations with explicit bias words in the training data and reduces plms' focus on implicit bias words to alleviate biases. implicit bias words are guaranteed to be interpretable by searching with integrated gradient and precise by iteratively amplifying bias training. extensive experiments on three classification tasks demonstrate that data-debias can extremely improve the fairness of plms while maintaining the predictive abilities. the implicit bias words have strong generalization and can be applied to large-scale plms under the prompt paradigm without fine-tuning the model.