Causal-Debias: Unifying Debiasing in Pretrained Language Models and Fine-tuning via Causal Invariant Learning
Year: 2023

Abstract:
Demographic biases and social stereotypes are common in pretrained language models (PLMs), and a burgeoning body of literature focuses on removing the unwanted stereotypical associations from PLMs. However, when fine-tuning these bias-mitigated PLMs in downstream natural language processing (NLP) applications, such as sentiment classification, the unwanted stereotypical associations resurface or even get amplified. Since pretrain&fine-tune is a major paradigm in NLP applications, separating the debiasing procedure of PLMs from fine-tuning would eventually harm the actual downstream utility. In this paper, we propose a unified debiasing framework Causal-Debias to remove unwanted stereotypical associations in PLMs during fine-tuning. Specifically, CausalDebias mitigates bias from a causal invariant perspective by leveraging the specific downstream task to identify bias-relevant and labelrelevant factors. We propose that bias-relevant factors are non-causal as they should have little impact on downstream tasks, while labelrelevant factors are causal. We perform interventions on non-causal factors in different demographic groups and design an invariant risk minimization loss to mitigate bias while maintaining task performance. Experimental results on three downstream tasks show that our proposed method can remarkably reduce unwanted stereotypical associations after PLMs are finetuned, while simultaneously minimizing the impact on PLMs and downstream applications.

heading: 1 Introduction
text: Pretrained language models (PLMs) have achieved remarkable success in many natural language processing (NLP) tasks. However, PLMs often encoded undesired social stereotypes and biases, and thus mitigating such biases has become an emerging and important task (Meade et al., 2022). Prior bias mitigation methods often focus on removing
∗Corresponding author
unwanted stereotypical associations in PLMs. For example, some works (Zmigrod et al., 2019) pretrain a language model using original and counterfactual corpus in order to cancel-out biased associations, some works (Liang et al., 2020) focus on debiasing post-hoc sentence representations, and others (Guo et al., 2022; Cheng et al., 2021) design bias-equalizing objectives to fine-tune PLM’s parameters.
However, a problem with existing debiasing strategies is that they are separate from downstream NLP tasks. If people take a debiased PLM which is supposed to have certain stereotypical associations removed, and then fine-tune it on downstream task, the unwanted associations will re-enter or even get amplified in the fine-tuned language model (Goldfarb-Tarrant et al., 2021). Consider gender debiasing using Auto-Debias (Guo et al., 2022) as an example. Fig. 1a and 1b shows the BERTbased sentence embeddings using t-SNE (Van der Maaten and Hinton, 2008) before and after AutoDebias, respectively. The gender bias is clearly less
4227
prominent than the original BERT, i.e., non-genderspecific concepts (in black) are more equidistant to both genders after Auto-Debias (Fig. 1b). However, when applying BERT model that is debiased by Auto-Debias to downstream tasks (Fig. 1c), the finetuning procedure almost, if not all, “neutralizes" the effect of PLM debiasing. The phenomenon that biases encode in a fine-tuned PLM is as worrisome as in a vanilla PLM, because the fine-tuned PLM is more likely to be deployed in real-world scenarios serving thousands to millions of end users.
Addressing the aforementioned bias resurgence issues is non-trivial. On the one hand, existing literature on bias mitigation mostly treats PLM debiasing as a standalone problem which is separate from downstream tasks. Incorporating debiasing objectives into the fine-tuning procedure can be a viable solution but it is still less explored. On the other hand, one may expect to use existing debiasing methods to re-debias the fine-tuned PLM. However, due to catastrophic forgetting concern (Kirkpatrick et al., 2017), a sequential combination of fine-tuning and debiasing may worsen the downstream task performance (Goodfellow et al., 2013). Therefore, there is a research gap in unifying debiasing in PLMs and fine-tuning for building fair and accountable NLP services.
In this work, we propose Causal-Debias, a Causal Invariant Debiasing Model to unify the debiasing with downstream fine-tuning. In principle, we analyze the cause and propagation of biases and introduce the Structure Causal Model (SCM) (Pearl et al., 2000, 2016) to address the bias mitigation problem by exploiting the inherent causal mechanism in the downstream datasets. Specifically, Causal-Debias first exploits a causal intervention module to distinguish causal and non-causal factors. It then generates counterfactual sentences which have different non-causal factors but the same semantic meanings. The generated counterfactual sentences, along with the original sentences, are fed into an invariant optimization function to ensure a trade-off between the performance of downstream tasks and the effectiveness of debiasing. As illustrated in Fig. 1(d), Causal-Debias can preserve the PLM debiasing effect even after fine-tuning on the downstream dataset.
We evaluate performance of Causal-Debias in mitigating the gender and racial biases in several popular PLMs, e.g., BERT (Devlin et al., 2019), ALBERT (Lan et al., 2020), and RoBERTa (Liu
et al., 2019), on three GLUE (Wang et al., 2018) tasks (SST-2, CoLA, and QNLI). The results demonstrate Causal-Debias can significantly mitigate PLM biases after downstream fine-tuning while also maintaining the downstream task performance. We hope this work provides empirical evidence that stereotypical associations can reenter language models during the fine-tuning step. Moreover, we hope that debiasing with a causal perspective offers a more generalizable and reliable way for building fair and accountable NLP applications. We release the anonymous implementation of Causal-Debias at https://github.com/ myZeratul/Causal-Debias.

heading: 2 Related Works
text: PLM Debiasing aims to remove biases, quantified as unwanted stereotypical associations, from pretrained language models. Existing works on PLM debiasing can be categorized into two lines based on whether downstream tasks are involved in the debiasing pipeline. (1) Non-Task-Specific: Counterfactual Data Augmentation (CDA) (Zmigrod et al., 2019) and Dropout (Webster et al., 2020) are two methods where debiasing happens in the pre-training stage (Meade et al., 2022). Auto-Debias (Guo et al., 2022), ContextDebias (Kaneko and Bollegala, 2021) and MABEL (He et al., 2022) remove biases in PLM by designing different bias-equalizing objectives. In this line of work, the parameters in the PLM are changed to meet the fairness criteria such as SEAT (May et al., 2019), and the ultimate goal is to remove bias associations from PLM, so that downstream tasks can benefit from the debiased models. However, we show that it is not this case. When the debiased models are fine-tuned on downstream tasks, the bias associations resurge, perhaps because the biases are not completely removed or maybe just covered up, or because downstream datasets are encoded with stereotypical associations. (2) Task-Specific: Existing works on this line, including Sent-Debias (Liang et al., 2020) and FairFil (Cheng et al., 2021), keep the parameters of PLMs untouched. Instead, they target on the sentence representations, and aim to remove bias associations from representations. Even though the sentence representations, which are the input for downstream tasks, are refined, downstream finetuning can still introduce new biases, as we show in experiments. In summary, our work differs from
existing PLM debiasing in that we aim to unify fine-tuning procedure with debiasing so that the fine-tuned models are free from bias associations.
In addition, prior literature also challenges the effectiveness of bias mitigation. Gonen and Goldberg (2019) showed that debiasing methods only cover-up biases in word embeddings but do not remove them. Meade et al. (2022) found that existing debiasing methods for PLMs hurt the language modeling capability of PLMs and thus the practical utility of debiasing warrants attention. Our work follows the spirit of this line of works in that we empirically demonstrate the bias resurgence problem in fine-tuning and suggest to development debiasing techniques with downstream utility. Causal Mechanism is crystallized in the invariant learning, suggesting that only associations invariant in the training set should be learned (Peters et al., 2016; Muandet et al., 2013). Invariant Risk Minimization (IRM) (Arjovsky et al., 2019) is a practical implementation of invariant learning, which is an optimization objective modifying the loss with a regularization term to enforce the invariant representations. Recent works have explored Structural Causal Model (SCM) (Schölkopf et al., 2012) to model auxiliary variables and show promising performance, ranging from domain generalization (Lv et al., 2022) in computer vision and intrinsic interpretability (Wu et al., 2022) in graph neural networks to factual knowledge (Li et al., 2022) and text classification (Qian et al., 2021) in NLP. In this work, we introduce SCM to PLM debiasing to discover the inherent causal relations between data and labels while achieving better debiasing performance. To our knowledge, this is the first work exploiting causality for debiasing PLMs.

heading: 3 Methodology
text: Our goal is to unify debiasing with downstream fine-tuning so that the fine-tuned language model can maintain solid performance with alleviated stereotypical associations. In other words, we want to prevent biases re-entering the language model during the fine-tuning. Unlike prior work that considers PLM debiasing as a standalone procedure, we mitigate language model biases during finetuning process. To this end, we propose CausalDebias, a debiasing framework from a causal view. We first provide the basic formalism and proceed with the details of Causal-Debias. Problem Definition: We denote a supervised NLP
task with dataset (X,Y ), we fine tune a pretrained language model to learn a mapping: M(X) ↦ Y . Our goal is to mitigate unwanted stereotypical associations in the fine-tuned model M.

heading: 3.1 Biases from a Causal View
text: We use a Structure Causal Model (SCM) to characterize biases in the fine-tuning procedure. As shown in Fig. 2, there are four variables: input raw sentence X , downstream ground-truth label Y , causal factor C and non-causal factor N . Among those, causal and non-causal factor C and N are latent variables. Whether a factor is causal or non-causal depends on the specific downstream tasks. For example, in sentiment classification task, causal factors could be adjective sentiment words such as good or bad, and non-causal factors could be nouns or pronouns. In contrast, in coreference resolution task, pronouns could be causal factors while adjective words could be non-causal. Here we explain the diagram in detail: • C → X ← N . The input raw sentence X is
a mix of two factors that are theoretically nonintersecting: causal factor C and non-causal factor N .
• C → Y . From a causal view, the ground-truth label Y is only determined by causal factor C.
• C ⇠⇢ N . The dashed arrow delegates additional probabilistic dependencies (Pearl et al., 2016, 2000) between causal factor C and noncausal factor N – cf. Appendix A for examples. However, N ⇠⇢ C → Y can create a spurious association between non-causal factor N and ground-truth label Y (denoted as Y ⫫̸ N ), so that C becomes a confounder between N and Y which opens a backdoor path N ← C → Y . Hence, the unwanted stereotypical associations, which we assume are non-causal factors, may re-enter the fine-tuned PLM because of the unwanted associations between label Y and non-causal factors N .
To mitigate the bias propagation and avoid such spurious association that N affects Y through C, we make the feature induction assumption spurred by the Independent Causal Mechanisms (ICM) principle (Peters et al., 2017; Schölkopf et al., 2012), i.e., the intervention on non-causal factor N should be independent of the ground truth Y . According to causal inference (Pearl et al., 2016, 2000), there is a directed link from each of its parent variables P (X) to X , if and only if the causal mechanism X = fX(P (X)) exists for each variable X . That is, there exists a function from causal factor C to label Y without N ’s influence, making the causal association C → Y invariant across different N . More formal assumptions in terms of N ⇠⇢ C are presented in Appendix A. In other words, ICM principle can assure the language model to not pick up the non-causal factors, i.e., unwanted associations, in the fine-tuning procedure.
Generally, only X and Y are observed during the fine-tuning, while neither the causal factor C nor the mapping from C to Y is available. We factorize the language model as a combination of two modules inspired by (Wu et al., 2022), i.e., M = mY ◦ mC , where mC ∶ X → C discovers the causal factor from the observed X , and mY ∶ C → Y outputs the prediction Y . Empirical risk minimization (ERM) is often used as the optimization strategy to train mC and mY (Seo et al., 2022; Zhou et al., 2022):
min mC ,mY R(mY ◦mC(X), Y ), (1) where R(⋅, ⋅) can be any loss function (e.g., crossentropy loss). However, ERM heavily relies on the statistical dependency between the input sentences and labels, ignoring the critical condition Y ⫫ N ∣ C to guarantee the invariant causal association C → Y across different N , leading to the resurgence of undesired spurious associations.
Recent causal learning literature has proposed to use invariant risk minimization (IRM) objective to replace ERM (Arjovsky et al., 2019; Chang et al., 2020). The causal invariant learning encourages causal factor C to seek the patterns that are stable across different “enviornments”, while abandoning the unstable non-causal patterns. We follow this line of literature, and propose to use causal invariant learning objective to mitigate fine-tuning biases:
min mC ,mY R(mY ◦mC(X), Y ), s.t. Y ⫫ N ∣ C, (2)
where N = X \ C is the non-causal factor. However, IRM often leverages multiple “enviornments” to facilitate causal learning. For example, Peyrard et al. (2021) use prior knowledge to partition the training set to form different environments. In NLP tasks such as sentiment classification, how to construct multiple “enviornments” in the context of language model fine-tuning is less studied. Next, we propose a causal intervention method to construct “enviornments” for causal learning.

heading: 3.2 Causal-Debias
text: Causal Intervention. The high-level idea for causal intervention is to create interventional distributions with respect to different demographic groups. Our interventional distribution is obtained by augmenting and expanding the original data distribution. First, let Wa and Wt denote attribute words and target words, respectively. In the case of gender bias, for instance, target words consist of gender-neutral words (e.g., nurse, engineer, professor), and attribute words are composed of the feminine (e.g., she, woman, mother) and masculine words (e.g. he, man, father) (Liang et al., 2020). Then we can obtain an augmented datasets Xd:
Xd = Xo ∪Xc, (3)
where Xo denotes the original sentences from the downstream dataset containing any Wa or Wt, and Xc represents the counterfactual sentences via performing attribute word counterfactual augmentation on Xo. For counterfactual augmentation, we create counterfactual sentences by replacing the attribute word to its corresponding pair (e.g., he>she). However, the augmented dataset Xd still has limitations as it is not sufficient to cover the diversity of demographic groups, which may cause debiasing performance degradation problems (cf. Section 4.3 for empirical study). To create sufficiently complex interventional distributions and obtain the most different demographic groups, we conduct the interventions by doing semantic matching between Xd with external corpora E, expanding Xo and Xc to Xõ and Xc̃ (see Table 1 for an example), respectively, as
Xd̃ = Xõ ∪Xc̃ = Topk(sim(Xd, E)) ∪Xd, (4) where sim(⋅) denotes the cosine similarity for semantic matching, and Topk(⋅) selects the top-k semantic similar sentences. After obtaining the
intervened sentences, we reconstruct the interventional distribution by combining Xd̃ and the rest bias-unrelated downstream dataset, both of which are applied for causal invariant learning. Table 1 shows an example of original sentence, and its corresponding counterfactual and expansion sentences.
Causal Invariant Learning. Once we obtain the intervened dataset containing original and interventional data distributions, we proceed with causal invariant learning. Specifically, we do n-intervention do(N = n), which removes all links from their parents P (N) to the variable N while fixing N to the number of demographic n (e.g. n = 2 in the case of gender), to identify C whose relationship with Y is stable across different distributions: minLinvariant = En(R) + Varn(R), (5) where R = R(M(X), Y ∣ do(N = n)) computes the risk under the n-interventional distribution; En(⋅) denotes the risks of different ninterventional distributions; Var(⋅) denotes the variance of risks over n-interventional distributions.
To calculate Linvariant, the PLM is required to predict the same results on the sentences Xõ and Xc̃, which have equivalent semantics but different attribute words according to the IRM theory (Arjovsky et al., 2019). Thus in optimization, we have the interventional risk derived from Equation (5):
R(M(Xd̃), Y ∣ do(N = ñ)) = EC=mC (x),N=ñl(ỹ, y), (6)
where x ∈ Xd̃ is a sentence instance with its prediction ỹ under the intervention do(N = ñ), and y ∈ Y is ground-truth label, and l(⋅) denotes interventional loss function on a single sentence. Here we choose Wasserstein distance (Ramdas et al., 2017) as a loss function due to its ability to measure the agreement between the prediction of original and post-intervention sentences. The Wasserstein distance between ỹ and y is formalized as below:
DWasser(ỹ, y) = inf γ(ỹ,y)∈∏ E(ỹ,y)∼γ∣∣ỹ − y∣∣, (7)
PLM Fine-tuning. The above procedure leverages augmented datasets for causal invariant learning, and we can incorporate the invariant loss with specific downstream tasks to fine-tune a language model. This way, we can balance the trade-off between debiasing performance and downstream task performance (Meade et al., 2022). The overall objective of Causal-Debias is:
min τ
Lprediction + τLinvariant, (8)
where τ is the trade-off coefficiency, and Lprediction is the loss function of a specific downstream task, such as cross-entropy loss for classification and mean squared error loss for regression.

heading: 4 Experiments

heading: 4.1 Experimental Settings
text: Debiasing Benchmarks. We compare CausalDebias with the following benchmarks. Non-TaskSpecific methods including: CDA, Dropout (Webster et al., 2020), Context-Debias (Kaneko and Bollegala, 2021), Auto-Debias (Guo et al., 2022), and MABEL (He et al., 2022), and two TaskSpecific methods including Sent-Debias (Liang et al., 2020) and FairFil (Cheng et al., 2021). In the Non-Task-Specific benchmarks, the debiasing stage is independent of fine-tuning downstream tasks.
Causal-Debias belongs to Task-Specific methods as downstream fine-tuning tasks are involved. Pretrained Language Models. We use three representative PLMs as the backbone: BERT (Devlin et al., 2019), ALBERT (Lan et al., 2020), and RoBERTa (Liu et al., 2019). Following (Guo et al., 2022), we implement them using the Huggingface Transformers library (Wolf et al., 2020). Bias Word Lists. Following previous studies, we use human-being-created stereotype/attribute word lists to investigate and mitigate biases in PLMs. They are based on the research of the social science literature and other disciplines, which can reflect cultural or offensive biases. In particular, we consider the gender and race word lists used in (Kaneko and Bollegala, 2021) and (Manzini et al., 2019), respectively – cf. Appendix B for details. External Corpora. For fair comparison, we exploit the same external corpora used in baselines, which are composed of 183,060 sentences from following sources: WikiText-2 (Merity et al., 2017), Standford Sentimente Treebank (Socher et al., 2013), Reddit, MELD (Poria et al., 2019) and POM (Park et al., 2014) – cf. Appendix C for details. Evaluating Metrics: We evaluate biases in PLM embeddings with SEAT (May et al., 2019) and CrowS-Pair (Nangia et al., 2020). An ideally unbiased model should exhibit no difference in relative similarity. Following Guo et al. (2022); Liang et al. (2020); Kaneko and Bollegala (2021), we apply SEAT 6, 6b, 7, 7b, 8, and 8b tests to measure the gender bias, and use SEAT 3, 3b, 4, 5, 5b tests for racial bias evaluation. We report the effect size in the SEAT evaluation – the closer to 0, the lower bias a model has. More details about SEAT tests are presented in Appendix D. We also use Crowdsourced Stereotype Pairs (CrowS-Pair) (Nangia et al., 2020) as another metric to evaluate gender bias. CrowS-Pair is a dataset containing 1,508 examples covering different types of biases, where each example is a stereotype/anti-stereotype sentence pair with minimal semantics. The CrowSPair score closer to 50% is less stereotypical, indicating that the model assigns an equal probability to male and female sentences. Other Details. As the related studies (Cheng et al., 2021; Liang et al., 2020), we conduct experiments on three downstream tasks, including a sentiment classification task SST-2, a grammatical acceptability judgment task CoLA, and a question-answering task QNLI. We follow the same PLMs as benchmarks: 1) BERT-base-uncased, ALBERT-large-v2, and RoBERTa-base in gender case; and 2) BERTbase-uncased and ALBERT-base-v2 in racial case. We trained Causal-Debias in 5 epochs with learning rate 2 × e−5. The reported results are the average of 5 runs for all downstream tasks.

heading: 4.2 Results on Mitigating Gender Bias
text: SEAT Tests and Downstream Tasks Evaluation. Table 2 summarizes the debiasing results of models before and after fine-tuning on three downstream tasks , as well as the accuracy (Acc.) evaluations on downstream applications, from which we have the following Observations. (O1): Causal-Debias is more effective in mitigating gender bias than previous benchmarks, as it achieves the lowest average SEAT scores in all three downstream tasks. For example, CausalDebias surpasses Task-Specific SOTA (FairFil) by 0.07, 0.01, and 0.07, and Non-Task-Specific SOTA (Auto-Debias) by 0.27, 0.21, and 0.09 on BERT, respectively. The excellent debiasing results can attribute to the following two characteristics of Causal-Debias: 1) combining debiasing PLMs and fine-tuning to avoid new biases, and 2) the causal intervention-based invariant learning that alleviates the impact of non-causal factors. (O2): From the SEAT results after fine-tuning downstream tasks, the PLMs become more biased for almost all non-task-specific debiasing models. In particular, the latest debiasing methods AutoDebias is greatly limited to the bias resurgence issue in all three tasks, although it achieved good performance on debiasing PLMs. These results verified not only the existence of bias resurgence but also the motivation of this study to attenuate the intrinsic bias of PLMs and fine-tuning bias jointly. Interestingly, the original PLMs do not suffer from this problem, as the bias can be mitigated after finetuning the downstream tasks in most cases. This result suggests that fine-tuning itself is an effective way of debiasing PLMs; combining with debiased models, however, will introduce extra bias. To deal with this dilemma, debiasing models should consider downstream tasks as a unity. Two task-specific models, i.e., Sent-Debias and FairFil, cannot effectively alleviate the application bias even using downstream datasets for debiasing PLMs, because their focus is still intrinsic bias – by contrast, Causal-Debias unifies the two debiasing procedure and provides an systematic solution to
attenuates both biases simultaneously. (O3): Previous benchmarks may significantly degrade the performance on downstream tasks after debiasing the PLMs. This is a natural result of existing debiasing models as they need to change the representations to mitigate bias and therefore inevitably decrease accuracy after debiasing (Liang et al., 2020). In addition to its superior debiasing effect, Causal-Debias achieves better performance on downstream tasks, e.g., exceeding all benchmarks on BERT in terms of performance. This result demonstrates the ability of Causal-Debias to minimize the disagreements among different demographic groups with identical semantic information, which is attained by intervening in downstream datasets to mitigate the bias recurrence.
CrowS-Pairs. Table 3 reports CrowS-Pairs scores
before and after fine-tuning on SST-2. Note that the deviation from 50 is usually used to measure the debiasing effect. Obviously, Causal-Debias achieves the lowest deviation among the fine-tuned debiasing models, which proves its best debiasing performance. In addition, the deviation values of previous methods increase after fine-tuning, which further confirms the existence of application bias ignored by existing methods. Again, BERT itself can alleviate the bias after fine-tuning, which is consistent with our observation O2 from Table 2.

heading: 4.3 Ablation Study
text: To quantify the effect of the devised causal intervention and invariant risk learning, we build three variants of Causal-Debias: • (V1) w/o E removes the intervention from exter-
nal corpora in Eq. (4). • (V2) w/o E&Linvariant removes both interven-
tions of external corpora in Eq. (4) and the invariant loss in Eq. (8) but maintains counterfactual augmentation of data in downstream tasks. • (V3) w/o Linvariant only remains the prediction loss Lprediction in Eq. (8). Fig. 4 compares the variants with full CausalDebias on SST-2 task. The variant V1 performs worst on debiasing, indicating that external corpora contribute the most to model debiasing. However, external corpora also deteriorate the task perfor-
mance when comparing the accuracy of V2 and V3, because the augmented sentences introduce extra semantics and obfuscate the PLMs. The adverse impact can be minimized by learning the invariant representations by the proposed causal interventions on downstream tasks via Linvariant, which can be justified by the great discrepancy between Causal-Debias and V3 in terms of SST-2 accuracy.

heading: 4.4 Results on Mitigating Racial Bias
text: Racial debiasing refers to examining the association difference between EuropeanAmerican/African American names/terms and the stereotype words (pleasant vs. unpleasant) (Caliskan et al., 2017). Unlike gender debiasing, few prior studies investigated the racial debiasing problem, due to the difficulty of mitigating racial bias (Meade et al., 2022). A critical challenge is the potential word ambiguity (e.g., white, black) in various contexts (Guo et al., 2022). Table 4 reports the performance of Causal-Debias and Auto-Debias – the state-of-the-art racial debiasing model. Causal-Debias substantially decreases the racial biases on PLMs after fine-tuning, while obtaining comparable downstream performance. Auto-Debias, in contrast, still suffers from bias recurrence issue. Compared to Auto-Debias, Causal-Debias is more effective as it exploits
downstream datasets for debiasing, which allows us to alleviate the influence of ambiguous words. Besides, the causal invariant learning in CausalDebias encourages the model to learn consistent representations and clear meanings of ambiguous words so as to avoid bias-related associations.

heading: 5 Conclusion
text: In this paper, we propose a debiasing framework Causal-Debias to unify PLM bias mitigation with fine-tuning. Different from prior literature that treats PLM bias mitigation as a standalone task, Causal-Debias incorporates bias mitigation objective with downstream fine-tuning using causal invariant learning. Causal-Debias essentially “kills two birds with one stone”, because it prevents the unwanted stereotypical associations re-entering the fine-tuned model and also maintains favorable performance in downstream tasks. The experiment shows that fine-tuning existing debiased models will encode or even amplify unwanted bias associations (in gender and race). We also show that Causal-Debias can effectively reduce bias associations in fine-tuned language model without sacrificing downstream task performance. Our paper contributes to the NLP fairness fields by proposing a novel debiasing method from a causal invariant view. More importantly, we highlight the fact that biases can happen at any stage in PLM training pipeline, including the final fine-tuning steps. Even if unwanted stereotypical associations are removed, or covered up (Kirkpatrick et al., 2017), in the pretraining stage, the associations will re-surge in the downstream models. Hence, prior literature that focuses on pretrained model debiasing may be ineffective if used in a pretrain/fine-tune pipeline. We hope this study can shed light on mitigating biases for building fair and accountable NLP systems.

heading: Acknowledgements
text: This work was supported by the Natural Science Foundation of Sichuan Province, China, under Grant 2022NSFSC0505, and National Natural Science Foundation of China under Grants 62176043 and 62072077.
Limitations
We now discuss limitations of Causal-Debias. In consideration of the fairness, we follow the prior bias mitigation work (Guo et al., 2022; Cheng et al., 2021; He et al., 2022) and use human-collected lists of gender and racial pairs for counterfactual data augmentation and intervened distribution generation. It is obvious that the bias word lists are inadequate to cover all the bias-related demographic groups, while we believe the general list is exhaustive. We consider there is a possible model improvement that leverages the perturbation augmentation on bias-related sentences along multiple demographic axes (Qian et al., 2022). Another possible improvement would be to generate bias words by using prompts to probe the biases that may lead to a bad effect.
Moreover, we also considered the use of external corpora. The external corpora have been significantly investigated in prior works (Liang et al., 2020; Cheng et al., 2021) and are utilized as an intervention corpora. Recently, He et al. (2022) used two natural language inference data (SNLI and MNLI with gender terms) to produce generalpurpose debiased representations. There are several other corpora including News-commentary-v1 (Kaneko and Bollegala, 2021), Wikipedia (Zmigrod et al., 2019; Webster et al., 2020), and Wikitext-2 (Guo et al., 2022). A possible future direction of debiasing is how to mitigate the biases without heavily relying on any corpora and just using internal knowledge.
Moreover, in the paper we primarily focus on studying gender and racial bias mitigation. It is also worth exploring intersectional biases mitigation (Lalor et al., 2022) and domain-specific bias mitigation (Chuang and Yang, 2022; Abbasi et al., 2021).
We would also like to note that although CausalDebias shows a satisfactory performance on SEAT tests and Crows-Pairs, these results should not be interpreted as a complete bias mitigation. Interestingly, He et al. (2022) expressed the same opinion. The main metrics (like CrowS-Pairs) are mainly
against North American social biases and only reflect positive predictive power. They detect the presence of the biases but not their absence (Meade et al., 2022). He et al. (2022) did not use the SEAT tests and evaluated their model on various metrics. From the perspective of different usage scenarios, we need a more general and reliable debias metric for comparison between different models. The lack of universality and agreement in existing evaluation frameworks is a fundamental challenge in this field.
Ethics Statement
Regarding ethical concerns, we would like to note that our contributions are mainly about methodologies. The datasets and evaluation metrics in our work are also widely used in prior works. One ethical concern is the binarization of the genders and races, which is an over-simplification and is not proper to practical situations. Binarization is the common problem among most debiasing methods and we totally agree and support the development of more inclusive methodological tools, datasets, and evaluation methods.
Under our framework, we consider gender or race isolatedly and neglect the particular intersectional biases. It is apparent that the pretrained language model cannot be applied or operated in an ideal environment, and should be able to handle complex combinations of biases simultaneously.
Another ethical consideration is that CausalDebias is entirely based on a English system. Such an assumption may not be a problem now but sooner or later it will be. The debias studies have to be situated on the high-resource languages while considering not only high-resource language systems but how to debias on the low-resource languages.
For instance, some languages such as Spanish, German, Chinese, or Japanese contain various words to describe masculine or feminine forms. The detection and removal of biases are greatly complicated by the need to consider both linguistic and social gender.
For the reasons above, practitioners should be very cautious when applying our framework to real-world use cases. In its current state, CausalDebias should be seen not as a panacea for addressing biases in NLP, but rather as another initial effort to illuminate and undercut a critical, elusive, multifaceted problem.

heading: B Bias Words List
text: We used the gender attribute words and target words lists proposed in (Kaneko and Bollegala, 2021), which is widely used in debiasing studies (Guo et al., 2022; Liang et al., 2020). In addition, we used the race attribute words and attribute words provided in (Manzini et al., 2019).

heading: C External Corpora
text: Table 5 summarizes the five external datasets along with examples of the numerous templates occurring across various individuals, settings and in both written and spoken text. The external corpora E used to expand the downstream dataset is created from (Liang et al., 2020), including: 1) WikiText-2 (Merity et al., 2017) – a dataset of formally written Wikipedia articles, where only the first 10% of WikiText-2 is used and verified sufficiently to
capture formally written text); 2) Stanford Sentiment Treebank (Socher et al., 2013) is a collection of 10000 polarized written movie reviews; 3) Reddit data collected from discussion forums relating to politics, electronics, and relationships; 4) MELD (Poria et al., 2019) – a large-scale multimodal multi-party emotional dialog dataset collected from the TV-series Friends; and 5) POM (Park et al., 2014) – a dataset of spoken review videos collected across 1,000 individuals spanning multiple topics. These datasets have also been used in recent research in language understanding (Merity et al., 2017; Liu et al., 2019) and multimodal human language (Liang et al., 2019).

heading: D SEAT Details
text: The WEAT metric measures the bias by comparing two sets of attribute words Wa (i.e., M and F ) and two sets of target words Wt (i.e., A and B). In the case of gender, M denotes masculine words like “he”, and F denotes feminine words like “she”. Meanwhile, A and B are gender-neutral words (e.g., career or adjectives) whose embeddings should be equivalent between M and F . Formally, the bias degree of each word w is defined as:
s(w,A,B) = 1∣A∣ ∑ a∈A cos(w, a) − 1∣B∣ ∑ b∈B cos(w, b), (11)
where cos(⋅, ⋅) denotes the cosine similarity. Based on Equation (11), the WEAT effect size is:
dWEAT = µ({s(m,A,B)}m∈M) − µ({s(f,A,B)}f∈F )
σ({s(t, A,B)}t∈A∪B)) , (12)
where µ and σ denote the mean and standard deviation, respectively. The SEAT metric generalizes the WEAT via replacing the word embeddings with a few simple sentence templates (e.g., “This is the <word>”). We can conclude from Equation (12) that the absolute SEAT effect size closer to 0 means lower biases. We list more details about the SEAT tests that are used in our experiments in Table 6, which are adapted from (Caliskan et al., 2017).
ACL 2023 Responsible NLP Checklist

heading: A For every submission:
text: 3 A1. Did you describe the limitations of your work?
7
3 A2. Did you discuss any potential risks of your work? 7,8
3 A3. Do the abstract and introduction summarize the paper’s main claims? 1
7 A4. Have you used AI writing assistants when working on this paper? Left blank.
B 3 Did you use or create scientific artifacts? 4
3 B1. Did you cite the creators of artifacts you used? 4
3 B2. Did you discuss the license or terms for use and / or distribution of any artifacts? 4
3 B3. Did you discuss if your use of existing artifact(s) was consistent with their intended use, provided that it was specified? For the artifacts you create, do you specify intended use and whether that is compatible with the original access conditions (in particular, derivatives of data accessed for research purposes should not be used outside of research contexts)? 4
3 B4. Did you discuss the steps taken to check whether the data that was collected / used contains any information that names or uniquely identifies individual people or offensive content, and the steps taken to protect / anonymize it? 4
3 B5. Did you provide documentation of the artifacts, e.g., coverage of domains, languages, and linguistic phenomena, demographic groups represented, etc.? 4
3 B6. Did you report relevant statistics like the number of examples, details of train / test / dev splits, etc. for the data that you used / created? Even for commonly-used benchmark datasets, include the number of examples in train / validation / test splits, as these provide necessary context for a reader to understand experimental results. For example, small differences in accuracy on large test sets may be significant, while on small test sets they may not be. 4
C 3 Did you run computational experiments? 4
3 C1. Did you report the number of parameters in the models used, the total computational budget (e.g., GPU hours), and computing infrastructure used? 4
The Responsible NLP Checklist used at ACL 2023 is adopted from NAACL 2022, with the addition of a question on AI writing assistance.
3 C2. Did you discuss the experimental setup, including hyperparameter search and best-found hyperparameter values? 4
3 C3. Did you report descriptive statistics about your results (e.g., error bars around results, summary statistics from sets of experiments), and is it transparent whether you are reporting the max, mean, etc. or just a single run? 4
3 C4. If you used existing packages (e.g., for preprocessing, for normalization, or for evaluation), did you report the implementation, model, and parameter settings used (e.g., NLTK, Spacy, ROUGE, etc.)? 4
D 7 Did you use human annotators (e.g., crowdworkers) or research with human participants? Left blank.
D1. Did you report the full text of instructions given to participants, including e.g., screenshots, disclaimers of any risks to participants or annotators, etc.? No response.
D2. Did you report information about how you recruited (e.g., crowdsourcing platform, students) and paid participants, and discuss if such payment is adequate given the participants’ demographic (e.g., country of residence)? No response.
D3. Did you discuss whether and how consent was obtained from people whose data you’re using/curating? For example, if you collected data via crowdsourcing, did your instructions to crowdworkers explain how the data would be used? No response.
D4. Was the data collection protocol approved (or determined exempt) by an ethics review board? No response.
D5. Did you report the basic demographic and geographic characteristics of the annotator population that is the source of the data? No response.

heading: Future Work
text: A possible future direction of debiasing is how to mitigate the biases without heavily relying on any corpora and just using internal knowledge. Moreover, in the paper we primarily focus on studying gender and racial bias mitigation. It is also worth exploring intersectional biases mitigation (Lalor et al. , 2022) and domain-specific bias mitigation (Chuang and Yang, 2022; Abbasi et al. , 2021). We would also like to note that although CausalDebias shows a satisfactory performance on SEAT tests and Crows-Pairs, these results should not be interpreted as a complete bias mitigation. Interestingly, He et al. (2022) expressed the same opinion. The main metrics (like CrowS-Pairs) are mainly
against North American social biases and only reflect positive predictive power. They detect the presence of the biases but not their absence (Meade et al. , 2022). He et al. (2022) did not use the SEAT tests and evaluated their model on various metrics. From the perspective of different usage scenarios, we need a more general and reliable debias metric for comparison between different models. The lack of universality and agreement in existing evaluation frameworks is a fundamental challenge in this field