Causal-Debias: Unifying Debiasing in Pretrained Language Models and Fine-tuning via Causal Invariant Learning (2023)

Abstract: Demographic biases and social stereotypes are common in pretrained language models (PLMs), and a burgeoning body of literature focuses on removing the unwanted stereotypical associations from PLMs. However, when fine-tuning these bias-mitigated PLMs in downstream natural language processing (NLP) applications, such as sentiment classification, the unwanted stereotypical associations resurface or even get amplified. Since pretrain&fine-tune is a major paradigm in NLP applications, separating the debiasing procedure of PLMs from fine-tuning would eventually harm the actual downstream utility. In this paper, we propose a unified debiasing framework Causal-Debias to remove unwanted stereotypical associations in PLMs during fine-tuning. Specifically, CausalDebias mitigates bias from a causal invariant perspective by leveraging the specific downstream task to identify bias-relevant and labelrelevant factors. We propose that bias-relevant factors are non-causal as they should have little impact on downstream tasks, while labelrelevant factors are causal. We perform interventions on non-causal factors in different demographic groups and design an invariant risk minimization loss to mitigate bias while maintaining task performance. Experimental results on three downstream tasks show that our proposed method can remarkably reduce unwanted stereotypical associations after PLMs are finetuned, while simultaneously minimizing the impact on PLMs and downstream applications.

## introduction pretrained language models (plms) have achieved remarkable success in many natural language processing (nlp) tasks. however, plms often encoded undesired social stereotypes and biases, and thus mitigating such biases has become an emerging and important task @xcite . prior bias mitigation methods often focus on removing * corresponding author unwanted stereotypical associations in plms. for example, some works @xcite pretrain a language model using original and counterfactual corpus in order to cancel-out biased associations, some works @xcite focus on debiasing post-hoc sentence representations, and others @xcite design bias-equalizing objectives to fine-tune plm's parameters. figure 1: motivation of causal-debias. t-sne plots of average sentence representations of each word across its sentence templates on sst-2 task @xcite . however, a problem with existing debiasing strategies is that they are separate from downstream nlp tasks. if people take a debiased plm which is supposed to have certain stereotypical associations removed, and then fine-tune it on downstream task, the unwanted associations will re-enter or even get amplified in the fine-tuned language model @xcite . consider gender debiasing using auto-debias @xcite as an example. fig. 1a and 1b shows the bertbased sentence embeddings using t-sne (van der @xcite before and after auto-debias, respectively. the gender bias is clearly less prominent than the original bert, i.e., non-genderspecific concepts (in black) are more equidistant to both genders after auto-debias (fig. 1b ). however, when applying bert model that is debiased by auto-debias to downstream tasks (fig. 1c ), the finetuning procedure almost, if not all, "neutralizes" the effect of plm debiasing. the phenomenon that biases encode in a fine-tuned plm is as worrisome as in a vanilla plm, because the fine-tuned plm is more likely to be deployed in real-world scenarios serving thousands to millions of end users. addressing the aforementioned bias resurgence issues is non-trivial. on the one hand, existing literature on bias mitigation mostly treats plm debiasing as a standalone problem which is separate from downstream tasks. incorporating debiasing objectives into the fine-tuning procedure can be a viable solution but it is still less explored. on the other hand, one may expect to use existing debiasing methods to re-debias the fine-tuned plm. however, due to catastrophic forgetting concern @xcite , a sequential combination of fine-tuning and debiasing may worsen the downstream task performance @xcite . therefore, there is a research gap in unifying debiasing in plms and fine-tuning for building fair and accountable nlp services. in this work, we propose causal-debias, a causal invariant debiasing model to unify the debiasing with downstream fine-tuning. in principle, we analyze the cause and propagation of biases and introduce the structure causal model (scm) @xcite to address the bias mitigation problem by exploiting the inherent causal mechanism in the downstream datasets. specifically, causal-debias first exploits a causal intervention module to distinguish causal and non-causal factors. it then generates counterfactual sentences which have different non-causal factors but the same semantic meanings. the generated counterfactual sentences, along with the original sentences, are fed into an invariant optimization function to ensure a trade-off between the performance of downstream tasks and the effectiveness of debiasing. as illustrated in fig. 1(d ), causal-debias can preserve the plm debiasing effect even after fine-tuning on the downstream dataset. we evaluate performance of causal-debias in mitigating the gender and racial biases in several popular plms, e.g., bert @xcite , albert @xcite , and roberta @xcite , on three glue @xcite tasks (sst-2, cola, and qnli). the results demonstrate causal-debias can significantly mitigate plm biases after downstream fine-tuning while also maintaining the downstream task performance. we hope this work provides empirical evidence that stereotypical associations can reenter language models during the fine-tuning step. moreover, we hope that debiasing with a causal perspective offers a more generalizable and reliable way for building fair and accountable nlp applications. we release the anonymous implementation of causal-debias at https://github.com/ myzeratul/causal-debias . in addition, prior literature also challenges the effectiveness of bias mitigation. @xcite showed that debiasing methods only cover-up biases in word embeddings but do not remove them. @xcite found that existing debiasing methods for plms hurt the language modeling capability of plms and thus the practical utility of debiasing warrants attention. our work follows the spirit of this line of works in that we empirically demonstrate the bias resurgence problem in fine-tuning and suggest to development debiasing techniques with downstream utility. causal mechanism is crystallized in the invariant learning, suggesting that only associations invariant in the training set should be learned @xcite . invariant risk minimization (irm) @xcite is a practical implementation of invariant learning, which is an optimization objective modifying the loss with a regularization term to enforce the invariant representations. recent works have explored structural causal model (scm) @xcite to model auxiliary variables and show promising performance, ranging from domain generalization @xcite in computer vision and intrinsic interpretability @xcite in graph neural networks to factual knowledge @xcite ) and text classification @xcite in nlp. in this work, we introduce scm to plm debiasing to discover the inherent causal relations between data and labels while achieving better debiasing performance. to our knowledge, this is the first work exploiting causality for debiasing plms. ## methodology our goal is to unify debiasing with downstream fine-tuning so that the fine-tuned language model can maintain solid performance with alleviated stereotypical associations. in other words, we want to prevent biases re-entering the language model during the fine-tuning. unlike prior work that considers plm debiasing as a standalone procedure, we mitigate language model biases during finetuning process. to this end, we propose causal-debias, a debiasing framework from a causal view. we first provide the basic formalism and proceed with the details of causal-debias. problem definition: we denote a supervised nlp task with dataset (x, y ), we fine tune a pretrained language model to learn a mapping: m(x) ↦ y . our goal is to mitigate unwanted stereotypical associations in the fine-tuned model m. ## conclusion in this paper, we propose a debiasing framework causal-debias to unify plm bias mitigation with fine-tuning. different from prior literature that treats plm bias mitigation as a standalone task, causal-debias incorporates bias mitigation objective with downstream fine-tuning using causal invariant learning. causal-debias essentially "kills two birds with one stone", because it prevents the unwanted stereotypical associations re-entering the fine-tuned model and also maintains favorable performance in downstream tasks. the experiment shows that fine-tuning existing debiased models will encode or even amplify unwanted bias associations (in gender and race). we also show that causal-debias can effectively reduce bias associations in fine-tuned language model without sacrificing downstream task performance. our paper contributes to the nlp fairness fields by proposing a novel debiasing method from a causal invariant view. more importantly, we highlight the fact that biases can happen at any stage in plm training pipeline, including the final fine-tuning steps. even if unwanted stereotypical associations are removed, or covered up @xcite , in the pretraining stage, the associations will re-surge in the downstream models. hence, prior literature that focuses on pretrained model debiasing may be ineffective if used in a pretrain/fine-tune pipeline. we hope this study can shed light on mitigating biases for building fair and accountable nlp systems.