C o ^2 PT : Mitigating Bias in Pre-trained Language Models through Counterfactual Contrastive Prompt Tuning (2023)

Abstract: Pre-trained Language Models are widely used in many important real-world applications. However, recent studies show that these models can encode social biases from large pre-training corpora and even amplify biases in downstream applications. To address this challenge, we propose Co ^2 PT, an efficient and effective *debias-while-prompt tuning* method for mitigating biases via counterfactual contrastive prompt tuning on downstream tasks. Our experiments conducted on three extrinsic bias benchmarks demonstrate the effectiveness of Co ^2 PT on bias mitigation during the prompt tuning process and its adaptability to existing upstream debiased language models. These findings indicate the strength of Co ^2 PT and provide promising avenues for further enhancement in bias mitigation on downstream tasks.

## introduction pre-trained language models (plms) are widely used in many real-world applications, demonstrating remarkable performance @xcite . however, it has been demonstrated that plms encode unfair social biases in their parameters based on their pre-training step over large-scale text corpora @xcite . furthermore, these biases -for example, based on gender, race, or religion -can easily propagate to the downstream tasks that use these plms @xcite . for example, "she is a nurse" can have a higher conditional likelihood than "he is a nurse" in the language modeling task, and "nurse" can have higher coreference scores to "she" than "he" in the coreference resolution task @xcite . considering that nlp applications like machine translation systems, resume filtering systems, dialogue systems, and speech recognition @xcite are widely used by millions of users globally, it is crucial to mitigate the social biases present in plms and strive for models that will not propagate discriminatory predictions or offensive outputs towards specific groups before being deployed. much prior effort has focused primarily on debiasing the representations learned during the pretraining process, e.g., through projection @xcite @xcite , further pre-training on unbiased external corpora @xcite @xcite , or finetuning to debias @xcite . the effectiveness of such debiasing efforts is typically measured on intrinsic benchmarks like seat (sentence encoding association test) which computes the association between demographic terms (e.g., woman, man) and stereotype terms (e.g., science, art). an unbiased model should display no difference in the similarity between the representations of these terms @xcite . while these existing approaches help reduce social biases under intrinsic measures, these debiasthen-finetune methods are based on the hypothesis that if an upstream model is unbiased, it will also preserve its fairness effects on downstream tasks during the fine-tuning process. however, recent research investigating the relationship between intrinsic and extrinsic benchmarks (which evaluate fairness in downstream applications) finds these two benchmarks correlate weakly @xcite . furthermore, they observe that models, even after being debiased, tend to re-acquire or even amplify biases (e.g., instance-related biases and labelrelated biases) during the fine-tuning process on downstream tasks @xcite . thus, this mismatch leads to our motivating research question -how can we develop an efficient and effective method to mitigate bias on downstream tasks? to answer the aforementioned question, we propose co 2 pt, a debias-while-prompt tuning approach through counterfactual contrastive prompt pre-trained language model two dogs are running. the woman is playing the piano. the man is playing the piano. tuning. in this method, we first freeze all parameters of the plm and add tunable continuous prompts for every layer. unlike the previous debias-then-finetune methods that require expensive re-training of the original plm and risk knowledge forgetting, this deep prompt tuning framework saves computational and memory resources while preserving the original pre-trained knowledge and language modeling ability @xcite . to ensure that a fair system generates unbiased results regardless of the demographic terms used, we construct counterfactual pairs directly from the training data, eliminating the need for external corpora that heavily depend on their quality for debiasing. specifically, we replace demographic terms associated with either the dominant or minoritized group in the training data with terms representing the opposite group. then, we integrate the ability to mitigate bias into the prompt parameters through a contrastive objective between counterfactual pairs while maintaining the parameters of plms frozen. co 2 pt can be integrated into existing debiased models to help them mitigate biases on downstream tasks and offer flexibility in addressing different kinds of bias. these advantages establish co 2 pt as an efficient and effective method for mitigating bias in downstream tasks. in conclusion, the proposed co 2 pt mitigates bias on downstream tasks through prompt tuning, making the following contributions: ## related work several approaches have been proposed for debiasing pre-trained language models such as projectionbased methods @xcite @xcite , post-hoc text generation techniques @xcite , adversarial methods @xcite , finetuning on biased prompts @xcite , with contrastive objective @xcite or with augmented data @xcite , additional pretraining methods on re-balanced corpus through counterfactual data augmentation @xcite @xcite or with a contrastive objective on gender-balanced entailment pairs @xcite , using dropout regularization @xcite , through parameterefficient methods @xcite @xcite or with a contrastive objective @xcite . while some works do not require access to an external corpus or do not require retraining the entire model, most prior methods primarily focus on mitigating bias within the model's intrinsic characteristics and evaluate the effectiveness of bias mitigation through intrinsic bias benchmarks, e.g., seat @xcite , stereoset @xcite , and crows-pairs @xcite . subsequently, they fine-tune the debiased models on downstream tasks and demonstrate that their debiased models retain the language modeling ability and the performance on downstream tasks or extrinsic bias benchmarks, which evaluate fairness in downstream tasks by testing whether the models exhibit different performances among different populations. nevertheless, recent research shows that these debias-then-finetune methods will re-acquire or even amplify biases during the fine-tuning process on downstream tasks and that intrinsic and extrinsic evaluation bias benchmarks correlate poorly @xcite @xcite . they encourage researchers to focus directly on extrinsic measures of bias of specific applications when addressing bias mitigation @xcite . thus, we focus in this paper on mitigating bias on downstream tasks and evaluate using extrinsic evaluation benchmarks directly. in addition, different from the previous methods requiring further pre-training on the counterfactually augmented sentences from an external corpus, e.g., english wikipedia @xcite @xcite , bookcorpus @xcite , news-commentary v15 @xcite or nli @xcite , our methods achieve time and memory efficiency by eliminating the need for external corpus access or model retraining. ## co 2 pt: debiasing via counterfactual contrastive prompt tuning we propose co 2 pt, a debias-while-prompt tuning parameter-efficient method for mitigating biases on downstream tasks via counterfactual contrastive prompt tuning, presented in figure 1 . concretely, co 2 pt mitigates bias in plms by leveraging counterfactual pairs from training data to produce debiased representations during prompt tuning. ## experimental setup we design experiments to test the effectiveness of our proposed co 2 pt approach toward answering four questions: rq1: will co 2 pt mitigate bias on downstream tasks effectively? rq2: how will the existing intrinsic debiased methods perform on the downstream tasks when they are combined with co 2 pt? rq3: what impact do different modules have on the design of co 2 pt? rq4: how do hyperparameters affect co 2 pt? ## debiasing effectiveness (rq1) we now investigate the effectiveness of co 2 pt in mitigating bias on three extrinsic bias benchmarks. ## impact of design (rq3) we perform an extensive ablation study to show how different components affect co 2 pt in impact of counterfactual module. first, we perform counterfactual data augmentation on the training data containing bias-attribute terms. then we conduct prompt tuning only on these augmented pairs (denoted as pt+cda). pt+cda reduces the bias score from 0.321 in pt to 0.291, showing the effectiveness of the straightforward counterfactual data augmentation approach. however, the improvement is less than co 2 pt, implying the necessity of the contrastive learning module. impact of contrastive module. to investigate the impact of the contrastive module, instead of employing constructed counterfactual sentence pairs as positive pairs for contrastive loss, we use unsupervised contrastive loss by encoding the same input twice and get two embeddings with different dropout masks z, z ′ @xcite . then the contrastive objective becomes: datasets in @xcite as task-agnostic entailment pairs for sts-b task instead of using the taskspecific counterfactual pairs augmented from the training set (denoted as pt+nli+cl). we notice that although pt+nli+cl does not outperform co 2 pt, it shows a strong ability to mitigate bias compared to other baseline methods. thus, when working on a moderate amount of training data, it is better to use counterfactually augmented pairs from the training data. compare co 2 pt with other contrastive objective. for sentence-pair classification tasks, we also explore the contrastive loss that encourages the inter-association of entailment pairs @xcite . for the original input pair (s i1 , s i2 ) and its augmented pair (s ′ i 1 , s ′ i2 ), s i1 and s i2 are treated as positive pairs while s i1 and s ′ i2 and other inbatch s j2 are negatives, and vice versa (denoted as cl p ). when using task-specific counterfactual pairs, pt+cda+cl p decreases the bias score to 0.271. similarly, using task-agnostic counterfactual pairs, pt+nli+cl p also reduces the bias score to 0.271. however, the bias mitigation effect is not as significant as that achieved by co 2 pt, which indicates the effectiveness of the contrastive module in co 2 pt. ## impact of hyperparameters (rq4) finally, we investigate the impact of three hyperparameters: (i) the continuous prompt length; (ii) the temperature τ of contrastive loss l cl ; and (iii) the coefficient α of total learning objective l. ## conclusion and future work we propose co 2 pt, an efficient and effective debiasing method for mitigating bias in downstream tasks. we evaluate its effectiveness on bias mitigation and applicability to existing debiased upstream models, and investigate how the design of each component and the selection of hyperparameters impact both its bias reduction capabilities and downstream task performance.