Co$^2$PT: Mitigating Bias in Pre-trained Language Models through Counterfactual Contrastive Prompt Tuning
Authors: Xiangjue Dong
Ziwei Zhu
Zhuoer Wang
Maria Teleki
James Caverlee
Venue: EMNLP 2023 Findings (2023)
Keywords: bias
contrastive learning
prompt tuning
TL;DR: We propose Co$^2$PT, an efficient and effective method for mitigating biases via counterfactual contrastive prompt tuning on downstream tasks.

Abstract: Pre-trained Language Models are widely used in many important real-world applications. However, recent studies show that these models can encode social biases from large pre-training corpora and even amplify biases in downstream applications. To address this challenge, we propose Co$^2$PT, an efficient and effective *debias-while-prompt tuning* method for mitigating biases via counterfactual contrastive prompt tuning on downstream tasks. Our experiments conducted on three extrinsic bias benchmarks demonstrate the effectiveness of Co$^2$PT on bias mitigation during the prompt tuning process and its adaptability to existing upstream debiased language models. These findings indicate the strength of Co$^2$PT and provide promising avenues for further enhancement in bias mitigation on downstream tasks.

Co²PT: Mitigating Bias in Pre-trained Language Models through Counterfactual Contrastive Prompt Tuning
Xiangjue Dong Ziwei Zhu Zhuoer Wang Maria Teleki James Caverlee
Texas A&M University 2 George Mason University
(xj.dong, wang, mariateleki, saverlee)@tamu.cem zzhu200gmu.edin
Abstract
will not propagate discriminatory predictions offensive outputs towards specific groups before being deployed.
Pre-trained Language Models are widely used many important real-world applications. However, recent studies show that these models can encode social biases from large pre-training corpora and even amplify biases in downstream applications. To address this challenge, pro- pose CT an efficient and effective debias- while-prompt tuning method for mitigating bi- ases via counterfactual contrastive prompt tun- ing on downstream tasks. Our experiments conducted three extrinsic bias benchmarks demonstrate the effectiveness of Co on bias mitigation during the prompt tuning process and its adaptability to existing upstream debi- ased language models. These findings indicate the strength of Co"PT and provide promising avenues for further enhancement in bias miti- gation on downstream tasks.
Much prior effort has focused primarily de- biasing the representations learned during the pre- training process, e.g., through projection (Dev al., 2020; Liang et al., 2020; Ravfogel et al., 2020; Kaneko and Bollegala, 2021), further pre-training on unbiased external corpora (Webster et al., 2020; Lauscher et al., 2021; He et al., 2022), or fine- tuning to debias (Cheng et al., 2021; Guo et al., 2022). The effectiveness of such debiasing efforts is typically measured on intrinsic benchmarks like SEAT (Sentence Encoding Association Test) which computes the association between demographic terms (e.g., woman, man) and stereotype terms (e.g., science, art). An unbiased model should dis- play no difference in the similarity between the representations of these terms (May et al., 2019).
1 Introduction
While these existing approaches help reduce so- cial biases under intrinsic measures, these debias- fien-finetuee methods are based on the hypothesis that if an upstream model is unbiased, it will also preserve its fairness effeets on downstream tasks during the fine-tuning process. However, recent research investigating the relationship between in- trinsic and extrinsic benchmarks (which evaluate fairness downstream applications) finds these two benchmarks correlate weakly (Kaneko et al., 2022). Furthermore, they observe that models, even after being debiased, tend to re-acquire even am- plify biases (e.g., instance-related biases and label- related biases) during the fine-tuning process downstream tasks (Zhao al., 2017: Leino et al., 2019). Thus, this mismatch leads our motivat- ing research question How can we develop an efficient and effective method to mitigate bias on downstream tasks?
Pre-trained language models (PLMs) are widely used in many real-world applications, demonstrat- ing remarkable performance (Devlin et al., 2019; Brown et al., 2020). However, has been demon- strated that PLMs encode unfair social biases in their parameters based on their pre-training step over large-scale text corpora (May et al., 2019). Furthermore, these biases for example, based on gender, race, religion- can easily propagate to the downstream tasks that use these PLMs (Kaneko and Bollegala, 2021). For example, "She a nurse" can have a higher conditional likelihood than "He is nurse" " in the language modeling task, and "nurse" " can have higher coreference scores to "she" than "he" in the coreference resolution task (Lu et al., 2020). Considering that NLP applications like machine translation systems, resume filtering systems, dialogue systems, and speech recogni- tion (Tatman, 2017) are widely used by millions users globally, it is crucial to mitigate the social biases present in PLMs and strive for models that
To answer the aforementioned question, we propose Co trebias-whilee while-pronpt tuning ap- proach through Counterfactual Contrastive Prompt

---

Two dogs are running.
The man is playing the piano.
The woman is playing the piano.
Pre-trained Language Model
Original impul
Class Label (with linear head)
Frozen
Augmented input,
The overview of Co First, we construct counterfactual pairs from the training data. Then, we learn debiased continuous prompts by simultaneously optimizing prompt tuning loss Lyt on downstream tasks and contrastive loss L between the counterfactual pairs.
Figure I:
Tuning. In this method, we first freeze all pa-
Furthermore, Co²PT can be extended to existing debiased language models, effectively bridging the gap between debiased upstream models and downstream tasks.
rameters of the PLM and add tunable continu- prompts for every layer. Unlike the previous bobiia-then-innttur methods that require expen- sive re-training of the original PLM and risk knowl- edge forgetting, this deep prompt tuning framework saves computational and memory resources while preserving the original pre-trained knowledge and language modeling ability (Li and Liang, 2021; Liu et al., 2022). To ensure that fair system gener- ates unbiased results regardless of the demographic terms used, construct counterfactual pairs di- rectly from the training data, climinating the need for external corpora that heavily depend on their quality for debiasing. Specifically, we replace de- mographic terms associated with either the domi- nant or minoritized group in the training data with terms representing the opposite group. Then, we integrate the ability to mitigate bias into the prompt parameters through contrastive objective between counterfactual pairs while maintaining the parame- ters of PLMs frozen. CP can be integrated into existing debiased models help them mitigate bi- ases on downstream tasks and offer flexibility in addressing different kinds of bias. These advan- tages establish Co²PT an efficient and effective method for mitigating bias in downstream tasks. In conclusion, proposed CP mitigates bias on downstream tasks through prompt tuning, making the following contributions:
2 Related Work
Several approaches have been proposed for debias- ing pre-trained language models such as projection- based methods (Dev et al., 2020; Liang et al., 2020; Ravfogel et al., 2020; Kaneko and Bollegala, 2021). post-hoc text generation techniques (Schick et al., 2021), adversarial methods (Han et al., 2021), fine- tuning on biased prompts (Guo et al., 2022), with contrastive objective (Cheng et al., 2021) or with augmented data (Zhao et al., 2018), additional pre- training methods re-balanced corpus through counterfactual data augmentation (Webster et al., 2020; Lauscher et al., 2021; Meade et al., 2022) with a contrastive objective on sender-balancee en- tailment pairs (He et al., 2022), using dropout regu- larization (Webster et al., 2020), through parameter- efficient methods (Lauscher et al., 2021; Yang et al., 2022; Xie and Lukasiewicz, 2023) or with con- trastive objective (Li et al., 2023). While some works do not require access to an external corpus or do require retraining the entire model, most prior methods primarily focus on mitigating bias within the model's intrinsic characteristics and eval- uate the effectiveness of bias mitigation through intrinsic bias benchmarks, e.g., SEAT (May et al., 2019), StereoSet (Nadeem et al., 2021), and CrowS- Pairs (Nangia et al., 2020). Subsequently, they fine-tune the debiased models downstream tasks and demonstrate that their debiased models retain the language modeling ability and the performance on downstream tasks or extrinsic bias benchmarks, which evaluate faimesss downstream tasks by
Co²PT achieves time and memory efficiency without requiring access external corpus or retraining the entire model.
Over three extrinsic bias benchmarks, we show that PT effectively mitigates bias ampli- fied during the prompt tuning process on down-
stream tasks.

---

testing whether the models exhibit different perfor- mances among different populations.
Counterfactual Pairs Construction. Then, the first key question is: how interject the de- biasing capability into the continuous prompis? An unbiased model should make the same pre- dictions independent of the bias-attribute term, thus we apply counterfactual data augmentation to generate counterparts of training examples from the training data during prompt tuning. Con- cretely, let represent the training corpus and let W {(u1, w2; ..... Wm N be a of N bias- attribute term pairs. For each sentence &; in S and each pair (Wi, W2,.... Wm) in W, for any w; in s, we replace with the term along the opposite bias direction. Take the binary-gender debiasing task shown in Figure/ for example, the bias-attribute terms are [(man, woman), (he, she), ...). The "man" is in the input sentence "The man is playing the piano". We replace it with "woman" while leav- ing non-attin non-attribute words unchanged. Then the coun- terfactually augmented sentence is "The woman is playing the piano", and vice versa. The obtained counterfactual sentence of the original sentence 8; is denoted as $/z-
Nevertheless, recent research shows that these feefas- methods will re-acquire or even amplify biases during the fine-tuning pro- cess on downstream tasks and that intrinsic and extrinsic evaluation bias benchmarks correlate poorly (Goldfarb- Caarrfarb-Tarrant al., 2021; Cao al., 2022: Kaneko et al., 2022). They encourage re- searchers to focus directly on extrinsic measures of bias of specific applications when addressing bias mitigation (Goldfarb- Tarrant et al., 2021).
Thus, we focus in this paper on mitigating bias on downstream tasks and evaluate using extrin- sic evaluation benchmarks directly. In addition, different from the previous methods requiring fur- ther pre-training on the counterfactually augmented sentences from an external corpus, e.g., English Wikipedia (Zmigrod et al., 2019; Wehster et al., 2020; Meade et al., 2022), BookCorpus (Lauscher et al., 2021), News- Cowmenmmmentrry v15 (Yang et al., 2022) or NLI (He et al., 2022), our methods achieve time and memory efficiency by eliminating the need for external corpus access model retraining.
Counterfactual Contrastive Learning. The coun- terfactual pairs construction allows us to achieve balance in inputs containing bias-attribute terms. However, how we ensure that model gener- ates consistent predictions for both 8; and sy which possess similar semantic meaning but differ in bias direction? To make the model generate predictions independent of biased attributes, it is important for sentences with similar semantics but along dif- ferent bias directions to be closer (Cheng et al., 2021; He et al., 2022). We apply contrastive learn- ing, of which the objective is to obtain meaningful representations by bringing semantically similar neighbors closer and pushing apart the dissimilar neighbors (Gao et al., 2021; Dong et al., 2023b; Li et al., 2023). In this work, input sentence 8; and its counterpart sy are semantically related but in opposite bias directions. We let h, and h' denote the representations of s; and s and then concate- nate with the continuous prompt representation p as positive pairs. Then take the cross-entropy objective with in-batch negatives (Gao et al., 2021). The training objective for (h, h!) with mini-batch of N pairs is:
3 Co²PT: Debiasing via Counterfactual Contrastive Prompt Tuning
We propose Co²PT, a debias-while- promptt tuning parameter-effiieent method for mitigating biases on downstream tasks via counterfactual contrastive prompt tuning, presented in Figure 1. Concretely, Co²PT mitigates bias in PLMs by leveraging coun- terfactual pairs from training data produce debi- ased representations during prompt tuning.
Deep Prompt Tuning. First, we introduce the backbone framework of CO - deep prompt tun- ing. We incorporate continuous prompts prefix tokens in every layer of the PLM. By doing this, we have more tunable task-specific parameters to enhance per-task capacity while maintaining pa- rameter efficiency (Li and Liang. 2021; Liu et al., 2022; Wang et al., 2022; Dong et al., 2023a). Be- sides, it can achieve comparable performance to fine-tuning, outperforming methods that only add trainable continuous prompts into the input embed- ding layer (Lester et al., 2021; Liu al., 2021), which underperform the fine-tuning methods, espe- cially when the model size is not large (Liu al., 2022). The prompt tuning loss of proposed CP on the downstream task is represented as Cpt. e.g., cross-entropy loss for a classification task.
Ld = log ssimppsh..p00)//
2==1 hammpoh..pot/// (1)
where sim(x; (x;,;) y:) is the cosine similarity of x; and

---

Ygi y:). x,y,/ | Iy @ is the con- catenation of two representations, and is a tem- perature hyperparameter.
4.1 Bias Evaluation
Extrinsic bias benchmarks assess bias via perfor- mance gap between different groups in downstream tasks. In this work, we evaluate Co²PT on three widely used extrinsic bias benchmarks: Bias-STS- B, Bias-NLI, and Bias-in-Bios.
For counterfactual pairs s() in the single- sentence classification task, is the original sen- tence from the training data and of is the augmented sentence that has the same semantic meaning 8; but in different bias direction. For sentence-pair classification, like in the SNLI task, with 2; as the premise and Yi as the hypothesis, 6; is the origi- nal wremise-hyootheee pair (I,, 34) while s is the counterfactual augmented (x',, v/). Similarly, the sentence representations are concatenated with continuous prompts to calculate the contrastive loss through Equation 1.
Bias-STS-B (Webster et al., 2020) is adapted from the STS-B task to evaluate gendered correlations, which requires models to predict the semantic simi- larity between pairs of sentences. Specifically, 276 sentences are collected from the test set as tem- plates and then gendered terms (man, woman) and professional terms from Rudinger et al. (2018) are inserted into each template, forming 16,980 sen- tence pairs. For instance, if the template is "A man is walking", then the sentence pairs are ("A man is walking", "A nurse is walking") and ("A woman is walking". "A nurse is walking"). If a model is un- biased towards gender terms, it should assign equal similarity scores to both pairs. We calculate the average absolute difference between the similarity scores of sentence pairs containing male and fe- male terms, and how often the difference between "male" and "female" sentence pairs т, where we report the results for 0.1 and r 0.3 (Webster et al., 2020). A lower value indicates less bias.
hhytthessss pair
Learning Objectives. Finally, the continuous prompts teebbis by simultaneously opti- mizing the prompt tuning loss ppt on downstream tasks and contrastive loss Lu between the counter- factual pairs:
L Cpt + aeCols
(2)
where is a tunable coefficient hyperparameter. As stated before, we only tune the parameters of the de- biasing continuous prompts while maintaining the parameters of PLMs frozen throughout the train- ing. After the counterfactual contrastive prompt tuning, the debiasing knowledge is stored in the prompt parameters. This approach not only retains the knowledge within the original parameters of PLMs but is also flexible and adaptable to different downstream tasks. For example, we can train dif- ferent prompts for different bias dimensions such as gender, race, and religion. These prompts can then be combined and applied to downstream tasks, Morcover, considering that prior research primarily concentrates on binary gender. it is more efficient to extend its application to non-binary gender without requiring re-training new debiased models.
Bias-NLI (Dev et al., 2020) is natural language in- ference dataset consisting of neutral sentence pairs to evaluate the gender-occupation bias. It is con- structed by populating the template: "The subject verb a/an object", leading to 1,936,512 instances. Concretely, the verb and object slots are filled with activitics, c.g., ate a bagel. Then they cre- ate neutral entailment pairs by filling the subject slot with occupation term with a strong gender correlation, c.g., "nurse", for the hypothesis, and "woman", for the premise, resulting in the instance: The woman ate a bagel; The nurse ate a bagel. neu- tral. Bias is defined as deviation from neutrality and measured by three metries: (1) Net Neutral (NN): the average probability that the model as- signs a neutral label for all instances, (2) Fraction Neutral (FN): the percentage of instances that the model predicts the neutral label and (3) Threshold: (T: T): the fraction of examples whose probabil- ity of neutral are above т. We report the results for t 0.5 and T 0.7 following (Lauscher et al., 2021: He et al., 2022). All three metries will attain for a bias-free model.
4 Experimental Setup
We design experiments to test the effectiveness of our proposed PP approach toward answering four questions: RQ1: Will CT mitigate bias on downstream tasks effectively? RQ2: How will the existing intrinsic debiased methods perform on the downstream tasks when they are combined with PP? RQ3: What impact do different modules have on the design of Co? RQ4: How do hy- perparameters affect Co?PT?
Bias-in-Bios (De-Artcaga et al., 2019) is a large- scale English dataset studying gender bias in oc-

---

cupation classification from the Common Crawl corpus. We report the overall accuracy of the task as well as the accuracy breakdown based on gender. To quantify gender bias, we compute the difference in true positive rates (TPR) between genders across various occupations, denoted as GAP and de- fined as follows:
ADELE and its variant ADELE-TA (Lauscher et al., 2021) inject and train debiased adapters via masked language modeling on the counterfactually augmented corpus. Context-Debias (Kaneko and Bollegala, 2021) is fine-tuned via an inner-product loss and squared distance loss. Auto-Debias (Guo et al., 2022) uses beam search to search for biased prompts and then uses these biased prompis to fine- tune PLMs by minimizing the disagreement be- tween predicted [MASK] token distributions. MA- BEL (He al., 2022) is additionally pre-trained on all entailment pairs that contain gendered terms from SNLI and MNLI data with a contrastive loss, an alignment loss, and an optional masked lan- guage modeling loss. BERT (Devlin et al., 2019) is fine-tuned BERT model on the downstream tasks while BERT+CDA is fine-tuned BERT model on the counterfactually augmented data from the training sets. PT (Liu et al., 2022) adds con- tinuous prompts to each layer of the models and then tunes the prompts on downstream tasks. The backbone models for ZariCDA and ZariDO are BERT-large- uncased whereas other baselines are BERT-base uncased (Devlin et al., 2019).'
GAP = T TPR..gl
(3)
where g represents the proportion of individu- als correctly predicted given their gender 9. and and ~g are binary genders. Following (Romanov et al., 2019; Ravfogel et al., 2020; He et al., 2022), we also calculate the root mean square of the per- occupation TPR gender gap GAP TPR pations
over all occu-
GAPEMS g
(G)PTR))
(4)
O
DEO
A value closer to 0 indicates lower degree of bias.
4.2 Datasets and Setup
STS-B and SNLL We fine-tune the models on SNLI and STS-B training sets and pick the one that performs best the validation set and then evaluate bias using Bias-STS- B and Bias-NLI, re- spectively. Bias-in-Bios. We use the same data as Ravfogel et al. (2020) which contains 393,423 biographies and 28 profession classes. We split cain//alidation/test by 65/25/10 following (De- Arteaga et al., 2019; Ravfogel et al., 2020). The dataset statistics are shown in Table 1. For SNLI and Bias-in- in- in-iios, we report accuracy over classifi- cation while we report the Pearson and Spearman correlations for STS-B.
4.4 Implementation Details
For fine-tuning debiased baselines and vanilla BERT, we use the models released by the au- thors. We set the max length of the sentence to be 128, the learning rate to be 2c 5, the batch size be 64, and train for 10 epochs. For AT+CDA, learning rate is 2e-5 and batch size is 128. For PT and CT² the backbone models are bert-base-uncast
with the learning rate set to le - 2 and the prompt length set to 20 and are trained for 30 epochs. The batch size is set to 32. For hyperparameters 7 and in Equation 1 and 2, we set = 0.05 and 1.0. All experi- ments are run on a single NVIDIA RIX A5000 24GB GPU. For each run, we save the model that performs the best on the development set and evaluate it on extrinsic benchmarks. We report the average results across three runs. All code and data are available at https: //gi thub. com/ jCcp/ additional experimental de- tails and standard deviations are in Appendix A and Appendix C, respectively.
4.3 Baseline Models
We compare Co²PT with six upstream debiased models fine-tuned on the downstream tasks, and three baselines fine-tune or prompt-tune BERT models on the downstream tasks: ZariCDA (Web- ster et al., 2020) is pre-trained from scratch over counterfactual data augmented from English Wikipedia and ZariDO (Webster et al., 2020) is ad- ditionally pre-trained with increased dropout rate.
Train
Validation
Bias- Test
STS-B SNLI
5,749
1,500
16,980
550,152
10,000
1,936,512
Bias-in Bias-in-Bioo Bios
255,710
39,369
98,344
'We keep the results of these two BERT- SERR--aage-ane largge-nrgeeeee checkpoints since I Lauscher et al. (2021) also compares these methods with their BERT-bal base-baseeueee base-uncased method.
Table 1: Dataset statistics.

---

5 Debiasing Effectiveness (RQ1)
Model
FN:
T:0.5t
T:0.71
Acc.
BERT
0.824
0.868
0.867
0.811
0.909
We now investigate the effectiveness Co in mitigating bias on three extrinsic bias benchmarks.
BERT+CDA
0.873
0,942
0.941
0.894
0,905
ZariCDA"
0.786
0.828
0.826
0.765
0,912
ZariDO
0.747
0.782
0.779
0.711
0.913
Context-Debias
0.873
0,919
0.919
0.877
0,910
Bias-STS-B. First, we focus on the Bias-STS-B benchmark. As Table 2 indicates, CP shows the lowest bias scores across all metrics and achieves similar model performance downstream tasks the other debiased baselines. We observe that some debiased models exhibit higher bias scores than the original BERT, indicating that debiased language models can relearn biases during fine- tuming on downstream tasks. For example, Auto- Dehias, one of the state-of-the-art debiased models, demonstrates strong fairness intrinsic bench- marks. such SEAT, scores 0.312 in average abso- lute difference, showing higher bias level than the original BERT and most of the other baselines. On the other hand, MABEL, which shows strong per- formance in debiasing downstream tasks, achieves a competitive score of 0.081. Furthermore, com- pared to fine-tuning the original BERT model on the STS-B training set, PT results in a higher bias score of 0.321 compared to 0.282. This suggests that while fine-tuning only the number of prompt tokens may be parameter efficient, it can result in increased bias due to the presence of an unbalanced dataset. For CT²PT, we observe a significant reduc- tion in the bias score with the average absolute difference decreasing from 0.321 to 0.058, from 0.749 to 0.167 when the difference exceeds 0. and from 0.369 0.005 when the difference ceeds 0.3. These findings indicate substantial improvement in the ability mitigate bias.
ADELE-TA
0.504
0.557
0.813
AT+CDAΣ
0.659
0.799
0.758
0.510
0.849
Auto-Debias
0.813
0,849
0,848
0.795
0,908
MABEL
0.853
0.892
0.891
0.846
0.911
PT 0.741
0.812
0.808
0.729
0.898
CO (ours)
0.877
0.965
0.962
0.905
0.886
Table Evaluation on Bias-NLL. t: results are fine- tuned MNLI and reported from the ADELE-TA model in the original paper; f: adapter tuning on coun- terfactually augmented data; backbone model is BERT- LERT-Large-uncase Other baselines are fine-tuned on SNLJ.
scores than the original BERT across all metrics while the other baseline methods amplify biases during the fine-tuning, Similarly, Auto-Debias performs well on the SEAT benchmark but ex- periences increase in bias when applied downstream tasks, mirroring the trend observed in the Bias-STS-B extrinsic benchmark. Moreover, ADELE, another parameter- forms poorly in both bias mitigation and model accuracy. Similar to Bias-STS-B extrinsic bench- mark, PT amplifies biases during the tuning pro- cess, resulting in decline in the NN score from 0.824 to 0.741 and the FN score from 0.868 0.812. By employing Co²PT, we observe signif- icant improvements with the NN score rising 0.877 (from 0.741) and the FN score reaching 0.965 (from 0.812), indicating the effectiveness of PT on bias mitigation.
method, per-
Bias-NLL Next, we focus on the Bias-NLI extrin- sic benchmark shown Table 3. BERT+CDA, Context-Debias and MABEL achieve lower bias
Bias-in-Bios. Next we show the performance on the Bias-in-Bios benchmark in Table 4. Among all the baselines, the ZariCDA achieves the low- GAPTPR score of 2.667 while the BERT+CDA
Model
Dift.
r:0.1
r:0.3]
Pear: / Spear.
BERT
0.282
0.867
0.417
0,883/ 0.879
BERT+CDA
0.131
0.511
0.080
0.885 0.881
Model BERT
GAP
GAPPMS
Aec.
2,822
0.119
0.830
Z
0.112
0.445
0.048
0.892/ 0.889
BERT+CDA
2,758
0.113
0.841
*
0.347
0.922
0.585
0.880/ / 0.878
7
2,667
0.135
0.848
ADELE!
0.121
0.889
ZariDO
2,720
0.135
0.847
Context-Dehias
0.332
0.916
0.539
0.879 0.876
Context- -Deeias
3,031
0.119
0.831
Auto-Dehias
0.312
0.902
0.502
0.884 0.880
Auto- Debias
2,690
0,125
0.831
MABEL
0.066
0.204
0.013
0.889/ 0.885
MABEL
2,839
0.126
0.826
PT
0.321
0.749
0.369
0.885
PT CT (ours)
3.171
0.129
0.820
CP (ours)
0.058
0.167
0.005
0.884/ / 0.880
2.537
0.123
0.824
Table 2: Evaluation on Bias-STS-B. results are re-
Table 4: Evaluation on Bias-in-Bin -ioo....B..... backbone model BERT-large BERT-large-unche lew.gaendcasddd...... The results of ADELF. F. on this benchmark are not reported in the original paper.
ported from the ADELE model in the original paper; backbone model is HERT-Large-uncaser

---

achieves the lowest GAPRMS score 0.113, Fur- thermore, PT exacerbates bias and results in an increase in the GAPTPR score from 2.822 to 3.171 and GAPRMS from 0.119 to 0. 1.9 In contrast, CO reduces the GAPTPR score from 3.171 to 2.537 and GAPRMS from 021 to O. 0.12 demon- strating its effectiveness in mitigating bias in the occupation classification task.
the bias score from 0.321 in PT 0.291, showing the effectiveness of the straightforward counterfac- tual data augmentation approach. However, the improvement is less than Co²PT, implying the ne- cessity of the contrastive learning module.
Impact of contrastive module. To investigate the impact of the contrastive module, instead of em- ploying constructed counterfactual sentence pairs as positive pairs for contrastive loss, supervised contrastive loss by encoding the same input twice and get two embeddings with different dropout masks 2, z' (Gao al., 2021). Then the contrastive objective becomes:
6 Integrating Co²PT with Existing Debiased Models (RQ2)
un-
One benefit of a commpt-then-fineee model like Co2PT that it can be easily integrated with ex- isting upstream debiasing methods. Here we in- vestigate the applicability CT²PT to three exist- ing debiased models to bridge the gup in utilizing upstream debiased models for downstream tasks. Based on the comparison results - before and after applying Co²PT - shown in Table 5, Co²PT signif- icantly reduces the bias scores for Context-Debias and Auto-Debias: 0.088 versus 0.332, and 0.068 versus 0.312, respectively. For MABEL, which achieves low bias scores, there is no significant effect on the bias score. Additionally, Co im- proves the model performance on the downstream tasks. These results clearly demonstrate the ef- fectiveness of integrating CT²PT into established debiased models for downstream tasks. This ability enables the existing debiased models to achieve strong performance downstream tasks while si- multancously maintaining a low bias level.
esim(pan pSh
Led log
(5)
ΣN sim(pan pan
which is optimized with the prompt tuning loss Sn together (denoted as PT+SCL). PT+SCL sur- passes both PT and PT+CDA by achieving a large reduction in bias score to 0. 161, demonstrating the effectiveness of the contrastive module.
Impact of adding contrastive loss for non- augmented inputs. In Co²PT, we only consider counterfactually augmented pairs in the contrastive module. To explore the necessity of including in- puts without demographic terms in the contrastive module, we incorporate unsupervised contrastive loss for non-augmented input like Equation 5 as L'd and tuned with contrastive loss La for counter- factually augmented pairs (Equation 1) and E (de- noted as Co PT+CLL))) The bias score of 0.117 achieved by CO*PT+ is higher than Co²PT and this indicates that incorporating contrastive loss for non augmenttd inputs in the training set is unnecessary.
Model
Dift.
r:0.1
7:0.3]
Pear. / Spear.
Context-Debias
0.332
0.916
0.539
0.879 00876
+ Co'PPT
0.088
0.361
0.010
0.885 0,881
Auto-Debias
0.312
0.902
0.502
0.884 0.880
+ CoAPT
0.068
0.231
0.005
0.883 0,878
MABEL 0.066
0.204
0.013
0.889/ 0.885
+ CTPPT
0.068
0.228
0.005
0.892 0.889
Compare CP with task-agnostic counterfac- tual pairs. To investigate whether integrating task- agnostic neutral entailment pairs can benefit debi- asing on the task, we use 142,158 entailment pairs augmented from SNLI and MNLI
Table 5: Performance of integrating Co Co PT with debi- ased models on Bias-STS-B.
7 Impact of Design (RQ3)
We perform an extensive ablation study to show how different components affect Co in Table 6. We use Bias-STS-B as the representative task for computational efficiency.
Model
Diff.
7:0.1|
r:0.3l
Pear: Spear.
PT
0.321
0.749
0.369
0.889 / 0.885
PT+CDA
0.291
0.747
0.351
0.890/ / 0.886
PT4SCL
0.161
0.548
0.133
0.883 / 10878
COLPT-SCL,
0.117
0.467
0.056
0.884 / 00878
Impact of counterfactual module. First, we per- form counterfactual data augmentation on the train- ing data containing bias-attribute terms. Then we conduct prompt tuning only on these augmented pairs (denoted as PT+CDA). PT+CDA reduces
PTANLICCL
0.080
0.280
0.022
0.881 7.887
PT+NLI+CI,
0.687
00881
PT+CDA+CL,
0.271
0.725
0.886 00883
Co'PT (P+CCD+CC)
0.058
0.167
0.005
0.884 0.880
Table Impact of different components.

---

datasets in He et al. (2022) as task-agnostic entail- ment pairs for STS-B task instead of using the task- specific counterfactual pairs augmented from the training set (denoted as PT+NLI+CL). We notice that although PT+NLI+CL does not outperform Co²PT, it shows a strong ability to mitigate bias compared other baseline methods. Thus, when working moderate amount of training data, it is better use counterfactually augmented pairs from the training data.
0.25
0.05
5.06
0.25
Scac
Dies
2.84
0.15
0.30
5,0
0.05
Figure in {0.005, 0.05, 0.5) while a = 1.0.
0.40
Compare CO with other contrastive objec- tive. For sentence-pair classification tasks, we also explore the contrastive loss that encourages the inter-association of entailment pairs (He et al., 2022). For the original input pair (s11,822 and its augmented pair (s/, 122,, 8i1 and 822 are treated positive pairs while 8;1 and 5/2 and other in- batch 8y2 are negatives, and vice versa (denoted as CLp). When using task-specific counterfactual pairs, + decreases the bias score to 0.271. Similarly, using task-agnostic counterfac- tual pairs, PT+NLI+CI, also reduces the bias score to 0.271. However, the bias mitigation ef- fect is not as significant as that achieved by CT which indicates the effectiveness of the contrastive module in CT²PT.
0.30
bles Score
0.25
State
05
0.35
0.30
0.82
0.00
Epoch
Figure 4: a in {0.1, 0.5, 1.0} while r = 0.05.
larger prompt length enables the model to achieve better model performance on downstream tasks more rapidly while still maintaining lower bias score. However, is important to consider that using larger prompt lengths means tuning more parameters, thus posing trade-off.
Impact of т. Then, we vary temperature 5 in (0.005, 0.05, 0.5). Figure 3 shows close signifi- cant bias mitigation effects when is set to 0.005 and 0.05 while exhibiting less effectiveness when is 0.5. This observation implies that higher temperature value corresponds less weight of the cosine similarity calculation, resulting in decreased effectiveness in bias mitigation.
8 Impact of Hyperparameters (RQ4)
Finally, we investigate the impact of three hyperpa- rameters: (i) the continuous prompt length; (ii) the temperature of contrastive loss La; and (iii) the coefficient a of total learning objective L.
Impact of prompt length. First, we experiment with the prompt length varying in {10, 20, 50}. as illustrated in Figure 2. Generally speaking, with more tunable prompt parameters, the model performs better downstream tasks. In addi- tion, when the prompt length is 10, Co shows a higher increase in bias score compared to the prompt length of 20 and 50. This indicates that
Impact of a. Last, we study the impact coeffi- cient a and vary the value in 0.5, 1.0). Fig- ure emphasizes that reducing the value of a at constant т, thus assigning less weight to the con- trastive module, leads to decreased bias mitigation effects. This analysis underscores the importance of carefully selecting appropriate hyperparameters.
Conclusion and Future Work
0.2
0.85
We propose Co²PT, an efficient and effective de- biasing method for mitigating bias in downstream tasks. We evaluate its effectiveness bias mit- igation and applicability to existing debiased up- stream models, and investigate how the design of each component and the selection of hyperparame- ters impact both its bias reduction capabilities and downstream task performance.
score
0.0
0.84
0.82
Epoch
Figure 2: Impact of prompt length.

---

Mitigating, non-gender and intersectional bias. Mitigating non-gender biases is challenging as some debiasing methods work well on reducing gender biases but show poor generalization capa- bilities in addressing biases beyond gender (Meade et al., 2022). Without re-training the model, Co²PT is flexible apply in order to mitigate different bias types in downstream applications. One can train different debiasing prompts tackle different bias dimensions such as gender, race, and religion. Fur- thermore, these debiasing prompts can be applied to mitigate intersectional bias by simply combining the corresponding prompts in downstream tasks.
faimess evaluation metrics for contextualized lan- guage representations. In Proceedings of the 60th Annual Meeting the Association for Computational Linguistics ( (Volume 2: Short Papers), pages 561-570, Dublin, Ireland. Association for Computational Lin- guistics.
Pengyu Cheng, Weituo Hao, Siyang Yuan, Shijing Si, and Lawrence Carin. 2021. FairFil: Contrastive Neu- Debiasing Method for Pretrained Text Encoders. In International Conference on Learning Representa- tions.
Maria De-Arteaga, Alexey Romanov, Hanna Wal- lach, Jennifer Chayes, Christian Borgs, Alexandra Chouldechova, Sahin Geyik, Krishnaram Kenthapadi, and Adam Tauman Kalai. 2019. Bias in bios: A case study of semantic representation bias in high- stakes setting. In Proceedings of the Conference on Fairness, Accountability, and Transparency, pages 120-128.
Limitations
While this work primarily addresses bias in English, we acknowledge the presence of more complicated bias cases in other languages. Therefore. future ex- ploration of existing methods or the development of new techniques to mitigate bias in other lan- guages would be valuable. Furthermore, despite the efficiency and comparable performance of deep prompt tuning compared to fine-tuning, it still un- derperforms fine-tuning on certain datasets when the model size is small. This will also limit the model performance of our method.
Sunipa Dev, Tao Li, Jeff M Phillips, and Vivek Sriku- mar. 2020. On measuring and mitigating biased in- ferences of word embeddings. In Proceedings of the AAAI Conference on Artificial Intelligence, vol- ume 34, pages 7659-7666.
Jacob Devlin, Ming-Was Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language under- standing. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguisties: Human Language Tech- nologies, Volume (Long and Short Papers), pages 4171-4186. Minneapolis, Minnesota. Association for Computational Linguisties.
Ethics Statement
this work, when investigating gender bias in pre-trained language models, we focus on the bi- nary definition of gender as the targeted attribute of discrimination. However, it is important to ac- knowledge that future research should also consider non-binary genders and other multi-class scenarios comprehensively address bias.
Xiangjue Dong, Yun He, Ziwei Zhu, and James Caver- lec. 2023a. PromptAttack: Probing dialogue state trackers with adversarial prompts. In Findings of the Association for Computational Linguistics: ACI. 2023, pages 1065 110666, Toronto, Canada. Associ- ation for Computational Linguistics.
Xiangjue Dong, Jiaying Lu, Jianling Wang, and James Caverlee. 2023b. Closed-book question generation via contrastive learning. Proceedings of the 17th Conference of the European Chapter of the Asso- ciation for Computational Linguistics, pages 3150- 3162, Dubrovnik, Croatia. Association for Computa- tional Linguistics.
Acknowledgements
We thank Nicholas Meade for the initial discussion and anonymous reviewers for valuable feedback.
Tianyu Gao, Xingcheng Yao, and Danqi Chen. 2021. SimCSE: Simple contrastive learning of sentence em- beddings. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Process- ing, pages 6894-69 I0, Online and Punta Cana, Do- minican Republie. Association for Computational Linguistics.
References
Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020. Language models are few-shot learners. Advances in Neural Information Processing Systems, 1371
Seraphina Goldfarb Tarrant, Rebecca Marchant, Ri- cardo Muñoz Sánchez, Mugdha Pandya, and Adam Lopez. 2021. Intrinsic bias metries do not correlate with application bias. In Proceedings of the 59th Au- nual Meeting of the Association for Computational
Yang Trista Cao, Yada Pruksachatkun, Kai-Wei Chang. Rahul Gupta, Varun Kumar, Jwala Dhamala, and Aram Galstyan, 2022. On the intrinsic and extrinsic

---

Linguisties and the 11th International Joint Confer- ence on Natural Language Processing (Volume 1: Long Papers), pages 1926-1940, Online. Association for Computational Linguistics.
Quentin Lhoest, Albert Villanova del Moral, Yacine Jernite, Abhishek Thakur, Patrick von Platen, Suraj Patil, Julien Chaumond, Mariama Drame, Julien Plu, Lewis Tunstall, Joe Davison, Mario Saško, Gun- jan Chhablani, Bhavitvya Malik, Simon Brandeis, Teven Le Scao, Victor Sanh, Canwen Xu, Nicolas Patry, Angelina McMillan-Major, Philipp Schmid, Sylvain Gugger, Clément Delangue, Théo Matus- sière. Lysandre Debut, Stas Bekman, Pierric Cis- tac, Thibault Goehringer, Victor Mustar, François Lagunas, Alexander Rush, and Thomas Wolf. 2021. Datasets: A community library for natural language processing. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Process- ing: System Demonstrations, pages 175-184, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics.
Yue Guo, Yi Yang, and Ahmed Abbasi. 2022. Auto- debias: Debiasing masked language models with automated biased prompts. In Proceedings of the 60th Annual Meeting of the Association for Compu- tational Linguistics (Volume I: Long Papers), pages 1012-1023, Dublin, Ireland. Association for Compu- tational Linguistics.
Xudong Han, Timothy Baldwin, and Trevor Cohn. 2021. Diverse adversaries for mitigating bias in training. In Proceedings of the 16th Conference of the Euro- pean Chapter of the Association for Computational Linguistics: Main Volume, pages 2760-2765, Online. Association for Computational Linguistics.
Xiang Lisa Li and Percy Liang. 2021. Prefix-tuning: Optimizing continuous prompts for generation. In Proceedings of the 59th Annual Meeting of the Asso- ciation for Computational Linguistics and the I 11th International Joint Conference on Natural Language Processing ( (Volume 1: Long Papers), pages 4582- 4597, Online. Association for Computational Lin- guistics.
Jacqueline He. Mengzhou Xia, Christiane Fellbaum, and Danqi Chen. 2022. MABEL: Attenuating gender bias using textual entailment data. In Proceedings of the 2022 Conference Empirical Methods in Nat- ural Language Processing, pages 9681-999 188022 Abu Dhabi, United Arab Emirates. Association for Com- putational Linguistics.
Yingji Li, Mengnan Du, Xin Wang, and Ying Wang- 2023. Prompt tuning pushes farther, contrastive learn- ing pulls closer: A two-stage approach to mitigate so- cial biases. In Proceedings of the 61st Annual Meet- ing of the Association for Computational Linguis- rics (Volume I: Long Papers), pages 14254-14267, Toronto, Canada. Association for Computational Lin- guistics.
Masahiro Kaneko and Danushka Bollegala, 2021. De- biasing pre- pre-trained contextualised embeddings. In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Lin- guistics: Main Volume, pages 1256-1266, Online. Association for Computational Linguistics.
Masahiro Kaneko, Danushka Bollegala, and Naoaki Okazaki. 2022. Debiasing isn't t enough! on the effectiveness of debiasing MLMs and their social biases in downstream tasks. In Proceedings of the 29th International Conference Computational Lin- guistics, pages 1299-1310, Gyeongju, Republic of Korea. International Committee on Computational Linguistics.
Paul Pu Liang. Irene Mengze Li, Emily Zheng. Yao Chong Lim, Ruslan Salakhutdinov, and Louis- Philippe Morency. 2020. Towards debiasing sentence representations. In Proceedings of the 58th Annual Meeting of the Association for Computational Lin- guistics, pages 5502-5515, Online. Association for Computational Linguistics.
Xiao Liu, Kaixuan Ji, Yicheng Fu, Weng Tam, Zhengx- iao Du, Zhilin Yang, and Jie Tang. 2022. P-tuning: Prompt tuning can be comparable to fine-tuning across scales and tasks. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 61-68. Dublin, Ireland. Association for Computational Lin- guistics.
Anne Lauscher, Tobias Lueken, and Goran Glavaš. 2021. Sustainable modular debiasing of language models. In Findings of the Association for Computational Linguistics: EMNLP 2021, pages 4782-4797 Punta Cana, Dominican Republic, Association for Compu- tational Linguistics.
Klas Leino, Matt Fredrikson, Emily Black, Shayak Sen, and Anupam Datta. 2019. Feature-Wise Bias Am- plification. International Conference on Learning Representations (ICLR).
Xiao Liu, Yanan Zheng, Zhengxiao Du, Ming Ding. Yujie Qian, Zhilin Yang. and Jie Tang. 2021. GPT understands, too, arXiv preprint 2rXiv:2103.10388
Kaiji Lu, Piotr Mardziel, Fangjing Wu, Preetam Aman- charla, and Anupam Datta. 2020. Gender bias in neural natural language processing. Logic, Language, and Security: Essays Dedicated to Andre Scedrov on the Occasion of His 65th Birthday, pages 189-202.
Brian Lester, Rami Al-Rfou, and Noah Constant. 2021. The power of scale for parameter-efficien prompt tuning. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 3045-3059, Online and Punta Cana, Domini- can Republic. Association for Computational Lin- guistics.
Chandler May, Alex Wang, Shikha Bordia, Samuel R. Bowman, and Rachel Rudinger. 2019. On measuring

---

social biases in sentence encoders. In Proceedings of the 2019 Conference of the North American Chap- ter of the Association for Computational Linguistics: Human Language Technologies, Volume (Long and Short Papers), pages 622-628, Minneapolis, Min- nesota. Association for Computational Linguistics.
Rachael Tatman. 2017. Gender and dialect bias in YouTube's 's automatic captions. In Proceedings of the First ACI. Workshop on Ethics in Natural Language Processing, pages 53-59, Valencia, Spain, Associa- tion for Computational Linguistics.
Yibo Wang, Congying Xia, Guan Wang, and Philip S. Yu. 2022. Continuous Prompt Tuning Based Textual Entailment Model for E-commerce Entity Typing. In 2022 IEEE International Conference on Big Data (Big Data), pages 1383-1388.
Nicholas Meade, Elinor Poole- Poole-Dayan, and Siva Reddy. 2022. An empirical survey of the effectiveness of debiasing techniques for pre-trained language models, In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics ( (Volume
I: Long Papers). pages 1878-1898, Dublin, Ireland. Association for Computational Linguistics.
Kellie Webster, Xuezhi Wang, Ian Tenney, Alex Beutel, Emily Pitler, Ellie Pavlick, Jilin Chen, Ed Chi, and Slav Petrov. 2020. Measuring and reducing gendered correlations in pre-trained models. arXiv preprint arXiv:2010.06032.
Moin Nadeem, Anna Bethke, and Siva Reddy. 2021. StercoSet: Measuring stereotypical bias in pretrained language models. In Proceedings of the 59th Annual Meeting of the Association for Computational Lin- guistics and the 1 1ith International Joint Conference on Natural Language Processing (Volume I: Long Papers), pages 5356-5371, Online. Association for Computational Linguistics.
Zhongbin Xie and Thomas Lukasiewiez. 2023. An em- pirical amalysis of parameter-efficient methods for debiasing pre- pre-trained language models. In Proceed- ings of the 61st Annual Meeting of the Association for Computational Linguistics ( (Volume I: Long Papers), pages 15730-15745 Toronto, Canada. Association for Computational Linguistics.
Nikita Nangia, Clara Vania, Rasika Bhalerao, and Samuel R. Bowman. 2020. CrowS-pairs: A chal- lenge dataset for measuring social biases in masked language models. In Proceedings of the 2020 Con- ference on Empirical Methods in Natural Language Processing (EMNLP), pages 1953-1967, Online. As- sociation for Computational Linguistics.
Ke Yang. Charles Yu, Fung, Manling Li, and Heng Ji. 2022. ADEPT: A DEbiasing PrompT' Framework. arXiv preprint arXiv:2211 205412211..444.444
Jieyu Zhao, Tianlu Wang. Mark Yatskar, Vicente Or- donez, and Kai-Wei Chang. 2017. Men also like shopping: Reducing gender bias amplification using corpus-level constraints. In Proceedings of the 2017 Conference on Empirical Methods in Natural Lan- guage Processing, pages Denmark. Association for Computational Linguis- tics.
Shauli Ravfogel, Yanai Elazar, Hila Gonen, Michael Twiton, and Yoav Goldherg. 2020. Null out Guard- ing protected attributes by iterative nullspace projec- tion. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 7237-7256, Online. Association for Computational Linguistics.
Copenhagen.
Jieyu Zhao, Tianlu Wang. Mark Yatskar, Vicente Or- donez, and Kai- Wei Chang. 2018. Gender bias in coreference resolution: Evaluation and debiasing methods. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguisties: Human Language Tech- nologies, Volume (Short Papers), pages 15-20, New Orleans. Louisiana. Association for Computational Linguistics.
Alexey Romanov, Maria De-Arteaga, Hanna Wal- lach, Jennifer Chayes, Christian Borgs, Alexandra Chouldechova, Sahin Geyik, Krishnaram Kenthapadi, Anna Rumshisky, and Adam Kalai. 2019. What's in a name? Reducing bias in bios without access protected attributes. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Volume I (Long and Short Papers), pages 4187-4195, Minncapolis, Minnesota. Association for Computational Linguistics.
Ran Zmigrod, Sabrina J. Mielke, Hanna Wallach, and Ryan Cotterell. 2019. Counterfactual data augmenta- tion for mitigating gender stereotypes in languages with rich morphology- In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 1651-1661, Florence, Italy. Asso- ciation for Computational Linguistics.
Rachel Rudinger, Jason Naradowsky, Brian Leonard, and Benjamin Van Durme. 2018. Gender bias in coreference resolution. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Short Papers), pages& 8- 14, New Orleans, Louisiana. Association for Computational Linguistics.
Timo Schick. Sahana Udupa, and Hinrich Schüitze. 2021. Self-diagnosis and self-dchiasing: A proposal for re- ducing corpus-based bias in NLP. Transactions of the Association for Computational Linguistics, 9: 9:1408- 1424.

---

Appendix
B More Ablation Studies
A More Implementation Details
Pooling method. Besides using pooled output, we also conduct experiments that use the average to- ken representation from the model's ' last hidden state as sentence representation, shown in Table 7. However, upon analyzing the results, Co²PTavg avg. is considerably more challenging for the prompts acquire the debiasing capability when using the average token representation compared to when uti- lizing the CLS token as the sentence representation.
A.1 Dataset and Extrinsic Bias Benchmarks
STS-B and SNLI datasets are from the Hugging Face Datasets library (Lhoest et al., 2021).² We use the same Btass-nn-Bite dataset used in Ravfogel et al. (2020).', the Bias-STS-B B used in Lauscher et al. (2021) and the Bia used in He et al. (2022). The code used to assess Bias-STS-B is modified from Lauscher et al. (2021), while the code for evaluating Bias-NLI and Bias-in-Bios is adapted from He et al. (2022). Different from the code employed in He et al. (2022), we conduct our
Model
Diff.
7:0.1
7:0.3
Pear. Spear.
PT
0.321
0.749
0.369
0.889 / 0,885
Co
0.390
0.823
0.497
0.891 / 0.86
Co (ours)
0.058
0.167
0.005
0.884 / 0,880
evaluation on the entire Bias-NLI dataset rather than just the top 10% it.
Table 7: More ablation results.
A.2 Debiased Baselines
C Standard Deviation of Results
Please refer the footnotes here for the source of released debiased models ZariCDA, ZariDO (Webster et al., 2020), Context- Debias (Kaneko and Bollegala, 2021). Auto- Debias (Guo et al., 2022), MABEL? (He et al., 2022) used on downstream tasks.
The standard deviation of evaluation results on ex- trinsic bias benchmarks Bias-STS-R Bias-NLI, and Bias-in-Bios are presented in Tables 8 to 10, respectively. addition, the results of integrating Co with existing debiased models and ablation study are shown in Table 11 and Table 12. These results indicate that Co consistently performs well with relatively low variability, demonstrating its effectiveness and reliability.
A.3 Implementation Details
We use BERT-baas-uncased in our experiments. For the single-sentence classification task, we prepend the [CLS] token before the input sentences and feed it into the BERT model to get the embed- ding of the [CLS] token as the sentence representa- tion. For the sentence-pais classification task, c.g., SNLI task, we prepend [CLS] before the premise and [SEP] token to separate premise and hypoth- esis y and feed it into the BERT model to get the embedding of the [CLS] token as the sentence rep- resentation. Fine-tuning and prompt tuning code rely on the Huggingface implementation."
Model
Dift_
T:0.1
7:0.3.
Pear. / Spear.
BERT
0.018
0.024
0.056
0.002 / 0.000
BERT+CDA
0.020
0.058
0.042
0.001 / /0.000
7ariCDA
0.030
0.146
0.031
0.004 / 0.003
ZariDO
0.020
0.011
0.044
0.003 / 0.003
Context-Debiar
0.042
0.052
0.098
0.005 / 0.005
Auto-Debias
0.006
0.018
0.020
0.002 / 0.001
MABEL
0.011
0.046
0.013
0,004/0,000
PT
0.018
0.016
0.024
0.001 0.000
CT²PT (ours)
0.009
0.056
0.001
0.001 0.000
Table 8: Results standard deviation Bias-STS-B.. the backbone models are BERT-Large-uncee
/https://githu/ https///gggtubb...muugggggnggggn accttaaassss.aaeaaaattttsss..s. Apache License 2.0.
D Visualization of Bias Mitigation Effects along Epochs
'The data is downloaded through https://github:
master cesterrddwnloodd_aaas caaddddddaa_hhhaaaaaaaaass sh MIT License.
"The checkpoints these two models are from https://
We visualize the changes of bias scores along epochs in Figure 5. The bias score of PT keeps increasing as the Pearson score increases, while CO²PT consistently maintains a low bias score, which indicates the effectiveness of 2P on bias mitigation.
gi ttub..cm.. wtt. cww/gooole- prooggeerreearcc--dttttttt///
Apache-
2.0 license.
'The checkpoints are from kanekomasahiro// cttpir////ooteet-eeiia.ass MIT license. "The checkpoints are from httpp://github... com/ rrenehere/Auto--ebbiss..
com/
'The checkpoints are from https:// https://hugiingaace siincetoonnnppmaaeelbberrr beeert-bas-uncaee MIT License.
"https: http://gttuuuuummgggiiiee/ com/huggingface

---

Model
NNT
FNT
T:0.51
T:0.71
Acc.
BERT
0.027
0.042
0.041
0.036
0,002
BERT+CDA
0.012
0.010
0.010
0.002
0.003
ZariCDA"
0.082
0.081
0.082
0.097
0.001
"ariDOO
0.078
0,083
0.083
0.093
0.001
Context-Dehias
0.030
0.046
0.046
0.050
0.001
Auto-Debias
0.010
0.019
0.019
0.012
0.002
MABEL
0.018
0.003
0.002
0.014
0.001
PT
0.023
0.032
0.031
0.028
0.001
Co (ours)
0.016
0.021
0.022
0.027
0.004
Table Results standard deviation Bias-NLI. the backbone models are BERT- BERT-Large-uncased
Model BERT
GAPUR! +
GAPRMS
+
Acc.
0.138
0.004
0.001
BERT+CDA
0.034
0.001
0.002
ZariCDA*
0.069
0.006
ZariDO
0.074
0.010
0.003
Context-Debias
0.053
0.006
0.003
Auto-Debias
0.146
0.014
0.002
MABEL
0.114
0.006
0.001
PT
0.066
0.005
0.001
PT (ours)
0.025
0,005
0.007
Table Results standard deviation on Bias-in-Bioc the backbone models are BERT-Large-uncar
Model
Dift.
r:0.1
r:0.3]
Pear. Spear.
Context- Context-Debias
0.042
0.052
0.098
0.005 0.005
CT
0.031
0.194
0.011
0.000 0.001
Auto-Dehias
0.006
0.018
0.020
0.002 0.001
+ CT
0.008
0.037
0.005
0.002 0.002
MABEL
0.011
0.046
0.013
0.004 0.004
+ CT
0.021
0.103
0.000
Table 11: Performance of integrating CT with debi- ased models on Bias-STS-B.
Model
Diff.j
r:0.1]
r:0.3]
Pear. / Spear.
PT
0.018
0.016
0.024
0.001 0.001
PT+CDA
0.008
0.008
0.007
0.000 0.000
PT+SCL
0.025
0.056
0.039
0.000 0.000
CoLPT-SCL,
0.023
0.094
0.033
0.001 0.000
PTANLI+CL
0.013
0.070
0.012
0.001 0.000
PTNNLICCI,
0.025
0.042
0.053
0.001 0.001
PE+CDA+CI,
0.030
0.031
0.051
0.001 0.002
Co (avg.)
0.032
0.022
0.040
0.001 0.000
CoPPT (PT+CDA+CL)
0.001
Table 12: Results standard deviation of ablation study.
OUTS
-0.08
0.84
ours
0.20
0.80
0.00
Epach
Figure 5: Visualization of bias mitigation effects and model performance along epochs.


Reviews:
[{"review_id": "V4hxZrfPuW", "reviewer": "Reviewer_eamV", "summary": "", "questions": "Line 378 (Equation 5), should there be a superscript TPR on GAP? \n\nIn Table 4, I am not sure how GAP^{TPR} is different from GAP^{RMS}. As defined in Equation 4,  GAP^{TPR}_{g, o} is calculated for each occupation and gender seperately. Is GAP^{TPR} simply obtained by averaging across all occupations and genders? If yes, does the only difference between GAP^{TPR} and GAP^{RMS} is that the absolute errors are averaged instead of the squared errors (followed by a square-root)? If yes, then the observed trends are slightly puzzling, as according to GAP^{TPR}  there is a reduction in bias over the BERT model directly fine-tuned, but that doesn't hold according to GAP^{RMS}.", "limitations": "", "rating": 3, "confidence": 4, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We appreciate your positive comments that our proposed method is scalable and the paper is well-written with detailed experiments. In the following, we address your points in turn:\n\n**Weakness 1a) To my best understanding all the baselines except perhaps ADELE are upstream debiased models.**\n\n1) As stated in Section 1 (Introduction), much prior effort has focused primarily on debiasing the representations learned during the pre-training process. Previous work is based on the hypothesis that if an upstream model is unbiased, it will also preserve its fairness effects on downstream tasks during the fine-tuning process. However, after fine-tuning the upstream debiased models on downstream tasks, our experimental results show that even if the upstream models have been debiased, those debiased models tend to re-acquire or even amplify biases during the fine-tuning process on downstream tasks, which is consistent with recent research findings and leads to our motivation – how to mitigate bias directly on downstream tasks more efficiently and effectively. \n\n2) For baselines that are ***not upstream*** –  ADELE, PT+CDA, and FT+CDA (the one added in the next question), we notice that Co$^{2}$PT also shows better performance on bias mitigation on downstream tasks, demonstrating the effectiveness of our proposed methods.\n\n**Weakness 1b)  PT+CDA performs CDA with prompt-tuning instead of full-fine tuning.**\n\n1) Co$^{2}$PT is a prompt-based method. Thus, we perform CDA with prompt tuning as an ablation study. This ablation study aims to evaluate the effectiveness and necessity of integrating the CDA module within the proposed Co$^{2}$PT framework.\n\n2) Co$^{2}$PT applies prompt tuning instead of full fine-tuning on downstream tasks, thus it is ***not fair*** to compare it with full fine-tuning. In response to the reviewer's suggestions, we explore FT+CDA, which performs CDA with full fine-tuning BERT model on downstream tasks:\n\n|Model |Diff. $\\downarrow$ | $\\tau$:0.1 $\\downarrow$ | $\\tau$:0.3 $\\downarrow$ | Pear./Spear. |\n|---|---|---|---|---|\n|FT | 0.282 | 0.867 | 0.417 | 0.883 / 0.879 |\n|**FT+CDA** | 0.131 | 0.511 | 0.080 | 0.885 / 0.881 |\n|Co$^{2}$PT | 0.058 | 0.167 | 0.005 | 0.884 / 0.880 |\n\nFT+CDA shows better performance on bias mitigation compared with the FT model (vanilla fine-tuning). These results underscore the effectiveness of CDA, mirroring the same tendency observed in PT and PT+CDA. Thus, incorporating these findings might appear *redundant*. \n\nWe can still add the results of FT+CDA, update Table 6 (Impact of different components), and add analysis in Section 7 (Impact of Design). We will also add the standard deviation 0.020, 0.058, 0.042, 0.001 / 0.000 of FT+CDA to Table 11 in the appendix.\n\n**Weakness 1c) ZariCDA and ZariDO are bert-large-uncased instead of bert-base-uncased.**\n\nWe kept the results of bert-large-uncased checkpoints for 3 reasons: \n\n1) The authors of [4] did not release bert-base-uncased checkpoints, nor code to obtain those checkpoints (https://github.com/google-research-datasets/Zari); \n2) [3] performed the same comparisons in their work (compare ADELE with ZariCDA and ZariDO); \n3) For the baseline that finetunes BERT on downstream task STS-B, the BERT-large-uncased model achieves better task performance and less bias on Bias-STS-B.\n\n|Model |Diff. $\\downarrow$ | $\\tau$:0.1 $\\downarrow$ | $\\tau$:0.3 $\\downarrow$ | Pear./Spear. |\n|---|---|---|---|---|\nBERT-base | 0.282 | 0.867 | 0.417 | 0.883 / 0.879 |\n**BERT-large** | 0.210 | 0.795 | 0.216 | 0.885 / 0.883\n\nThus, the inclusion of these two baselines where the backbone models are BERT-large is actually ***advantageous***, as we learn that: while Co$^{2}$PT only uses BERT-base as the backbone model, it can achieve even better performance on bias mitigation than the larger model. \n\n**Weakness 2a: I can understand, that downstream tasks for bias evaluation might be sparse for other dimensions, but perhaps one can synthetically generate data (like in case of Bias-STS-B and Bias-NLI) for dimensions like race, and religion as well.**\n\n1) The majority of previous works focus on one dimension (gender) [1,2,3,4,5]. We follow these previous works. We are interested in experimenting with our method in other dimensions (such as race and religion, as you suggest), and will update Section 9 to clarify our interest in this future work.\n\n2) Just like you said, the sentences containing predefined word lists of race and religion (six pairs for each dimension in [6]) are really sparse in the STS-B and NLI datasets. Therefore, formatting Bias-STS-B and Bias-NLI in this same manner may not yield a good selection of data. We will need to experiment with other, additional datasets in the future to investigate our method in these other dimensions (race, religion, etc.).\n\n**Weakness 2b: There are no experiments to support the claim to mitigate intersectional biases along these dimensions in downstream tasks.**\n\nTo make it clearer, this is the discussion of our future work, which we put in the *\"Conclusion and Discussion\"* section. Based on the results we get in the paper, the structure of our framework is flexible and has the potential to be applied to different dimensions, e.g., religion, race, and intersectional biases. Specifically, we can apply Co$^{2}$PT to train different continuous prompts for gender, religion, and race separately, and then concatenate the prompts to determine whether or not it can mitigate the intersectional biases on downstream tasks. On the opposite, previous methods need to train different individual models for different bias dimensions, which is time-consuming and expensive. We hypothesize that the generic structure of Co$^{2}$PT may make it more suitable and efficient for mitigating intersectional biases.\n\nWe will update the title of Section 9 to *\"Conclusion and Future Work Discussion\"*.\n\n**Question 1(a): Should there be a superscript TPR on GAP?**\n\nYes, thank you for asking about this. We will update it. \n\n**Question 1(b): I am not sure how GAP$^{TPR}$ is different from GAP$^{RMS}$.**\n\nFollowing previous work [1,2] GAP$^{TPR}$ denotes the gap between the true positive rates of the male prediction results and the female prediction results over all occupations, while the GAP$^{RMS}$ represents the root mean square of the true positive rates difference for each occupation.  \n\nThe motivation for using the root mean square follows from previous work: larger values have a larger effect in GAP$^{RMS}$. We will make it more clear and concise by updating the descriptions and subscripts in equation (4) from GAP_{g,o}^{TPR} = |TPR_{g,o} - TPR_{\\~g,o}| \n  to GAP^{TPR} = |TPR_{g} - TPR_{\\~g}|.\n\n**References**\n\n[1] Jacqueline He, Mengzhou Xia, Christiane Fellbaum, and Danqi Chen. 2022. MABEL: Attenuating Gender Bias using Textual Entailment Data. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 9681–9702, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics.\n\n[2] Yingji Li, Mengnan Du, Xin Wang, and Ying Wang. Prompt Tuning Pushes Farther, Contrastive Learning Pulls Closer: A Two-Stage Approach to Mitigate Social Biases. ACL 2023.\n\n[3] Anne Lauscher, Tobias Lueken, and Goran Glavaš. 2021. Sustainable Modular Debiasing of Language Models. In Findings of the Association for Computational Linguistics: EMNLP 2021, pages 4782–4797, Punta Cana, Dominican Republic. Association for Computational Linguistics.\n\n[4] Kellie Webster, Xuezhi Wang, Ian Tenney, Alex Beutel, Emily Pitler, Ellie Pavlick, Jilin Chen, Ed Chi, and Slav Petrov. 2020. Measuring and reducing gendered correlations in pre-trained models. arXiv preprint arXiv:2010.06032.\n\n[5] Masahiro Kaneko and Danushka Bollegala. 2021. Debiasing Pre-trained Contextualised Embeddings. In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume, pages 1256–1266, Online. Association for Computational Linguistics.\n\n[6] Nicholas Meade, Elinor Poole-Dayan, and Siva Reddy. 2022. An Empirical Survey of the Effectiveness of Debiasing Techniques for Pre-trained Language Models. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1878–1898, Dublin, Ireland. Association for Computational Linguistics.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. The proposed method is intuitive, easy-to-use, and scalable. The results indicate that the method does a reasonable job in mitigating the downstream task biases while maintaining the task performance.\n2. The experimental setup is quite detailed, with multiple datasets and baselines, as well as ablation studies and impact of different hyperparameters that help answer most of the questions the reader would have about the effectiveness of the proposed method. The claims of the paper are well supported with the experimental evidence. I have a few reservations about the choice of baselines which I detail more in the next question.\n3. The paper has nice presentation, is written well and easy to follow along.", "reasons_to_reject": "1. Baselines: To my best understanding all the baselines except perhaps ADELE are upstream debiased i.e. biases are mitigated in the pre-training representations and then those models are directly fine-tuned on the downstream tasks without any bias mitigation. To me this comparison sounds a bit unfair as the proposed method is fine-tuned to mitigate biases directly on the downstream task in contrast to the baselines. One obvious baseline that I can think of is performing CDA on the training data of downstream task and fine-tuning on the augmented data as in Zhao et al. 2018 [1] (a version of this is done in the ablation section when comparing with PT+CDA, but that versions performs CDA with prompt-tuning instead of full-fine tuning). In case, I have missed something here, I would be willing to revisit my stance on this point in the response period. Additionally, I feel the ZariCDA and ZariDO do not provide fair comparison as the base pre-trained model for these baselines is bert-large-uncased while for all other baselines and the proposed method it is bert-base-uncased. CDA and Dropout on bert-base-uncased instead would have been a more appropriate comparison. I can understand pre-training these models from scratch for these baselines can be quite expensive, but as it stands the comparison doesn't seem technically sound.\n\n2. Mitigating biases beyond gender dimension: One of the advantages of the proposed method as highlighted by the authors is that it can be easily extended to different bias dimensions (gender, race, religion), without having to fine-tune the entire model. In that light, it would have been valuable to see the effectiveness of the method on the dimensions other than gender. I can understand, downstream tasks for bias evaluation might be sparse for other dimensions, but perhaps one can synthetically generate data (like in case of Bias-STS-B and Bias-NLI) for dimensions like race, and religion as well. The paper also claims that the proposed method can be effective to mitigate intersectional biases along these dimensions in downstream tasks, but again there are no experiments to support the claim.\n\n\n[1] Jieyu Zhao, Tianlu Wang, Mark Yatskar, Vicente Ordonez, and Kai-Wei Chang. 2018. Gender Bias in Coreference Resolution: Evaluation and Debiasing Methods. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Short Papers), pages 15–20, New Orleans, Louisiana. Association for Computational Linguistics.", "questions_to_authors": "Line 378 (Equation 5), should there be a superscript TPR on GAP? \n\nIn Table 4, I am not sure how GAP^{TPR} is different from GAP^{RMS}. As defined in Equation 4,  GAP^{TPR}_{g, o} is calculated for each occupation and gender seperately. Is GAP^{TPR} simply obtained by averaging across all occupations and genders? If yes, does the only difference between GAP^{TPR} and GAP^{RMS} is that the absolute errors are averaged instead of the squared errors (followed by a square-root)? If yes, then the observed trends are slightly puzzling, as according to GAP^{TPR}  there is a reduction in bias over the BERT model directly fine-tuned, but that doesn't hold according to GAP^{RMS}.", "paper_topic_and_main_contributions": "The paper proposes a method for debiasing pre-trained language models using prompt-tuning approach that relies on counterfactuals generated and augmented in the training data accompanying a contrastive loss in addition to objective for the downstream task. Unlike most previous works that debias the representations learned during pre-training, the authors mitigate the biases directly for the downstream task as they point out referring to previous work that biases might re-surface once model is fine-tuned. The method is evaluated on a sentence similarity task, natural language inference (NLI) task, and on Bias-In-Bios dataset involving occupation classification. In all these three datasets, the proposed method outperforms the baselines, often with a significant margin. The authors further show that the method can be augmented with the pre-trained debiased models to prevent the resurgence of biases during fine-tuning. Finally, authors present an ablation study for the components involved in their method and show both the counterfactual data as well as contrastive learning objective aid significantly to the performance of their method.", "excitement": 3, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "XnIhEKZACi", "reviewer": "Reviewer_vAfa", "summary": "", "questions": "Q1. On lines 267-271, for a context-embedded PLM, how do you obtain a contextual representation of tokens (e.g., e^[CLS])?\n\nQ2. Explain the motivation for using prompts and what benefits it brings to Co2PT.\n\nQ3. The debiasing of models often affects the performance of the model because counterfactual pairs are different from the real corpus, how did the authors balance the debiasing of the model with the performance?", "limitations": "", "rating": 2, "confidence": 4, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We appreciate your positive comments that our proposed method is novel, efficient, and effective and the paper is well-written and clear. We turn now to each of your points:\n\n**R1: The specific role of prompts lacks explainability.**\n\nThere are two main reasons for our use of prompts:\n\n1) **Efficiency in tuning.** In this paper, we adopt deep prompt tuning as the backbone framework of Co$^2$PT. We incorporate continuous prompts as prefix tokens in every layer of the PLM. We should note that Co$^2$PT uses continuous prompts, which perform prompting directly in the embedding space of the model instead of discrete prompts, e.g., natural language phrases. As the goal of constructing prompts is to find a method that allows an LM to effectively perform a task rather than being for human consumption, it is not necessary to limit the prompt to human-interpretable natural language. Continuous prompts have their own parameters that can be tuned based on training data from the downstream task [1]. Thus, instead of fine-tuning the whole parameters of a pre-trained model, we only tune 0.1%-3% of parameters, which substantially reduces training time memory cost and per-task storage cost [2]. \n\nCo$^2$PT is efficient especially i) when the LLMs have **large sizes of parameters** (e.g., 11B), which is extremely expensive to be tuned; ii) the models are **black-box** and the users can not get access to the model parameters.\n\n2) **Effectiveness on bias mitigation.** Applying the counterfactual contrastive prompt tuning, we aim to store the debiasing knowledge in the prompt parameters. This approach not only retains the knowledge within the original parameters of PLMs but is also flexible and adaptable to different downstream tasks. In addition, it can be flexibly integrated with different existing debiased models for bias mitigation on downstream tasks.\n\n**R2: It is not possible to clarify the effect of prompt on debiasing methods, such as ablation experiments.**\n\n1) In our ablation experiments in Section 8 (Impact of Hyperparameters), we experiment with the effectiveness of continuous prompts in different lengths varying in {10, 20, 50}. Generally speaking, with more tunable prompt parameters, the model performs better on downstream tasks. This indicates that a larger prompt length enables the model to achieve better model performance on downstream tasks more rapidly while still maintaining a lower bias score. \n2) In addition, when the prompt length is set, assigning less weight to the contrastive module leads to decreased bias mitigation effects, as shown in Figure 4.\n\n**R3: Using prompts and counterfactual pairs is not necessarily a simpler resource than external corpora.**\n\n1) Not every downstream task has a corresponding appropriate external corpora, and using the same (perhaps non-corresponding) external corpora for different downstream tasks cannot guarantee the effectiveness of bias mitigation due to issues such as instance-related biases and label-related biases in the downstream datasets. For example: debiasing an upstream model using balanced Wikipedia data may not mitigate the end bias on a sentiment classification task. Further, bias can still be re-acquired or amplified during fine-tuning on sentiment classification tasks.\n2) Finding the proper external corpora for different downstream tasks is time-consuming and laborious. Thus, it can be more convenient and efficient to construct counterfactual pairs from existing downstream datasets.\n\n**Q1: For a context-embedded PLM, how do you obtain a contextual representation of tokens?**\n\nWe use BERT-base-uncased in our experiments. Thus, for the single-sentence classification task, we prepend the [CLS] token before the input sentence s and feed it into the BERT model to get the embedding of the [CLS] token as sentence representation. For sentence-pair classification task, e.g., SNLI task, we prepend [CLS] before the premise x and [SEP] token to separate premise x and hypothesis y and feed it into the BERT model to get the embedding of [CLS] token as sentence representation. \n\nWe will add this detailed explanation to the Appendix in Section A (More Implementation Details).\n\n**Q2. Explain the motivation for using prompts and what benefits it brings to Co$^2$PT.**\n\n1) The previous debias-then-finetune methods require expensive re-training of the original PLM and risk knowledge forgetting. By incorporating continuous prompts as prefix tokens in every layer of the PLM, we can save computational and memory resources while preserving the original pre-trained knowledge and language modeling ability (Li and Liang, 2021; Liu et al., 2022).\n\n2) In our main results (Tables 2-4), we show that Co$^2$PT achieves better performance on bias mitigation on downstream tasks. In addition, in Section 6 and in Table 5, we show that our prompt-based methods can be easily integrated with existing upstream debiased models; this is more efficient than full fine-tuning methods. Furthermore, without touching the parameters of debiased models, Co$^2$PT can preserve the debiased ability of existing upstream models while avoiding re-acquiring or amplifying bias on downstream tasks.\n\n**Q3. The debiasing of models often affects the performance of the model because counterfactual pairs are different from the real corpus, how did the authors balance the debiasing of the model with the performance?**\n\n1) To balance the debiasing of the model with the model performance on downstream tasks, we simultaneously optimize the prompt tuning loss L_{pt} on downstream tasks, which represents the model performance on downstream tasks and contrastive loss L_{cl} between the counterfactual pairs, which aims to pull the counterfactual pairs closer, shown in Equation (2) in Line 280. \n2) By performing a hyperparameter search to adjust the value of the tunable coefficient hyperparameter, we aim to find the appropriate hyperparameter that can achieve good bias mitigation performance while maintaining good downstream task performance, which is detailed described in Section 8 (Impact of Hyperparameters).\n\n**References**\n\n[1] Pengfei Liu, Weizhe Yuan, Jinlan Fu, Zhengbao Jiang, Hiroaki Hayashi, and Graham Neubig. 2023. Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processing. ACM Comput. Surv. 55, 9, Article 195 (September 2023), 35 pages. https://doi.org/10.1145/3560815\n\n[2] Xiao Liu, Kaixuan Ji, Yicheng Fu, Weng Tam, Zhengxiao Du, Zhilin Yang, and Jie Tang. 2022. P-Tuning: Prompt Tuning Can Be Comparable to Fine-tuning Across Scales and Tasks. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 61–68, Dublin, Ireland. Association for Computational Linguistics.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "A1. The authors propose a novel method for debiasing that is efficient and effective, and demonstrate its effectiveness through experiments.\n\nA2. The paper is well-written and clearly presents the authors' contributions and findings.", "reasons_to_reject": "R1. It seems reasonable to construct counterfactual pairs, and then fine-tune and debias the model, but the specific role of prompts still lacks explainability.\n\nR2. It is not possible to clarify the effect of prompt on debiasing methods, such as ablation experiments.\n\nR3. The authors claim that Co2PT does not require external corpora, but using prompts and counterfactual pairs is not necessarily a simpler resource than external corpora.", "questions_to_authors": "Q1. On lines 267-271, for a context-embedded PLM, how do you obtain a contextual representation of tokens (e.g., e^[CLS])?\n\nQ2. Explain the motivation for using prompts and what benefits it brings to Co2PT.\n\nQ3. The debiasing of models often affects the performance of the model because counterfactual pairs are different from the real corpus, how did the authors balance the debiasing of the model with the performance?", "paper_topic_and_main_contributions": "The main contribution of the paper is to propose a method called Co2PT (Counterfactual Contrastive Prompt Tuning) to mitigate social bias on pre-trained language models (PLMs) by de-biasing in downstream tasks via prompt tuning.The Co2PT method first adds tunable continuous prompts to the pre-trained model by adding tunable continuous prompts, and then incorporates the debiasing capability into the prompt parameters by constructing counterfactual pairs from the training data and optimizing the prompt parameters by comparing the objectives while keeping the PLM parameters frozen. The method saves computational resources and memory during the debiasing process and can effectively mitigate the bias amplified by the model during prompt tuning in the downstream task.", "excitement": 2, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "uWr3RB8Wns", "reviewer": "Reviewer_ppym", "summary": "", "questions": "A. Based on my understanding, when fine-tuning the eight baseline methods on downstream tasks, you use the original training set, not the augmented balanced training set, right? Given this, I'm confused by the results in Table 3 that ADELE's debiasing performance is much worse than the vanilla BERT. This may indicate some problems in your implementation. Is this due to that ADELE is trained on MNLI? If so, I think you should re-train ADELE using the same training set as yours for a fair comparison. \n\nB. In Table 6, you showed that PT+SCL can significantly reduce bias, which is achieved only by the contrastive loss of a sentence itself, without any bias terms telling the model what kind of bias (i.e., gender bias) it is to reduce. This seems a bit unreasonable to me. Could you explain more on this?\n\nC. I'm also curious about how PT+CDA in Table 6 is implemented. Do you assume the augmented instance has the same label as the original instance, and optimize the classification loss on both instances?\n\nD. You mentioned in lines 110-112 & 659-660 that different prompts can be combined to mitigate intersectional biases, but this is not obvious to me. Could you present a concrete method of combining different prompts? This is not a necessary problem for this paper, but would make the proposed method much more useful in practice.", "limitations": "", "rating": 3, "confidence": 4, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We appreciate your positive comments that our proposed method is parameter-efficient and compatible with upstream debiasing techniques and extensive ablation studies are conducted to justify the design. We do our best to address your concerns below:\n\n**Weakness 1: Some of the main ideas in this paper are not novel, e.g., augmenting balanced training sets and prompt tuning.**\n\nWe agree that training data augmentation and prompt tuning have been explored in prior literature. \n\nHowever: \n* As we point out in our contributions and related work, Co$^2$PT achieves time and memory efficiency without requiring access to an external corpus or retraining the entire model. \n* Co$^2$PT demonstrates better bias mitigation performance on downstream tasks. \n* Co$^2$PT can be flexibly integrated with existing debiased language models, effectively bridging the gap between debiased upstream models and downstream tasks. It can not only preserve the original debiased sentence representations by tuning prompts while freezing all parameters of the upstream models but also avoid re-acquiring or amplifying biases on downstream tasks.\n\n**Weakness 2(a): In Table 2, the reported results for ADELE are by finetuning all the parameters of a debiased language model, but I think the authors should compare to the more effective method called ADELE-TA.**\n\nWe report the ADELE-TA results on Bias-NLI because they were provided in their paper [1]. We report the ADELE results on Bias-STS-B because they were provided in their paper [1]. The results for ADELE-TA on Bias-STS-B, however, are not reported in their paper [1]. Hence, we do not report the ADELE-TA on Bias-STS-B in our work.\n\nWe will update the note \"results are reported from the original paper\" to \"results are reported from the ADELE model in the original paper” in Table 2 and “results are reported from the ADELE-TA model in the original paper\" in Table 3 to make it clearer.\n\n**Weakness 2(b): The proposed counterfactual contrastive loss is also applicable to fine-tuning or other parameter-efficient methods like adapter tuning, and I think these highly related baselines should be compared to justify the superiority of the authors' choice of prompt tuning.**\n\nIn our work, we not only aim to mitigate bias on downstream tasks, but also we aim to preserve the performance capabilities of our backbone model. Based on previous work [2], the deep prompt tuning framework shows better performance than other existing prompt tuning approaches (e.g., P-tuning (Liu et al., 2021), PromptTuning (Lester et al., 2021)., Prefix Tuning (Li and Liang, 2021), Soft Prompts (Qin and Eisner, 2021)). Deep prompt tuning incorporates continuous prompts in every layer of the PLM instead of other prompt tuning methods, like only adding continuous prompts in the input embedding layer, which have more tunable task-specific parameters to enhance per-task capacity while maintaining parameter efficiency (Li and Liang, 2021; Liu et al., 2022). Thus, we choose to use deep prompt tuning.\n\n**Question A: ADELE's debiasing performance is much worse than the vanilla BERT.**\n\nWe report ADELE-TA's results from their paper [1] showing the same findings as the recently published paper MABEL [3], where the TN and  FN of original BERT are 0.799 and 0.879--much better than ADELE.\n\n**Question B: PT+SCL can reduce bias, which is achieved only by the contrastive loss of a sentence itself., without any bias terms telling the model what kind of bias it is to reduce.**\n\nApplying contrastive loss to the sentence itself by encoding the same input twice and getting two embeddings with different dropout masks as noise, dropout acts as minimal “data augmentation” of hidden representations. It improves uniformity and the expressiveness of the representations [4], which we think is the reason why it also leads to bias reduction. While PT+SCL reduces bias, telling the model the biased terms (Co$^2$PT) reduces the bias further. This shows that emphasizing the biased signal to the model results in more robust debiasing.\n\n**Question C: I'm also curious about how PT+CDA in Table 6 is implemented. Do you assume the augmented instance has the same label as the original instance, and optimize the classification loss on both instances?**\n\nYes. As stated in the Counterfactual Pairs Construction paragraph (line 206 to line 228) we replace the bias-attribute terms in the training data with the terms in the opposite bias direction. Then the augmented instance has the same label as the original instance and optimizes the classification loss.\n\n**Question D:  You mentioned that different prompts can be combined to mitigate intersectional biases, but this is not obvious to me. Could you present a concrete method of combining different prompts? This is not a necessary problem for this paper, but would make the proposed method much more useful in practice.**\n\nOne intuitive way is to apply Co$^2$PT to train different continuous prompts for gender, religion, and race separately, and then concatenate the prompts to see whether it can mitigate these concatenated intersectional biases on downstream tasks. For example, a prompt trained to mitigate gender-related bias (e.g., women) and one to mitigate race-related bias (e.g., black), could be concatenated (to form one prompt from these two prompts) to mitigate intersectional bias (e.g., black woman).\n\nAnother way is to apply Co$^2$PT with intersectional data augmentation to get the prompts to mitigate intersectional biases. For example, in previous gender-related augmentation, we replace “man” with “woman” in the data, then for intersectional-related augmentation, we can replace “man” with “black woman”.\n\nThese are our initial ideas for future work. Feel free to collaborate if you are interested in the future work.\n\n**Add new citations in lines 115-117**\n\nThank you for pointing out the newly published papers in ACL 2023. We will add the citations as you suggested.\n\n**References:**\n\n[1] Anne Lauscher, Tobias Lueken, and Goran Glavaš. 2021. Sustainable Modular Debiasing of Language Models. In Findings of the Association for Computational Linguistics: EMNLP 2021, pages 4782–4797, Punta Cana, Dominican Republic. Association for Computational Linguistics.\n\n[2] Xiao Liu, Kaixuan Ji, Yicheng Fu, Weng Tam, Zhengxiao Du, Zhilin Yang, and Jie Tang. 2022. P-tuning: Prompt tuning can be comparable to fine-tuning across scales and tasks. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 61–68, Dublin, Ireland. Association for Computational Linguistics.\n\n[3] Jacqueline He, Mengzhou Xia, Christiane Fellbaum, and Danqi Chen. 2022. MABEL: Attenuating Gender Bias using Textual Entailment Data. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 9681–9702, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics.\n\n[4] Tianyu Gao, Xingcheng Yao, and Danqi Chen. 2021. SimCSE: Simple Contrastive Learning of Sentence Embeddings. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 6894–6910, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "The authors proposed a counterfactual contrastive loss and a prompt tuning-based method to mitigate gender bias in downstream tasks. The method is parameter-efficient and compatible to upstream debiasing techniques.\n\nThe authors conducted extensive ablation studies to justify their design, especially the counterfactual contrastive loss, and shared their codes for reproduction. The experimental setups such as reporting results from several runs, providing the validation details, etc., well support the transparency of the experiments.", "reasons_to_reject": "Some of the main ideas in this paper are not novel. (1) The idea of debiasing during adaptation to downstream tasks is not novel. Early works like [1][2] train coreference resolution models on augmented balanced training sets to mitigate gender bias in coreference resolution, which is similar to this paper's idea. (2) There are also recent works using prefix tuning or prompt tuning for bias mitigation[3]. \n\nThe debiasing baselines in this paper are a bit weak, which makes it unclear if the proposed method's performance is truly superior. E.g., (1) in Table 2, the reported results for ADELE is by finetuning all the parameters of a debiased language model, but in the ADELE paper[4], there's a more effective method called ADELE-TA, and I think the authors should compare to this baseline. (2) Besides, the proposed counterfactual contrastive loss is also applicable to fine-tuning or other parameter efficient methods like adapter tuning, and I think these highly related baselines should be compared with to justify the superiority of the authors' choice of prompt tuning.\n\n[1] Jieyu Zhao, Tianlu Wang, Mark Yatskar, Vicente Ordonez, Kai-Wei Chang. Gender Bias in Coreference Resolution: Evaluation and Debiasing Methods. NAACL 2018.\n\n[2] Jieyu Zhao, Tianlu Wang, Mark Yatskar, Ryan Cotterell, Vicente Ordonez, Kai-Wei Chang. Gender Bias in Contextualized Word Embeddings. NAACL 2019.\n\n[3] Ke Yang, Charles Yu, Yi R. Fung, Manling Li, Heng Ji. ADEPT: A DEbiasing PrompT Framework. AAAI 2023. (arXiv Dec 22)\n\n[4] Anne Lauscher, Tobias Lueken, and Goran Glavaš. Sustainable modular debiasing of language models. Findings of EMNLP 2021.", "questions_to_authors": "A. Based on my understanding, when fine-tuning the eight baseline methods on downstream tasks, you use the original training set, not the augmented balanced training set, right? Given this, I'm confused by the results in Table 3 that ADELE's debiasing performance is much worse than the vanilla BERT. This may indicate some problems in your implementation. Is this due to that ADELE is trained on MNLI? If so, I think you should re-train ADELE using the same training set as yours for a fair comparison. \n\nB. In Table 6, you showed that PT+SCL can significantly reduce bias, which is achieved only by the contrastive loss of a sentence itself, without any bias terms telling the model what kind of bias (i.e., gender bias) it is to reduce. This seems a bit unreasonable to me. Could you explain more on this?\n\nC. I'm also curious about how PT+CDA in Table 6 is implemented. Do you assume the augmented instance has the same label as the original instance, and optimize the classification loss on both instances?\n\nD. You mentioned in lines 110-112 & 659-660 that different prompts can be combined to mitigate intersectional biases, but this is not obvious to me. Could you present a concrete method of combining different prompts? This is not a necessary problem for this paper, but would make the proposed method much more useful in practice.", "paper_topic_and_main_contributions": "This paper aims to improve the efficiency and effectiveness of mitigating gender bias on (extrinsic) downstream tasks. The authors propose to integrate a counterfactual contrastive loss when adapting pre-trained language models to downstream tasks via deep prompt tuning. Main contributions seem to be (i) the method of debiasing during adaptation to downstream tasks (or the so-called ''debias-while-prompt-tuning'') and (ii) adopting deep prompt tuning for bias mitigation. The authors demonstrated the effectiveness of their method on three extrinsic bias benchmarks: Bias-STS-B, Bias-NLI and Bias-in-Bios.", "excitement": 3, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {"Typos_Grammar_Style_and_Presentation_Improvements": "There are concurrent works [1][2] adopting prefix tuning or prompt tuning for bias mitigation in downstream tasks, so the authors may cite them and revise the writings in lines 115-117: \" the proposed Co2 PT is the first to mitigate bias on downstream tasks through prompt tuning\".\n\n[1] Zhongbin Xie and Thomas Lukasiewicz. An Empirical Analysis of Parameter-Efficient Methods for Debiasing Pre-Trained Language Models. ACL 2023.\n\n[2] Yingji Li, Mengnan Du, Xin Wang, and Ying Wang. Prompt Tuning Pushes Farther, Contrastive Learning Pulls Closer: A Two-Stage Approach to Mitigate Social Biases. ACL 2023."}}]

Author rebuttal:

Meta-review: The paper "Co^2PT: Mitigating Bias in Pre-trained Language Models through Counterfactual Contrastive Prompt Tuning" presents work on debiasing through prompting  and testing this method based on three down-stream tasks.

Points mentioned for rejecting the paper refer to the experimental setup. 
Arguments for the paper is the clearly written and detailed information given, but also the approach taken.
Decision: Accept-Findings