An Empirical Analysis of Parameter-Efficient Methods for Debiasing Pre-Trained Language Models
Year: 2023

Abstract:
The increasingly large size of modern pretrained language models not only makes them inherit more human-like biases from the training corpora, but also makes it computationally expensive to mitigate such biases. In this paper, we investigate recent parameter-efficient methods in combination with counterfactual data augmentation (CDA) for bias mitigation. We conduct extensive experiments with prefix tuning, prompt tuning, and adapter tuning on different language models and bias types to evaluate their debiasing performance and abilities to preserve the internal knowledge of a pre-trained model. We find that the parameter-efficient methods (i) are effective in mitigating gender bias, where adapter tuning is consistently the most effective one and prompt tuning is more suitable for GPT-2 than BERT, (ii) are less effective when it comes to racial and religious bias, which may be attributed to the limitations of CDA, and (iii) can perform similarly to or sometimes better than full fine-tuning with improved time and memory efficiency, as well as maintain the internal knowledge in BERT and GPT-2, evaluated via fact retrieval and downstream fine-tuning.

heading: 1 Introduction
text: Pre-trained language models are able to encode rich linguistic and factual knowledge by learning the co-occurrence information of words in large realworld corpora (Devlin et al., 2019; Petroni et al., 2019; Raffel et al., 2020; Brown et al., 2020). Since most of these corpora are internet-based and not carefully curated, they are likely to contain unbalanced or stereotyped information for certain demographic groups. As a result, pre-trained language models are often demonstrated to inherit bias from human society and exhibit potential harms (Blodgett et al., 2020; Bender et al., 2021; May et al., 2019; Zhao et al., 2019; Sheng et al., 2019; Nangia et al., 2020; Nadeem et al., 2021). Hence, much re-
search effort has been devoted to debias pre-trained language models (Meade et al., 2022).
With the size of language models becoming incredibly large (Brown et al., 2020; Hoffmann et al., 2022; Smith et al., 2022), they are not only at higher risk of exhibiting biased behaviors (Bender et al., 2021), but also hard to debias because of prohibitive computational cost. Therefore, recent parameter-efficient methods (He et al., 2022; Ding et al., 2022) have been applied to bias mitigation, where only a small portion of the parameters are updated (Lauscher et al., 2021; Gira et al., 2022). However, these works are limited in terms of evaluation dimensions, making it unclear how different parameter-efficient methods’ performance compare to each other, whether one parameter-efficient method is effective across different types of language models, and whether they are also effective for mitigating religious and racial bias in addition to gender bias. Moreover, direct comparisons with strong post-hoc debiasing methods (Liang et al., 2020; Schick et al., 2021), as well as evaluations of bias mitigation’s impact on the language model’s internal knowledge, are often insufficient.
Given these observations, we investigate three popular parameter-efficient methods, i.e., prefix tuning (Li and Liang, 2021), prompt tuning (Lester et al., 2021), and adapter tuning (Houlsby et al., 2019), in combination with counterfactual data augmentation (CDA, Zhao et al., 2018; Zmigrod et al., 2019; Webster et al., 2020) to debias pretrained language models. We conduct extensive experiments to study the parameter-efficient methods’ performance on two types of language models (BERT (Devlin et al., 2019) for masked language models and GPT-2 (Radford et al., 2019) for autoregressive language models), three types of social biases (gender, race, and religion), and four types of performance measures (debiasing performance on CrowS-Pairs (Nangia et al., 2020) and StereoSet (Nadeem et al., 2021), language modeling per-
15730
formance on WikiText-2 (Merity et al., 2017) and StereoSet (Nadeem et al., 2021), fact retrieval performance on LAMA (Petroni et al., 2019), as well as downstream fine-tuning performance on WinoBias (Zhao et al., 2018)). We empirically compare to the performance of full fine-tuning and two posthoc debiasing methods (SentenceDebias (Liang et al., 2020) and SelfDebias (Schick et al., 2021)), aiming to comprehensively study the effectiveness of parameter-efficient methods for bias mitigation.1
Our main findings are as follows:
• The parameter-efficient methods are effective in mitigating gender bias. Within the three parameter-efficient methods, adapter tuning is consistently the most effective one for mitigating bias across different types of language models, while prompt tuning is more suitable for GPT-2 than BERT. Comparing to strong post-hoc debiasing methods, parameterefficient methods are better at preserving the language modeling ability, while still achieving a competitive and sometimes superior debiasing performance.
• The parameter-efficient methods are less effective when it comes to mitigating racial and religious bias, where the post-hoc debiasing methods could achieve a more favorable overall performance.
• The parameter-efficient methods can perform similarly to or sometimes better than full finetuning, with improved time and memory efficiency.
• The parameter-efficient methods can largely maintain the internal knowledge in both BERT and GPT-2, with the reduction in Precision@10 ranging from 0 to 6.8% across all the LAMA datasets when compared to the original pre-trained model, and with the reduction in average F1 scores less than 3.3% on the hard type-1 examples of WinoBias when compared to full fine-tuning.

heading: 2 Parameter-Efficient Methods
text: In this section, we briefly review three popular parameter-efficient methods investigated in our study: prefix tuning (Li and Liang, 2021), prompt
1The code of this paper is available at https://github. com/x-zb/pedb.
tuning (Lester et al., 2021), and adapter tuning (Pfeiffer et al., 2021). In contrast to traditional full fine-tuning where all the model parameters are updated during training, these parameter-efficient methods introduce a small number of extra tunable parameters φ on top of a frozen pre-trained language model.
Pre-trained language models usually adopt the transformer architecture (Vaswani et al., 2017) consisting of multiple stacked layers. Assume that there are Nlayer layers, and H (i) 0 ∈ RT×d is the input to the i-th layer, where T is the sequence length, and d is the model dimension. Then, H(i)0 is transformed by the following equations to obtain the output of the i-th layer H(i)5 , which is in turn adopted as the input for the (i+ 1)-th layer:
H (i) 1,h = Attn(H (i) 0 W (i) Q,h, H (i) 0 W (i) K,h, H (i) 0 W (i) V,h),
h = 1, 2, . . . , Nhead, (1)
H (i) 2 = [H (i) 1,1; . . . ;H (i) 1,Nhead ]W (i) O , (2) H (i) 3 = LayerNorm(H (i) 0 +H (i) 2 ), (3) H (i) 4 = ReLU(H (i) 3 W (i) 1 + b (i) 1 )W (i) 2 + b (i) 2 , (4) H (i) 5 = LayerNorm(H (i) 3 +H (i) 4 ). (5)
Here, Eqs. (1) and (2) constitute the multi-head attention sublayer, where W (i)Q,h, W (i) K,h, and W (i) V,h denote the projection matrix for the query, key, and value of the h-th attention head, respectively; Nhead is the number of attention heads, and H (i) 1,h ∈ RT×(d/Nhead). Eq. (4) denotes the feed-forward sublayer. [; ] denotes the concatenation operation. H
(i) j ∈ RT×d for j = 0, 2, 3, 4, 5. The input to the 1st layer is the embeddings of the input tokens H
(1) 0 = X ∈ RT×d.
Prefix tuning. Li and Liang (2021) prepend l tunable prefix vectors to the key vectors (H(i)0 W (i) K,h) and value vectors (H (i) 0 W (i) V,h) of the attention function in Eq. (1) for each layer:
H (i) 1,h = Attn(H (i) 0 W (i) Q,h, [P (i) K,h;H (i) 0 W (i) K,h],
[P (i) V,h;H (i) 0 W (i) V,h]), h = 1, 2, . . . , Nhead. (6)
Here, P (i)K,h, P (i) V,h∈Rl×(d/Nhead) denote the tunable prefix vectors, and the total tunable parameters are φ={P (i)K,h, P (i) K,h | h=1, 2, . . . , Nhead, i= 1, 2, . . . , Nlayer}.
Prompt tuning. Lester et al. (2021) prepend l tunable prompt vectors (continuous tokens) only to the input embeddings (X), and compute the activations of these prompt vectors in the subsequent layers using the pre-trained transformer’s parameters. So, the only modification is:
H (1) 0 = [P ;X] ∈ R(l+T )×d, (7)
where P ∈ Rl×d denotes the tunable prompt vectors, and φ = {P}. Adapter tuning. Houlsby et al. (2019) insert the following adapter module between the transformer’s sublayers:
H (i) j ← H (i) j + f(H (i) j W (i) down)W (i) up , (8)
where the intermediate activations H(i)j are first down-projected by W (i)down ∈ Rd×(d/r) to a lower dimension d/r, and then up-projected back by W (i) up ∈ R(d/r)×d to the model dimension d. The adapter also contains a non-linear function f and a residual connection. The hyperparameter r is called the reduction factor, which determines the bottleneck dimension d/r and controls the trade-off between parameter efficiency and model capacity.
In our implementation, we adopt Pfeiffer et al. (2021)’s setting where only a single adapter is inserted after the feed-forward sublayer, since it is found to be the optimal setting among other alternatives (Pfeiffer et al., 2021). Thus, all the tunable parameters are φ = {W (i)down,W (i) up | i = 1, 2, . . . , Nlayer}.2

heading: 3 Parameter-Efficient Debiasing through Counterfactual Data Augmentation
text: We adopt counterfactual data augmentation (CDA, Zhao et al., 2018; Zmigrod et al., 2019; Webster et al., 2020) as our debiasing method to work together with parameter-efficient tuning methods. Since the encoded biases in pre-trained language models originate from the unbalanced training corpora, it is natural to mitigate these biases by rebalancing the training corpora. For example, when we want to mitigate gender bias between the male and female demographic group and encounter the training sentence “He is a doctor.”, CDA would substitute the bias attribute word “He” with its
2Pfeiffer et al. (2021) also insert an additional “add & layer norm” sublayer before the adapter module, so the actual number of tunable parameters is a bit larger.
Algorithm 1 Counterfactual Data Augmentation Input: original corpus D0, # demographic groups
N , # samples S(≤ N − 1), bias attribute word list {(w(i)1 , . . . , w (i) N )}Mi=1 Output: augmented corpus D1 1: D1 ← ∅ 2: for text sequence x ∈ D0 do 3: Identify the number of demographic groups
n(≤ N) contained in x 4: if n > 0 then 5: Generate all the permutations of N de-
mographic groups considered n demographic groups at a time: Π = {πj}P n N
j=1, where πj = (g1, . . . , gn), {g1, . . . , gn}⊂ {1, . . . , N}
6: if n = N and (1, 2, . . . , N) ∈ Π then 7: Π← Π \ {(1, 2, . . . , N)} 8: end if 9: Sample w/o replacement S permutations
ΠS = {πs}Ss=1 from Π 10: for πs ∈ ΠS do 11: xs←Substitute all bias attribute words
w (i) k contained in x with w (i) πs[k]
12: D1 ← D1 ∪ {xs} 13: end for 14: D1 ← D1 ∪ {x} 15: end if 16: end for
counterpart “She” to obtain an additional training sentence “She is a doctor.”, so that both gender groups would have equal association with the gender-neutral word “doctor”. Once we have a list of bias attribute words like {(he, she), (man, woman), (husband, wife), . . . }, we could retrieve all the occurrences of these bias attribute words in the training corpus, and substitute all of them with their counterparts.
For religious and racial bias where more than two demographic groups are considered, we need to maintain two key properties: (i) we should guarantee consistency, i.e., we should avoid the case where some occurrences of the bias attribute words in group A are substituted with those in group B, while the other occurrences of (possibly different) bias attribute words in group A are substituted with those in group C, and (ii) we should avoid collisions, i.e., we should avoid the case where both groups A and B are substituted with group C. To this end, we should not consider each group inde-
pendently and adopt random substitution. Rather, we should substitute according to permutations of all the occurred demographic groups in a sentence. Our complete CDA method is formally summarized in Algorithm 1.
Note that in Algorithm 1, for convenience, we propose to sample a fixed number (S) of substitutions for each sentence. This is because the number of possible substitutions (PnN − 1) for each sentence may vary when the number of occurred demographic groups (n) in the sentence varies. In practice, we adopt N = 3 and S = 2 for religious and racial bias.
Finally, the parameter-efficient debiasing framework works as follows: we first use Algorithm 1 to augment an original corpus D0 and obtain the debiasing corpus D1; next, we use the parameterefficient tuning methods from Section 2 to solve the following optimization problem:
min φ L(θ0, φ;D1), (9)
where L is either the masked language modeling loss (Devlin et al., 2019) or causal language modeling loss (Radford et al., 2019), θ0 denotes the frozen parameters in the pre-trained language model, and φ denotes the tunable parameters defined in Section 2.

heading: 4 Conceptual Comparisons with Existing Debiasing Methods
text: Most existing debiasing methods are trainingbased, where they introduce a specific debiasing loss to fine-tune a pre-trained model on certain balanced debiasing corpora (Kaneko and Bollegala, 2021; Garimella et al., 2021; Ravfogel et al., 2020; Cheng et al., 2021; Guo et al., 2022). These methods are, in general, orthogonal to our parameterefficient debiasing framework in that we could substitute the (masked) language modeling loss in Eq. (9) with their specific debiasing loss. In this paper, we only focus on the simple language modeling loss, and leave other kinds of debiasing loss for future work.
Another important line of debiasing methods applies post-hoc mathematical operations on the frozen representations of a language model, such as SentenceDebias (Liang et al., 2020) and SelfDebias (Schick et al., 2021). We briefly review these methods below and make empirical comparisons to parameter-efficient debiasing methods in Section 5.
SentenceDebias. Liang et al. (2020) assume that there is a linear subspace that can capture demographic information in the embedding space, thus trying to identify and remove the demographic information via linear algebra operations. Specifically, they first leverage a procedure similar to CDA to extract and augment sentences containing bias attribute words from a source corpus. Then, they encode the sentences to embeddings with a pre-trained language model, and obtain a set of difference vectors between the embeddings of sentences in different demographic groups. Next, they perform principle component analysis on the set of difference vectors, and use the first K components to expand a bias subspace. Once the bias subspace is identified, we could debias a new sentence embedding by subtracting its projection on the bias subspace.
SelfDebias. Schick et al. (2021) assume that a pre-trained language model has a self-diagnosis ability, which can be used to adjust the output probabilities over the vocabulary during language generation. Specifically, SelfDebias relies on handcrafted descriptions for each type of bias. It first puts the bias description and the currently generated sentence into a self-diagnosis template, which encourages the language model to generate biased words for the next time step. Then, the probabilities of these detected biased words are scaled down in the actual generation process.
Although no training is needed for these posthoc debiasing methods, their strong assumptions about bias may harm the language modeling ability of a language model. On the contrary, CDA-based parameter-efficient methods adhere to the original language modeling loss without additional assumptions, which may largely reserve the language modeling ability. Another advantage of CDA-based parameter-efficient methods is that nearly no additional computation is required during inference.

heading: 5 Experiments on Bias Mitigation

heading: 5.1 Experimental Setup
text: Datasets. To measure gender, religious, and racial bias in pre-trained language models, we adopt two crowd-sourced datasets: CrowSPairs (Nangia et al., 2020) and StereoSet (Nadeem et al., 2021). CrowS-Pairs consists of pairs of contrasting sentences, where one is more stereotyping than the other. Its gender, religious, and
racial subsets contain 262, 105, and 516 examples, respectively. For StereoSet, we adopt its intrasentence test, where each example consists of a context sentence and three candidate completions corresponding to stereotypical, anti-stereotypical, and unrelated associations, respectively. We again only adopt the gender, religious, and racial subsets, whose sizes are 1026, 623, and 3996, respectively.
Evaluation Metrics. Our evaluation protocol follows Meade et al. (2022). We adopt the “stereotype score”, defined as the percentage of examples for which the language model favors the stereotypical association (or the stereotyping sentence) to the anti-stereotypical association (or the less stereotyping sentence), as the measure of bias. An ideal model that is free of the considered bias should achieve a stereotype score of 50%. To measure the language modeling ability, we adopt the first 10% of WikiText-2 (Merity et al., 2017) to compute the perplexity (for autoregressive language models) or pseudo-perplexity (Salazar et al., 2020, for masked language models). We also compute the “language modeling (LM) score” (Nadeem et al., 2021) on all the bias subsets of StereoSet as our second measure of language modeling ability.
Training Details. We choose to debias BERT (Devlin et al., 2019) and GPT-2 (Radford et al., 2019), which represent masked language models and autoregressive language models, respectively. Our implementation is based on the Hugging Face Transformers (Wolf et al., 2020) and Adapter Hub (Pfeiffer et al., 2020), and the adopted checkpoints are bert-base-uncased (109’514’298 parameters) and gpt2 (124’439’808 parameters). We adopt the English Wikipedia as our original debiasing corpus3, and counterfactually augment it using Algorithm 1. The adopted bias attribute words for each type of bias are listed in Appendix A. Next, we randomly down-sample 20% of the augmented Wikipedia as our debiasing corpus. All the CDAbased debiasing methods are trained for two epochs on one TITAN RTX GPU with 24 GB memory. We select optimal training hyperparameters according to the language modeling loss on a validation set (we use 5% of the augmented debiasing corpus for validation), since the language modeling loss on a balanced dataset is a reasonable proxy for both debiasing performance and language model-
3We also investigate the effect of different debiasing corpora for GPT-2. See Appendix C for details.
ing ability. We select hyperparameters using the default seed of 42, and re-train the models for four additional times with different random seeds, to account for CrowS-Pairs and StereoSet’s sensitivity to pre-training seeds (Aribandi et al., 2021). More details are in Appendix B.
Baselines. We compare the parameter-efficient methods to full fine-tuning, where all the parameters of a language model are tuned, For post-hoc debiasing methods, we compare to SentenceDebias (Liang et al., 2020) and Self-Debias (Schick et al., 2021), as described in Section 4.

heading: 5.2 Mitigating Gender Bias
text: For experiments on mitigating gender bias, we adopt a default reduction factor of r = 48 in adapter tuning, leading to 304’320 tunable parameters, which are less than 0.3% of all the parameters in BERT (109’514’298) or GPT-2 (124’439’808). For prefix tuning, we adopt a prefix length of l = 16 to obtain a similar amount of tunable parameters (294’912) to adapter tuning. Obtaining a similar amount of tunable parameters for prompt tuning would require an exceptionally large prompt length, even approaching the maximum acceptable sequence length of the pre-trained language models. Therefore, we only set the prompt length l = 16 (which corresponds to 12’288 tunable parameters) to compare with prefix tuning under the same number of prepending tokens. Evaluation results are shown in Table 1.4
In general, the parameter-efficient methods are effective in reducing stereotype scores, and the reductions are statistically significant (p < 0.05) under a permutation test (Ernst, 2004).
Among the three parameter-efficient methods, adapter tuning achieves the best debiasing performance on both CrowS-Pairs and StereoSet, for both BERT and GPT-2. This demonstrates adapter tuning to be a reliable parameter-efficient method for bias mitigation across different types of language models. Note that our results are also consistent with He et al. (2022)’s finding that modifying transformer representations at the feed-forward sublayers (adapter tuning) is more effective than modifying those at the multi-head attention sublayers (prefix tuning).
4Since SelfDebias preserves 32 tokens for its prefix templates, when measuring perplexity for all the methods in Table 1, the input sequence length is set to 480 (512-32) for BERT and 992 (1024-32) for GPT-2.
Prompt tuning is more effective on GPT-2 than BERT. Prompt tuning is ineffective in reducing the CrowS-Pairs stereotype score on BERT, but can successfully reduce it on GPT-2, where it even achieves a similar debiasing performance to prefix tuning. This is remarkable given that prompt tuning has much less tunable parameters than prefix tuning. This is also consistent with prompt tuning being more effective when T5 (Raffel et al., 2020) is continuously pre-trained with an autoregressive language modeling loss (Lester et al., 2021).
Comparing to post-hoc debiasing methods, parameter-efficient methods are better at maintaining the language modeling ability while achieving a similar debiasing performance. Note that post-hoc debiasing methods sometimes significantly worsen the language modeling ability, e.g., a perplexity of 7.070 for SelfDebias on BERT, a perplexity of 56.891, and a LM score of 87.43 for SentenceDebias on GPT-2. Since a completely random language model would achieve the perfect stereotype score (50), but is useless as a language model (Nadeem et al., 2021), the degraded language modeling ability of the post-hoc debiasing methods undermines their true effectiveness for bias mitigation. On the contrary, parameterefficient methods keep the language modeling loss during CDA training, which helps to preserve or even enhance the language modeling ability.
Comparing to full fine-tuning, parameter-efficient methods can achieve a better or similar
performance with improved time and memory efficiency. Since full fine-tuning updates all the parameters of the language model, it is computationally expensive and prone to be overfitting. When debiasing BERT, full fine-tuning consumes around 19 GB memory, while the parameter-efficient methods consume 12~17 GB memory. Training on the debiasing corpus for full fine-tuning lasts around 6 hours, while that for the parameter-efficient methods lasts 4~5 hours. For GPT-2, full fine-tuning consumes around 18 GB memory with the training time being around 7 hours, while the parameterefficient methods consume 15~16 GB memory and 5 hours of training time.

heading: 5.3 Mitigating Racial and Religious Bias
text: When mitigating racial and religious bias, we find that a prefix length of l = 16 (or, equivalently, a reduction factor of r = 48 for adapter tuning) is no longer sufficient for successful debiasing. Therefore, we search l in a broader range of {48, 96, 192, 384} (and, correspondingly, r in {16, 8, 4, 2}). The results are shown in Table 2.
In general, the parameter-efficient methods are less effective when it comes to racial and religious bias. Even the previously strongest method, adapter tuning, is ineffective in many cases such as debiasing BERT on the religion subsets of CrowSPairs and StereoSet, and GPT-2 on the race subset of CrowS-Pairs. For GPT-2, prompt tuning is consistently effective on the race subsets of both
CrowS-Pairs and StereoSet, but cannot obtain a similar performance on StereoSet’s religion subset. In three out of the eight debiasing cases, none of the parameter-efficient methods could reduce the stereotype score in a statistically significant way.
Moreover, SelfDebias exhibits a superior debiasing performance over the parameter-efficient methods, and its language modeling ability does not severely degenerate as in mitigating gender bias. Indeed, when we calculate the icat score (Nadeem et al., 2021), defined as lms ∗ min(ss, 100 − ss)/50 (lms stands for the LM score, and ss stands for the stereotype score on StereoSet), to integrate the debiasing performance and language modeling ability, we can clearly see a better overall perfor-
5For the race-debiased models, we set the input sequence length to 320 for BERT and 640 for GPT-2; for the religiondebiased models, we set the input sequence length to 128 for BERT and 640 for GPT-2.
mance of SelfDebias over adapter tuning (e.g., on StereoSet’s religion subset, the icat score of SelfDebias and adapter tuning is 72.00 vs. 69.37 for BERT, and 70.68 vs. 68.55 for GPT-2).
The less successful performance of parameterefficient methods may be attributed to some limitations of the CDA debiasing method. The bias attribute word lists for race and religion are shorter and contain more noise (i.e., words with multiple or ambiguous meanings) than that for gender, which may undermine the diversity and quality of the augmented training corpus. On the contrary, SelfDebias relies on bias descriptions that contain less noise and could generalize with the help of the language model’s own knowledge. Given this analysis, future work could explore how to adopt parameterefficient methods to debiasing techniques other than CDA to overcome these limitations.

heading: 6 Impact on Internal Knowledge

heading: 6.1 Fact Retrieval
text: To investigate the impact of bias mitigation on the factual knowledge encoded in pre-trained language models, we take the gender-debiased models from Section 5.2 and evaluate them on the four LAMA datasets (Petroni et al., 2019).6 The results are shown in Table 3. We report the results from a single run (with the default seed 42) to save computation in Table 3 and 4.
The parameter-efficient methods can largely maintain the factual knowledge of a language model, with the reduction in Precision@10 ranging from 0 to 6.8% across all the datasets and pre-trained models. Surprisingly, for BERT on SQuAD and GPT-2 on all the four datasets, quite a number of the results are actually improved. We attribute these improvements to the fact that Wikipedia contains a lot of factual knowledge, and continuously training on it can enhance the internal knowledge of a language model.
Comparing the performance between full finetuning and parameter-efficient tuning, we find that the former performs best on SQuAD with BERT and Google-RE with GPT-2, while the latter performs better in the rest of the settings. In general, the performance gaps are marginal.

heading: 6.2 Downstream Fine-Tuning
text: We further investigate the impact of bias mitigation on knowledge transfer to downstream tasks via fine-tuning. Since neural network models suffer from catastrophic forgetting (French, 1999), a debiased model may forget the encoded knowledge in the original language model, and conversely a finetuned model may forget the debiasing knowledge in the debiased model. Therefore, it is important to adopt an evaluation dataset that can simultaneously evaluate downstream task performance and debiasing performance. We choose the coreference resolution dataset WinoBias (Zhao et al., 2018) to fulfill the above requirements.
We append each example from WinoBias (e.g., The physician hired the secretary because he was overwhelmed with clients.) with the suffix “{Pronoun} refers to the {Candidate}.” ({Pronoun} is
6Instead of using the intersectional vocabulary of several pre-trained models, as in Petroni et al. (2019), we adopt each pre-trained model’s full vocabulary, since we do not aim to compare the performance across different pre-trained models.
“He” in this example), and then measure the probability of the model completing the sentence with different candidates (“physician” and “secretary” in this example) to determine the coreference result. We adopt both the type-1 and type-2 test sets of WinoBias, where type-1 examples are harder to resolve as they contain no syntactic cues. We adopt WinoBias’ dev set to fine-tune an original pre-trained language model using either full finetuning or parameter-efficient tuning.7 The results are shown in Table 4.
On type-1 examples, adapter tuning achieves a comparable performance to full fine-tuning for both BERT and GPT-2, with the reduction in average F1 scores less than 3.3%. On BERT, adapter tuning achieves a much better debiasing performance (Diff= 0.51) than full fine-tuning, while on GPT-2 it is slightly more biased. Nevertheless, both of them can be considered effective simultaneously on the coreference resolution task and debiasing task. The performance gap between full fine-tuning and prefix/prompt tuning is more significant, but the latter can still achieve a nearly perfect performance on the easier type-2 examples.

heading: 7 Conclusion
text: In this study, we investigated the performance of prefix tuning, prompt tuning, and adapter tuning on mitigating social bias and preserving the linguistic and factual knowledge for two types of pre-trained language models. Our results demonstrated the effectiveness and efficacy of parameter-efficient methods in combination with CDA, and also revealed their performance limitations by comparing to post-hoc debiasing methods. We hope that our study can make it more accessible for others to debias pre-trained language models with reduced computational requirements, and contribute to fair and inclusive NLP.

heading: 8 Limitations
text: Due to the restrictions of the adopted benchmarks and resources, our evaluation bears the following limitations: (i) We only focus on social biases in the English language and North American cultures. This is due to the fact that both CrowSPairs and StereoSet are generated by crowd workers from North America. Future work can extend our analysis to other languages and cultures with
7See Appendix B for more details.
BERT/GPT-2 are highlighted in green .
the corresponding resources such as the French CrowS-Pairs (Névéol et al., 2022) and multilingual WEAT (Lauscher and Glavaš, 2019). (ii) Our evaluation has a limited coverage over different kinds of harms according to Blodgett et al. (2020). CrowS-Pairs, StereoSet, and WinoBias all focus on stereotyping, a kind of representational harm, while others like allocational harms are untouched. Developing methods to measure these harms generally requires in-depth interactions between technologists and customers. Blodgett et al. (2021) also point out several conceputalization and operationalization pitfalls in the above three bias benchmarks, which limits the validity of the results evaluated on them. (iii) Due to the incomplete bias attribute word lists, our CDA-based debiasing method is by no means fair enough to cover all the minority groups (e.g., groups with non-binary genders). Therefore the current debiasing method in this paper can only be used to mitigate bias among the demographic groups mentioned in Appendix A. We
recommend more complete resources such as the gender-inclusive word list in (Cao and Daumé III, 2021) for real-world scenarios.

heading: Acknowledgements
text: This work was supported by the Alan Turing Institute under the EPSRC grant EP/N510129/1, by the AXA Research Fund, and by the EU TAILOR grant 952215. We also acknowledge the use of Oxford’s ARC facility, of the EPSRC-funded Tier 2 facilities JADE (EP/P020275/1), and of GPU computing support by Scan Computers International Ltd.

heading: A Bias Attribute Words
text: We adopt the same bias attribute words as Meade et al. (2022), where the list for gender is from Zhao et al. (2018) and that for religion is from Liang et al. (2020).
Gender: (actor, actress), (actors, actresses), (airman, airwoman), (uncle, aunt), (uncles, aunts), (boy, girl), (boys, girls), (groom, bride), (grooms, brides), (brother, sister), (brothers, sisters), (businessman, businesswoman), (businessmen, businesswomen), (chairman, chairwoman), (chairmen, chairwomen), (dude, chick), (dudes, chicks), (dad, mom), (dads, moms), (daddy, mommy), (daddies, mommies), (son, daughter), (sons, daughters), (father, mother), (fathers, mothers), (male, female), (males, females), (guy, gal), (guys, gals), (gentleman, lady), (gentlemen, ladies), (grandson, granddaughter), (grandsons, granddaughters), (guy, girl), (guys, girls), (he, she), (himself, herself ), (him, her), (his, her), (husband, wife), (husbands, wives), (king, queen), (kings, queens), (lord, lady), (lords, ladies), (sir, maam), (man, woman), (men, women), (sir, miss), (mr., mrs.), (mr., ms.), (policeman, policewoman), (prince, princess), (princes, princesses), (spokesman, spokeswoman), (spokesmen, spokeswomen)
Religion: (jewish, christian, muslim), (jews, christians, muslims), (torah, bible, quran), (synagogue, church, mosque), (rabbi, priest, imam), (judaism, christianity, islam)
Race: (black, caucasian, asian), (african, caucasian, asian), (black, white, asian), (africa, america, asia), (africa, america, china), (africa, europe, asia)

heading: B Additional Training Details
text: For all the experiments on parameter-efficient tuning methods and full fine-tuning, we use the default settings of the AdamW optimizer (Loshchilov and Hutter, 2019) and a linear learning rate scheduler from the Hugging Face library.
For the debiasing experiments trained on Wikipedia, we fix the number of training epochs to 2 and greedily search initial learning rate from {5e-1, 5e-2, 5e-3, 5e-4, 5e-5, 5e-6, 5e-7} according to the language modeling loss on the validation set (we use 5% of the augmented debiasing corpus for validation). For experiments trained on WinoBias, we greedily search training epochs from {10, 20,
30, 50, 100, 200} and initial learning rate from {5e-1, 5e-2, 5e-3, 5e-4, 5e-5, 5e-6, 5e-7} according to the Avg F1 score on type-1 examples in the validation set (we use 5% of the training set for validation). The hyperparameter values to reproduce our results in Sections 5 and 6 are in Table 5.
Implementations of SentenceDebias and SelfDebias are based on Meade et al. (2022)’s, where we also follow their default parameter settings.

heading: C Effect of the Debiasing Corpus for GPT-2
text: For consistency, we adopt the same debiasing corpus (the English Wikipedia) for both BERT and GPT-2 in Section 5.2, where each training example consists of a single sentence (the average sentence length in our corpus is around 107 tokens). However, this setting is different from the original pre-training settings of GPT-2 (Radford et al., 2019) in terms of example length and data source. Therefore, we further investigate debiasing GPT-2 on two other debiasing corpora: for one corpus, we still use Wikipedia but concatenate all the sentences into a long sequence and truncate it into examples of 1024 tokens; for the other corpus, we use 1% of OpenWebText8, which is a public replicate of GPT-2’s private pre-training corpus, and truncate it into examples of 1024 tokens. The results are shown in Table 6.9
Comparing the results on Wikipedia, with single sentence and example length 1024 tokens, in Table 6, we can see that the former is consistently better. This indicates that these methods favor shorter example lengths. We conjecture that this is due to GPT-2’s language modeling objective being an average over all the tokens in an example. Therefore, the counterfactual token’s signal will be less significant if it is close to the end of a long example.
Comparing the last two blocks, we can see that the results from the debiasing methods trained on OpenWebText are superior to those trained on Wikipedia under the same example length of 1024.
8https://skylion007.github.io/ OpenWebTextCorpus/
9We report the results from a single run (with the default seed 42) to save computation.
This indicates that using a similar data source to the original pre-training corpus is beneficial. For full fine-tuning, this can improve perplexity to 38.252. For the parameter-efficient methods, the improvements are more significant on stereotype scores. Given that parameter-efficient methods’ model capacity is limited, if we allocate some capacity for adapting to new data sources, it is reasonable for the debiasing performance to be negatively affected.
ACL 2023 Responsible NLP Checklist

heading: A For every submission:

heading: 3 A1. Did you describe the limitations of your work?
text: section 8 "Limitations"

heading: 3 A2. Did you discuss any potential risks of your work?
text: section 8 "Limitations"

heading: 3 A3. Do the abstract and introduction summarize the paper’s main claims?
text: in the abstract and section 1
7 A4. Have you used AI writing assistants when working on this paper? Left blank.
B 3 Did you use or create scientific artifacts? section 5, 6 and Appendix A, B, C

heading: 3 B1. Did you cite the creators of artifacts you used?
text: section 5, 6 and Appendix A, B, C
7 B2. Did you discuss the license or terms for use and / or distribution of any artifacts? we adhere to each artifact’s original licnese and terms.

heading: 3 B3. Did you discuss if your use of existing artifact(s) was consistent with their intended use, provided
text: that it was specified? For the artifacts you create, do you specify intended use and whether that is compatible with the original access conditions (in particular, derivatives of data accessed for research purposes should not be used outside of research contexts)? section 8 "Limitations"
B4. Did you discuss the steps taken to check whether the data that was collected / used contains any information that names or uniquely identifies individual people or offensive content, and the steps taken to protect / anonymize it? Not applicable. We follow previous work and the data creator’s practices.

heading: 3 B5. Did you provide documentation of the artifacts, e.g., coverage of domains, languages, and
text: linguistic phenomena, demographic groups represented, etc.? section 8 "Limitations", Appendix A
3 B6. Did you report relevant statistics like the number of examples, details of train / test / dev splits, etc. for the data that you used / created? Even for commonly-used benchmark datasets, include the number of examples in train / validation / test splits, as these provide necessary context for a reader to understand experimental results. For example, small differences in accuracy on large test sets may be significant, while on small test sets they may not be. section 5, 6 and Appendix B
C 3 Did you run computational experiments? section 5, 6

heading: 3 C1. Did you report the number of parameters in the models used, the total computational budget
text: (e.g., GPU hours), and computing infrastructure used? section 5, 6
The Responsible NLP Checklist used at ACL 2023 is adopted from NAACL 2022, with the addition of a question on AI writing assistance.

heading: 3 C2. Did you discuss the experimental setup, including hyperparameter search and best-found
text: hyperparameter values? section 5, 6 and Appendix B

heading: 3 C3. Did you report descriptive statistics about your results (e.g., error bars around results, summary
text: statistics from sets of experiments), and is it transparent whether you are reporting the max, mean, etc. or just a single run? section 5, 6

heading: 3 C4. If you used existing packages (e.g., for preprocessing, for normalization, or for evaluation), did
text: you report the implementation, model, and parameter settings used (e.g., NLTK, Spacy, ROUGE, etc.)? section 5 and Appendix B
D 7 Did you use human annotators (e.g., crowdworkers) or research with human participants? Left blank.
D1. Did you report the full text of instructions given to participants, including e.g., screenshots, disclaimers of any risks to participants or annotators, etc.? No response.
D2. Did you report information about how you recruited (e.g., crowdsourcing platform, students) and paid participants, and discuss if such payment is adequate given the participants’ demographic (e.g., country of residence)? No response.
D3. Did you discuss whether and how consent was obtained from people whose data you’re using/curating? For example, if you collected data via crowdsourcing, did your instructions to crowdworkers explain how the data would be used? No response.
D4. Was the data collection protocol approved (or determined exempt) by an ethics review board? No response.
D5. Did you report the basic demographic and geographic characteristics of the annotator population that is the source of the data? No response.

heading: Future Work
text: Future work can extend our analysis to other languages and cultures with
7See Appendix B for more details. BERT/GPT-2 are highlighted in green. the corresponding resources such as the French CrowS-Pairs (Névéol et al. , 2022) and multilingual WEAT (Lauscher and Glavaš, 2019). (ii) Our evaluation has a limited coverage over different kinds of harms according to Blodgett et al. (2020). CrowS-Pairs, StereoSet, and WinoBias all focus on stereotyping, a kind of representational harm, while others like allocational harms are untouched. Developing methods to measure these harms generally requires in-depth interactions between technologists and customers. Blodgett et al. (2021) also point out several conceputalization and operationalization pitfalls in the above three bias benchmarks, which limits the validity of the results evaluated on them. (iii) Due to the incomplete bias attribute word lists, our CDA-based debiasing method is by no means fair enough to cover all the minority groups (e. g. , groups with non-binary genders). Therefore the current debiasing method in this paper can only be used to mitigate bias among the demographic groups mentioned in Appendix A. We
recommend more complete resources such as the gender-inclusive word list in (Cao and Daumé III, 2021) for real-world scenarios