Gender-tuning: Empowering Fine-tuning for Debiasing Pre-trained Language Models
Year: 2023

Abstract:
Recent studies have revealed that the widelyused Pre-trained Language Models (PLMs) propagate societal biases from the large unmoderated pre-training corpora. Existing solutions require debiasing training processes and datasets for debiasing, which are resourceintensive and costly. Furthermore, these methods hurt the PLMs’ performance on downstream tasks. In this study, we propose Gendertuning, which debiases the PLMs through finetuning on downstream tasks’ datasets. For this aim, Gender-tuning integrates Masked Language Modeling (MLM) training objectives into fine-tuning’s training process. Comprehensive experiments show that Gender-tuning outperforms the state-of-the-art baselines in terms of average gender bias scores in PLMs while improving PLMs’ performance on downstream tasks solely using the downstream tasks’ dataset. Also, Gender-tuning is a deployable debiasing tool for any PLM that works with original fine-tuning.

heading: 1 Introduction
text: Pre-trained Language Models (PLMs) have achieved state-of-the-art performance across various tasks in natural language processing (Devlin et al., 2019; Liu et al., 2019; Clark et al., 2020). One of the crucial reasons for this success is pretraining on large-scale corpora, which is collected from unmoderated sources such as the internet. Prior studies (Caliskan et al., 2017; Zhao et al., 2018; May et al., 2019; Kurita et al., 2019; Gehman et al., 2020) have shown that PLMs capture a significant amount of social biases existing in the pretraining corpus. For instance, they showed that the PLMs learned that the word "he" is closer to the word "engineer" because of the frequent cooccurrence of this combination in the training corpora, which is known as social gender biases. Since PLMs are increasingly deployed in real-world sce-
narios, there is a serious concern that they propagate discriminative prediction and unfairness.
Several solutions for mitigating the social biases have been proposed, including: using banned word lists (Raffel et al., 2020), building deliberated training datasets (Bender et al., 2021), balancing the biased and unbiased terms in the training dataset (Dixon et al., 2018; Bordia and Bowman, 2019), debiasing embedding spaces (Liang et al., 2020; Cheng et al., 2021), and self-debiasing in text generation (Schick et al., 2021). Although all these solutions have shown different levels of success, they tend to limit the PLMs’ ability (Meade et al., 2022). For example, the banned words solution prevent gaining knowledge of topics related to banned words. Also, some of them hurt the PLMs’ performance on downstream tasks. Furthermore, dataset curation and pre-training are two resourceintensive tasks needed for most of the above solutions (Schick et al., 2021).
In this study, we address the challenges mentioned above by proposing an effective approach named Gender-tuning for debiasing the PLMs through fine-tuning on downstream tasks’ datasets. For this goal, Gender-tuning perturbs the training examples by first finding the gender-words in the training examples based on a given gender-word list. Then Gender-tuning replaces them with the new words to interrupt the association between the gender-words and other words in the training examples (Table 1). Finally, Gender-tuning classifies the examples with the replaced words according to the original training examples’ ground-truth labels to compute a joint loss from perturbation and classification for training the Gender-tuning. The key advantage of our method is integrating the debiasing process into the fine-tuning that allows the debiasing and fine-tuning to perform simultaneously. Thus, Gender-tuning does not require separate pre-training or additional training data. Also, this integration makes Gender-tuning a plug-and-
5448
play debiasing tool for any PLMs that works with original fine-tuning.
To evaluate the effectiveness of our proposed method, we conducted comprehensive experiments following two state-of-the-art debiasing baselines: SENT_DEBIAS (Sent-D) (Liang et al., 2020) and FairFil (FairF) (Cheng et al., 2021). The results show that Gender-tuning outperforms both baselines in terms of the average gender-bias scores in the BERT model while improving its performance on the downstream tasks. In addition, we reported the performance of Gender-tuning applied to the RoBERTa that shows considerable improvement. Finally, our ablation studies demonstrate that all components of Gender-tuning, including two training phases and joint loss, play an essential role in achieving success.

heading: 2 Methodology
text: We propose a novel debiasing approach, named Gender-tuning (Figure 1), that performs the debiasing process and fine-tuning simultaneously on the downstream tasks’ dataset. For this aim, Gender-tuning integrates two training objectives: 1) Masked Language Modeling (MLM) training objective for gender-word perturbation and 2) Finetuning for classification. In each training batch, Gender-tuning works as follows: Gender-tuning uses MLM to perturb training examples by masking the existing gender-word(s)1. The MLM training objective is to predict masked token(s) with a mean cross-entropy loss that we denote as perturbation-loss (Lperturb). The training examples with predicted tokens, called genderperturbed examples (Table 1), are fed into fine-
1We use the feminine and masculine word lists created by (Zhao et al., 2018)
tuning to be classified according to the original examples’ ground-truth label (y). Then pθ(y′ = y|x̂) is the fine-tuning classification function to predict the gender-perturbed example’s label (y′) based on the gender-perturbed example (x̂) to compute the fine-tuning loss (Lfine−tuning), where θ is the PLM’s parameters for the fine-tuning. A weighted aggregation of the perturbation loss and fine-tuning loss, called joint-loss (Ljoint), is used for training the Gender-tuning as follows:
Ljoint = α Lperturb + (1− α)Lfine−tuning (1)
where α is a weighting factor that is employed to adjust the contribution of the two training losses in computing the joint-loss.
The Gender-tuning training objective is to minimize joint-loss to ensure that the label of the perturbed example is the same as the label of the original training example. In the following, we present how joint-loss impacts the training process of Gender-tuning in each training batch:
Suppose the MLM predicts an incorrect token. For instance, the example: "the film affirms the power of the [actress]" changes to "the film affirms the power of the [trauma]". In this example, the predicted word [trauma] is a non-related genderword that raises perturbation-loss value (Lperturb > 0). In this case, even if fine-tuning classifies the perturbed example correctly, joint-loss is still big enough to force Gender-tuning to continue training.
Also, suppose Gender-tuning creates social gender bias through gender perturbation. For instance, the example: "angry black [actor]" changes to "angry black [woman]" that "woman" and "actor" are not close semantically that raises perturbation-loss
value (Lperturb > 0). In this case, the output of the fine-tuning might be correct (Lfine−tuning ≈ 0) due to the PLMs’ learned biases ("angry black woman" is a known gender/race bias). However, due to the big value of perturbation-loss, the joinloss is big enough to override fine-tuning results and forces Gender-tuning to continue training.
Moreover, we observed that sometimes example perturbation changes the concept/label of training examples. For instance, the input: "[He] is an excellent [actor] (label: positive)" changes to "[She] is a wonderful [murderer] (label: positive)", and fine-tuning classification output is correct (Lfine−tuning ≈ 0). In this example, the predicted word [murderer] is conceptually far from gender-related words [actor]. So, perturbation loss becomes significant, which creates a big value for joint-loss to force Gender-tuning to continue training. Finally, we found examples that MLM replaces the gender-word with the [UNK] token. In these examples, the perturbation-loss is close to zero (Lperturb ≈ 0) and the output of the finetuning classifier is incorrect (Lfine−tuning > 0). In this case, the joint-loss is big enough to continue training and provide a new chance for MLM to predict a meaningful token instead of a [UNK]. More analysis of our perturbation strategy can be found in Section 4.1 and Table 3.

heading: 3 Experimental Setup
text: To evaluate our proposed method, we conduct experiments by following the evaluation process of the two state-of-the-art baselines (Sent-D and FairF) such as the bias evaluation metric (SEAT), applied PLMs, and downstream tasks’ datasets.2
We report the SEAT effect size (e-size), average absolute e-size, and classification accuracy on downstream tasks for three different setups: 1) Origin: fine-tuning the PLMs on the downstream task datasets using huggingface transformers code (Wolf et al., 2020). 2) Gender-tuningrandom: instead of replacing the gender-words in an training example, Gender-tuningrandom replaces a certain percentage of an input tokens randomly (5% of each input sequence). 3) Gender-tuning: the proposed method. We used the same hyperparameter for all three setups for a fair comparison.

heading: 4 Results and Discussion
text: Table 2 illustrates SEAT absolute effect size (esize) (lower is better) on sentence templates of Terms/Names under different gender domains provided by (Caliskan et al., 2017), average absolute e-size (lower is better), and classification accuracy on downstream tasks (higher is better)
2Details of the baselines, bias evaluation metric, PLMs, datasets, and hyperparameters are presented in Appendix A.
for three experiment setups (Section 3) and two state-of-the-art baselines. The results show that Gender-tuning outperforms the baselines regarding the average absolute effect size for both PLMs on all datasets. Also, in contrast with the baselines, Gender-tuning improves the accuracy of both PLMs on all downstream tasks. It shows that the proposed method preserves the useful semantic information of the training data after debiasing. The Gender-tuningrandom results show an inconsistent effect on the bias scores. Although Gendertuningrandom improves the PLMs’ accuracy on the downstream tasks, it significantly magnifies the bias score in the BERT model on SST-2 and CoLA. Also, it slightly reduces the average bias score in the RoBERTa on all datasets and in BERT on the QNLI.

heading: 4.1 Perturbation Analysis
text: The PLMs achieved state-of-the-art performance on the downstream tasks datasets by applying the MLM for the example perturbation in pre-training phase. Thus we hypothesize that the MLM can generate realistic gender-perturbed examples that can considerably modify the gender relation between the input tokens without affecting the label. However, there is a concern that the pre-trained MLM transfers the gender bias through the perturbation process.
To address this concern, we investigate the predicted tokens that the pre-trained MLM replaces with the gender-words. We randomly select 300 examples from training dataset including 150 examples with feminine words and 150 examples with masculine words. Based on these 300 examples,
we observe five types of perturbation as shown through some examples in Table 3:
• Neutral; replace the gender-words with neutral word such as people, they, their, and etc.
• Convert-gender; replace the gender-words with opposite gender. the word "he" change to "she".
• Same-gender; replace the gender-words with the same gender. change the word "man" to "boy".
• Deleting; replace the gender-words with unknown token ([UNK]). In 300 examples, it only happens when there are several masked tokens.
• Identical; replace the gender-word with itself. It mostly happens when there is only one gender-word.
In our investigation with 300 examples, we had 46% Neutral, 29% Identical, 17% Convert-gender, 7% Same-gender, and 1% Deleting perturbation. As illustrated in Table 3, Gender-tuning does not make a meaningful change in identical and samegender perturbation. These examples likely conform to the gender biases in the MLM. Suppose identical, or same-gender perturbation gets the correct output from the perturbation process (Lperturb. ≈ 0). In this case, the only way to learn the biases in the MLM is to get the correct output from finetuning step and joint-loss close to zero. This issue stops the MLM and fine-tuning model from further update. However, joint-loss plays an essential role
in alleviating learning gender bias from identical and same-gender perturbations.
To clarify the role of joint-loss in overcoming above problem, we investigated fine-tuning output on identical and same-gender perturbations. We observed that fine-tuning gets the incorrect output from 60% of the identical and 75% of the samegender perturbation. Thus these examples return to training iteration because their joint-loss is large enough to update the language models and perform a new training iteration. New training iteration means re-perturbing and re-fine-tuning result on these examples. Therefore, training based on both training steps’ loss and computing joint-loss persistently prevents learning from gender bias in MLM as well as the PLM.

heading: 5 Ablation
text: We conduct the ablation experiments to demonstrate the effectiveness of Gender-tuning components, including 1) joint-training process and 2) joint-loss in Gender-tuning’s debiasing performance (Table 4). The experiments are as follows: 1) Gender-tuningno−joint−training: first we used MLM to train the PLM through the gender-word perturbation on downstream task datasets. Then we fine-tuned the PLM on the downstream task dataset. 2) Gender-tuningno−joint−loss: we train Gender-tuning based on only fine-tuning loss.
In both PLMs, results illustrate that Gendertuning is more effective for reducing the average gender bias than in two ablation experiments. The two ablation experiments magnify the bias scores noticeably, while Gender-tuning gains the smallest SEAT absolute effect size, especially in the BERT model. Results also show that the ablation experiment setups that do not benefit from jointloss cannot update the MLM and PLM when the output of the fine-tuning classification is correct (Lfine−tuning ≈ 0), even though the correct output likely bases on the gender biases in the PLMs.

heading: 6 Conclusion
text: We propose a novel approach for debiasing PLMs through fine-tuning on downstream tasks’ datasets. The proposed method is an aggregation of biasword perturbation using MLM and fine-tuning classification. In this study, we evaluated our proposed method on gender biases and named it Gendertuning. Comprehensive experiments prove that Gender-tuning outperforms two state-of-the-art debiasing methods while improving the performance of the PLMs on downstream tasks. The key advantage of our approach is using the fine-tuning setting that allows the training process to be carried out without needing additional training processes or datasets. Also, it makes Gender-tuning a plug-andplay debiasing tool deployable to any PLMs.

heading: 7 Limitation
text: Although Gender-tuning succeeds in reducing the gender bias scores in the pre-trained language models, there are some limitations to performing debiasing. Gender-tuning only works on gender-related words list. Thus Gender-tuning cannot cover the probable gender biases that do not exist in its’ list. We defer the gender-related word list modification to future research. All our experiments ran on English language texts with English gender-word morphology.

heading: A Appendix

heading: A.1 Baselines
text: For comparison purposes, we chose two stateof-the-art baselines which focus on debiasing sentence-level pre-trained text encoders in PLMs.

heading: A.1.1 SENT-DEBIAS
text: SENT-DEBIAS (Liang et al., 2020) is an extension of the HARD-DEBIAS method (Bolukbasi et al., 2016) to debias sentences for both binary and multi-class bias attributes spanning gender and religion. The key advantage of Sent-D is the contextualization step in which bias-attribute words are converted into bias-attribute sentences by using a diverse set of sentence templates from text corpora. Sent-D is a four-step process that involves: identifying words that exhibit biased attributes, contextualizing them in sentences that contain these biases, creating sentence representations, estimating the subspace of the bias represented in the sentences, and debiasing general sentences by removing the projection onto this subspace.

heading: A.1.2 FairFil
text: FairF (Cheng et al., 2021) is the first neural debiasing method for pretrained sentence encoders. For a given pretrained encoder, FairF learns a fair filter (FairFil) network, whose inputs are the original embedding of the encoder, and outputs are the debiased embedding. Inspired by the multi-view contrastive learning (Chen et al., 2020), for each training sentence, FairF first generates an augmentation that has the same semantic meaning but in a different potential bias direction. FairFil is contrastively trained by maximizing the mutual information between the debiased embeddings of the original sentences and corresponding augmentations. To further eliminate bias from sensitive words in sentences, FairF uses debiasing regularizer, which minimizes the mutual information between debiased embeddings and the sensitive words’ embeddings.

heading: A.2 Bias Evaluation Metric
text: Following the prior studies (Sent-D and FairF), we use Sentence Encoder Association Test (SEAT) (May et al., 2019) to measure the gender bias scores in the pre-trained language models that trained using Gender-tuning. SEAT extended the Word Embedding Association Test (WEAT; caliskan2017semantics) to sentence-level representations. WEAT compares the distance of two sets. Two sets of target words (e.g., {family, child, parent,...} and {work, office, profession,...} ) that characterize particular concepts family and career respectively. Two sets of attribute words (e.g., {man, he, him,...} and {woman, she, her,...} ) that characterize a type of bias. WEAT evaluates whether the representations for words from one particular attribute word set tend to be more closely associated with the representations for words from one particular target word set. For instance, if the female attribute words listed above tend to be more closely associated with the family target words, this may indicate bias within the word representations.
Let’s denote A and B as sets of attribute words and X and Y the set of target words. As described in (Caliskan et al., 2017) the WEAT test statistic is:
s(X,Y,A,B) = ∑
x∈X s(x,A,B)−
∑ y∈Y s(y,A,B)
(2) where for a specific word w , s(w,A,B) is defined as the difference between w’s mean cosine similarity with the words from A and w’s mean cosine
similarity with the word from B. They report an effective size given by:
d = µ([s(x,A,B)]x∈X − µ([s(y,A,B)]y∈Y )
σ([s(t,X, Y )]t∈A∪B) (3)
where µ and σ denote the mean and standard deviation respectively. Hence, an effect size closer to zero represents smaller degree of bias in the word representation. The SEAT test extended WEAT by replacing the word with a collection of template sentences (i.e., "this is a [word]", "that is a [word]"). Then the WEAT test statistic can be computed on a given sets of sentences including attribute and target words using sentence representations from a language model.

heading: A.3 PLMs
text: Two widely used pre-trained language models have been chosen for this study, BERT-base (Devlin et al., 2019)and RoBERTa-base (Liu et al., 2019). BERT-base is a bidirectional encoder with 12 layers and 110M parameters that is pre-trained on 16GB of text. RoBERTa-base has almost the same architecture as BERT but is pre-trained on ten times more data (160GB) with significantly more pretraining steps than BERT.

heading: A.4 Datasets
text: We conducted empirical studies on the following three tasks from the GLUE3 benchmark (Wang et al., 2019): (1) SST-2: Stanford Sentiment Treebank is used for binary classification for sentences extracted from movie reviews (Socher et al., 2013). It contains 67K training sentences. (2) CoLA: Corpus of Linguistic Acceptability (Warstadt et al., 2019) consists of English acceptability judgment. CoLA contains almost 9K training examples. (3) QNLI: Question Natural Language Inference (Wang et al., 2018) is a QA dataset which is derived from the Stanford Question Answering Dataset (Rajpurkar et al., 2016) and used for binary classification. QNLI contains 108K training pairs. Also, we use the feminine and masculine word lists created by (Zhao et al., 2018) for gender-word perturbation in Gender-tuning.
3https://gluebenchmark.com/tasks

heading: A.5 Hyperparameters
text: The hyperparameters of the models, except batch size, are set to their default4 values (e.g., epoch = 3, learning-rate = 2× 10−5, and etc.). After trying several trials run, the batch size has been selected among {8, 16, 32}. We empirically selected the optimal value for α by a grid search in 0 < α < 1 with 0.1 increments. For each downstream task, the best value of α sets to 0.7. All experiments were performed with three training epochs and using an NVIDIA V100 GPU.

heading: A.6 Related Works
text: Debiasing Database; The most straightforward approach for reducing the social biases in the training corpora is bias-neutralization. In this way, the training corpus is directly re-balanced by swapping or removing bias-related words and counterfactual data augmentation (CDA) (Zmigrod et al., 2019; Dinan et al., 2020; Webster et al., 2020; Dev et al., 2020; Barikeri et al., 2021). Also, Gehman et al. (2020) proposed domain-adaptive pre-training on unbiased corpora. Although the results showed these proposed methods mitigated the social biases in the pre-trained models, they need to be re-trained on a larger scale of the corpora. For example, Webster et al. (2020) proposed a CDA that needs an additional 100k steps of training on the augmented dataset. Data augmentation and collecting a large-scale unbiased corpus are both computationally costly.
Debiasing Embedding; There are several solutions for debiasing static word embedding (Bolukbasi et al., 2016; Kaneko and Bollegala, 2019; Manzini et al., 2019; Ravfogel et al., 2020) and debiasing contextualized word-embedding (Caliskan et al., 2017; Brunet et al., 2019) and sentence-embedding (Liang et al., 2020; Cheng et al., 2021). Compared to debiasing static word embedding, where the semantic representation of a word is limited to a single vector, contextualized word/sentence embedding models are more challenging (Kaneko and Bollegala, 2019). Since the key to the pre-trained language models’ success is due to powerful embedding layers (Liang et al., 2020), debiasing embedding might affect transferring of the accurate information and performance of these models on the downstream tasks. Also, they need some pre-training for
4https://github.com/huggingface/transformers
debiasing the embedding layer before fine-tuning on downstream tasks.
ACL 2023 Responsible NLP Checklist

heading: A For every submission:
text: 3 A1. Did you describe the limitations of your work?
7
A2. Did you discuss any potential risks of your work? Not applicable. Left blank.

heading: 3 A3. Do the abstract and introduction summarize the paper’s main claims?
text: Left blank.
7 A4. Have you used AI writing assistants when working on this paper? Left blank.
B 7 Did you use or create scientific artifacts?

heading: Left blank.
text: B1. Did you cite the creators of artifacts you used? Not applicable. Left blank.
B2. Did you discuss the license or terms for use and / or distribution of any artifacts? Not applicable. Left blank.
B3. Did you discuss if your use of existing artifact(s) was consistent with their intended use, provided that it was specified? For the artifacts you create, do you specify intended use and whether that is compatible with the original access conditions (in particular, derivatives of data accessed for research purposes should not be used outside of research contexts)? Not applicable. Left blank.
B4. Did you discuss the steps taken to check whether the data that was collected / used contains any information that names or uniquely identifies individual people or offensive content, and the steps taken to protect / anonymize it? Not applicable. Left blank.
B5. Did you provide documentation of the artifacts, e.g., coverage of domains, languages, and linguistic phenomena, demographic groups represented, etc.? Not applicable. Left blank.
3 B6. Did you report relevant statistics like the number of examples, details of train / test / dev splits, etc. for the data that you used / created? Even for commonly-used benchmark datasets, include the number of examples in train / validation / test splits, as these provide necessary context for a reader to understand experimental results. For example, small differences in accuracy on large test sets may be significant, while on small test sets they may not be. Left blank.
C 3 Did you run computational experiments? 3

heading: 3 C1. Did you report the number of parameters in the models used, the total computational budget
text: (e.g., GPU hours), and computing infrastructure used? 3
The Responsible NLP Checklist used at ACL 2023 is adopted from NAACL 2022, with the addition of a question on AI writing assistance.

heading: 3 C2. Did you discuss the experimental setup, including hyperparameter search and best-found
text: hyperparameter values? 3

heading: 3 C3. Did you report descriptive statistics about your results (e.g., error bars around results, summary
text: statistics from sets of experiments), and is it transparent whether you are reporting the max, mean, etc. or just a single run? 4

heading: 3 C4. If you used existing packages (e.g., for preprocessing, for normalization, or for evaluation), did
text: you report the implementation, model, and parameter settings used (e.g., NLTK, Spacy, ROUGE, etc.)? 3
D 7 Did you use human annotators (e.g., crowdworkers) or research with human participants?

heading: Left blank.
text: D1. Did you report the full text of instructions given to participants, including e.g., screenshots, disclaimers of any risks to participants or annotators, etc.? Not applicable. Left blank.
D2. Did you report information about how you recruited (e.g., crowdsourcing platform, students) and paid participants, and discuss if such payment is adequate given the participants’ demographic (e.g., country of residence)? Not applicable. Left blank.
D3. Did you discuss whether and how consent was obtained from people whose data you’re using/curating? For example, if you collected data via crowdsourcing, did your instructions to crowdworkers explain how the data would be used? Not applicable. Left blank.
D4. Was the data collection protocol approved (or determined exempt) by an ethics review board? Not applicable. Left blank.
D5. Did you report the basic demographic and geographic characteristics of the annotator population that is the source of the data? Not applicable. Left blank.

heading: Future Work
text: We defer the gender-related word list modification to future research. All our experiments ran on English language texts with English gender-word morphology