Language Models Get a Gender Makeover: Mitigating Gender Bias with Few-Shot Data Interventions (2023)

Abstract: Societal biases present in pre-trained large language models are a critical issue as these models have been shown to propagate biases in countless downstream applications, rendering them unfair towards specific groups of people. Since large-scale retraining of these models from scratch is both time and compute-expensive, a variety of approaches have been previously proposed that de-bias a pre-trained model. While the majority of current state-of-the-art debiasing methods focus on changes to the training regime, in this paper, we propose data intervention strategies as a powerful yet simple technique to reduce gender bias in pre-trained models. Specifically, we empirically show that by fine-tuning a pre-trained model on only 10 debiased (intervened) training examples, the tendency to favor any gender is significantly reduced. Since our proposed method only needs a few training examples, we argue that our few-shot de-biasing approach is highly feasible and practical. Through extensive experimentation, we show that our de-biasing technique performs better than competitive state-of-the-art baselines with minimal loss in language modeling ability.

## introduction recently, there has been a surge of interest in pretrained large language models (llm) in natural language processing (nlp). it has been shown that the pre-training + finetuning of a model drastically improves its performance on downstream tasks as the knowledge captured by the pre-training on a large corpus is transferred to the downstream application when finetuning the model. however, this also leads to societal biases like gender bias that were implicitly learned by the pre-trained models being transferred to crucial downstream applications like job recommendation engines @xcite @xcite . analyzing and mitigating bias without requiring significant re-training or compute resources is crucial to the widespread adoption of llms in downstream applications. previous work @xcite , @xcite , @xcite has attempted to quantify bias, and others such as @xcite and @xcite have attempted to remove it algorithmically from the models. closer to our work are data-manipulative techniques such as @xcite and @xcite that modify the dataset and further fine-tune the model. in this paper, we propose simple data intervention strategies and show that they can mitigate gender bias in pre-trained models with the help of few-shot fine-tuning. moreover, taking inspiration from @xcite , we find that by utilizing a biased pre-trained llm for mining for most gender-biased samples in a dataset, our methods can mitigate gender bias with very few training samples. finally, we perform an extensive evaluation of our debiasing technique on two recent bias benchmarks @xcite and show that our method outperforms three existing state-of-the-art techniques and performs comparably to the other two. our main contributions are the following: ## related work in recent years, there has been growing concern about the bias/stereotypical discriminatory behavior by nlp models, particularly concerning gender. several studies have investigated the presence of gender bias in various nlp tasks and proposed methods for mitigating it. one line of research has focused on analyzing the extent of gender bias in pre-trained language models such as bert and gpt-2. these studies have found that these models exhibit a significant amount of gender bias in their word embeddings for bert @xcite and for gpt-2 @xcite and are prone to making stereotypical gender-based predictions (e.g., assuming that a doctor is male and a nurse is female). a standard evaluation metric used in this line of research is stereotype metrics such as stereoset @xcite , which evaluates the model's ability to predict gender stereotypes and crows pairs @xcite which measure whether a model generally prefers more stereotypical sentences. a similar line of work is gender bias tests proposed in big-bench @xcite . the tests assess the language model's gender biases, stereotypes, and ability to infer gender information. it evaluates gender bias and stereotype between male and female, and gender minority bias and stereotype between majority and minority. it also examines the model's language modeling performance, which can be affected during de-biasing. another line of research has proposed methods for debiasing these models. these methods can be broadly categorized into two groups: data-based and algorithm-based. data-based methods aim to reduce bias by removing or altering biased words from the training set. in contrast, algorithm-based methods aim to modify the model's architecture or training procedure to reduce bias. one popular databased method is "uncertainty sampling" @xcite , where the model is trained on the instances that it is most uncertain about, which can help to reduce bias by forcing the model to learn from a diverse set of examples. a popular algorithmbased method is "adversarial debiasing" proposed by @xcite , which fine-tunes the model using an adversarial loss to make it less sensitive to sensitive attributes such as gender. oscar proposed by @xcite , is another algorithm based method that utilizes the idea of disentangling "problematic concepts" like occupation and gender relationship instead of removing them altogether. mabel @xcite has both algorithm and data-based components, as it first augments the training data by swapping gender words and then applies a contrastive learning objective and alignment via entailment pairs. their data augmentation strategy is similar in spirit to the data intervention techniques we propose, however our analysis does not require training auxiliary models and uses significantly lesser data. @xcite , which aims to equalize the representation of genderspecific words in the embedding space, the "counterfactual data augmentation" (cda) method proposed by zimmermann and hoffmann (2022), which generates counterfactual examples to improve the model's robustness to bias, and "name-based counterfactual data substitution" proposed by @xcite which reduces gender bias by replacing gender-informative names in the dataset with gender-neutral names. our proposed method is also a data-based method, which aims to effectively reduce gender bias by taking inspiration from different techniques such as uncertainty sampling and name-based counterfactual data substitution @xcite . ## data interventions in order to reduce gender bias in pre-trained models, we carefully select diverse and hard-biased examples and then replace gender words with more neutral or equality-focused phrases. this is achieved by using a wordlist to find gender terms in sentences and then segregating words as name and non-name words. we call our initial approach naive-masking as it does not require a word list for mapping gender words to gender-neutral words. instead, it replaces all gender words with the fixed word "person." in our next approach, neutral-masking, we swap words in a slightly more semantically accurate manner. in this, we use a word-pair list that goes from gender words to gender-neutral words. with both approaches, we intend to introduce new words in a model's vocabulary to make it more likely to choose a more neutral word in gender-biased sentences. in our final approach, we exploit the existing vocabulary of the model and try to balance the confidence of prediction on opposite-gender words by using phrases instead. thus, we call our final approach random-phrase-masking as we instead substitute words with phrases that reflect the equality of gender. this approach not only reduces gender bias but also preserves the original meaning of the sentence in most cases. in our approach, we chose the phrases and order of gender words at random with equal probability. intervention input word converted word naive-masking he person she person boy person neutral-masking he they her their schoolgirl schoolkid random-phrase-masking he he or she she she and he boy either girl or boy additionally, we hypothesize that the choice of the dataset for fine-tuning is also essential. we choose two datasets: the wikitext-2 @xcite dataset, which has implicit gender bias since its sources from wikipedia articles, and the stere-oset dataset @xcite , which has explicit/more gender bias as it has been designed to evaluate gender bias. wikitext-2 foot_1 has 600 train articles and roughly 2m tokens while stereoset foot_2 (dev) has 2123 samples out of which we only consider 800 samples which are not unrelated. naturally, our data intervention method should work better on a dataset with training examples with gender bias while being devoid of meaningful gender associations like "she needs a gynecologist," where the gender of the person is important. by testing our method on both datasets, we can understand the sensitivity of our approach to the quality of training samples used. ## bias evaluation metrics we focus on evaluating the bias of a model while also measuring its language modeling capability. the ideal model would not just be one with the least bias but also one which does not compromise its language modeling performance. the dual estimation of bias and performance of a model was proposed in the stereoset benchmark @xcite , with the language modeling score (lms) measuring the percentage of times a meaningful token is predicted for the mask as opposed to a meaningless token, the stereotype score (ss) measuring the percentage of times the model predicted a stereotypical word as compared to an anti-stereotypical word, and an idealized cat score (icat) combining the lms and ss score into a single metric. an ideal model has an icat score of 100, while the worst biased model has an icat score of 0. we additionally evaluate the crows-pairs benchmark @xcite , which captures data with greater diversity in both the stereotypes expressed and the structure of sentences (50 is ideal). however, we note that the crow-s benchmark is much more limited compared to stereoset @xcite in terms of both the volume and variety of linguistic phenomenon relating to gender bias it covers. ## experiments we compare our proposed interventions with five baselines, 4 of which are state-of-the-art methods and the original pre-trained model. our first baseline is the application of dropouts to neural networks, dropout proposed by @xcite . next, we consider an algorithmic de-biasing technique inlp technique proposed by @xcite . then, we consider a sentence embedding de-biasing approach sentencedebias @xcite . finally, we consider a data-based approach cda (zmigrod et al., 2019) that is closest to our work. for a fairer comparison, we run the baselines with the same size (100) of the training set as our method. for all of our experiments, we consider the "bert-base-uncased" pre-trained model available from huggingface. for fine-tuning our model, we select a varying number of most-biased training samples (10, 50, and 100) from the wikitext-2 and stereoset (we only use the dev set) datasets, as discussed in section 4. we also compare this to a random selection of data points as an ablation study. on the selected dataset, we apply our interventions and obtain the modified dataset, which is then used to fine-tune our pre-trained model using masked language modeling (mlm) loss. the key point is that we only fine-tune the model on the gender words conditioned on the remaining text, significantly reducing the fine-tuning time. we perform ablations on various types of interventions as discussed in ## results ## conclusion in this paper, we show that simple data interventions on limited training data effectively reduce gender bias in llms. we also show that a biased pretrained llm can be used to mine the most effective de-biasing training examples. evaluation of our methods on state-of-the-art bias benchmarks empirically suggests that our methods effectively reduce gender bias. given that our methods can work in a few-shot manner and do not require any auxiliary model training, we hope that our work benefits further research in the domain of human-in-the-loop bias mitigation techniques by making the creation of bias mitigation datasets feasible.