Auto-Debias: Debiasing Masked Language Models with Automated Biased Prompts (2022)

Abstract: Human-like biases and undesired social stereotypes exist in large pretrained language models. Given the wide adoption of these models in real-world applications, mitigating such biases has become an emerging and important task. In this paper, we propose an automatic method to mitigate the biases in pretrained language models. Different from previous debiasing work that uses external corpora to fine-tune the pretrained models, we instead directly probe the biases encoded in pretrained models through prompts. Specifically, we propose a variant of the beam search method to automatically search for biased prompts such that the cloze-style completions are the most different with respect to different demographic groups. Given the identified biased prompts, we then propose a distribution alignment loss to mitigate the biases. Experiment results on standard datasets and metrics show that our proposed Auto-Debias approach can significantly reduce biases, including gender and racial bias, in pretrained language models such as BERT, RoBERTa and ALBERT. Moreover, the improvement in fairness does not decrease the language models’ understanding abilities, as shown using the GLUE benchmark.

## introduction pretrained language models (plms), such as masked language models (mlms), have achieved remarkable success in many natural language processing (nlp) tasks @xcite @xcite . unfortunately, pretrained language models, which are trained on large human-written corpora, also inherit human-like biases and undesired social stereotypes @xcite @xcite . for example, in the fill-inthe-blank task, bert @xcite substitutes @xcite in the sentence "the man/woman had a job as [mask]" with "manager/receptionist" respectively, reflecting occupational gender bias. the human-like biases and stereotypes encoded in plms are worrisome as they can be propagated or even amplified in downstream nlp tasks such as sentiment classification @xcite , co-reference resolution @xcite , clinical text classification @xcite and psychometric analysis @xcite . however, although it is important to mitigate biases in plms, debiasing masked language models such as bert is still challenging, because the biases encoded in the contextualized models are hard to identify. to address this challenge, previous efforts seek to use additional corpora to retrieve the contextualized embeddings or locate the biases and then debias accordingly. for example, @xcite ; @xcite ; @xcite use external corpora to locate sentences containing the demographic-specific words (e.g., man and women) or stereotype words (e.g., manager and receptionist) and then use different debiasing losses to mitigate the biases. using external corpora to debias plms heavily relies on the quality of the corpora. empirical results show that different corpora have various effects on the debiasing results: some external corpora do mitigate the bias, while others introduce new biases to the plms @xcite . this is because the corpora used for debiasing may not have enough coverage of the biases encoded in the plms. nevertheless, our understanding of how to quantitatively assess the level of biases in a corpus remains limited @xcite . mitigating biases in plms without external corpora is an open research gap. recent work in language model prompting shows that through cloze-style prompts, one can probe and analyze the knowledge @xcite , biases @xcite or toxic content @xcite in plms. motivated by this, instead of refer-he man boy father .... she woman girl mother .... after we obtain the biased prompts, we probe the biased content with such prompts and then correct the model bias. we propose an equalizing loss to align the distributions between the [mask] tokens predictions, conditioned on the corresponding demographic words. in other words, while the automatically crafted biased prompts maximize the disagreement between the predicted [mask] token distributions, the equalizing loss minimizes such disagreement. combining the automatic prompts generation and the distribution alignment fine-tuning, our novel method, auto-debias can debias the plms without using any external corpus. auto-debias is illustrated in figure 1 . in the experiments, we evaluate the performance of auto-debias in mitigating gender and racial biases in three popular masked language models: bert, albert, and roberta. moreover, to alleviate the concern that model debias-ing may worsen a model's performance on natural language understanding (nlu) tasks @xcite , we also evaluate the debiased models on glue tasks. the results show that our proposed auto-debias approach can effectively mitigate the biases while maintaining the capability of language models. we have released the auto-debias implementation, debiased models, and evaluation scripts at https://github. com/irenehere/auto-debias . ## related works as nlp models are prevalent in real-world applications, a burgeoning body of literature has investigated human-like biases in nlp models. bias in nlp systems can stem from training data @xcite , pre-trained word embeddings or can be amplified by the machine learning models. most existing work focuses on the bias in pre-trained word embeddings due to their universal nature @xcite . prior work has found that traditional static word embeddings contain humanlike biases and stereotypes @xcite @xcite @xcite . debiasing strategies to mitigate static word embeddings have been proposed accordingly @xcite @xcite . contextualized embeddings such as bert have been replacing the traditional static word embeddings. researchers have also reported similar human-like biases and stereotypes in contextual embedding plms @xcite @xcite @xcite or in the text generation tasks @xcite . compared to static word embeddings, mitigating the biases in contextualized plms is more challenging since the representation of a word usually depends on the word's context. @xcite propose to augment the pretraining corpus with demographicbalanced sentences. liang et al. (2020); cheng et al. (2021) suggest removing the demographicdirection from sentence representations in a posthoc fashion. however, augmenting the pretraining corpus is costly and post-hoc debiasing does not mitigate the intrinsic biases encoded in plms. therefore, recent work has proposed to fine-tune the plms to mitigate biases by designing different debiasing objectives @xcite @xcite . they rely on external corpora, and the debiasing results based on these external corpora vary significantly @xcite . moreover, @xcite find that existing debiasing methods are generally ineffective: first, they do not generalize well beyond gender bias; second, they tend to worsen a model's language modeling ability and its performance on nlu tasks. in this work, we propose a debiasing method that does not necessitate referring to any external corpus. our debiased models are evaluated on both gender and racial biases, and we also evaluate their performance on nlu tasks. ## auto-debias: probing and debiasing using prompts we propose auto-debias, a debiasing technique for masked language models that does not entail referencing external corpora. auto-debias contains two stages: first, we automatically craft the biased prompts, such that the cloze-style completions have the highest disagreement in generating stereotype words with respect to demographic groups. second, after we obtain the biased prompts, we debias the language model by a distribution alignment loss, with the motivation that the prompt completion results should be independent to the choice of different demographic-specific words. ## debiasing performance we evaluate the performance of auto-debias in mitigating biases in masked language models. debiasing strategy benchmarks. we consider the following debiasing benchmarks. based on which stage the debiasing technique applies to, the benchmarks can be grouped into three categories. our proposed auto-debias approach belongs to the fine-tuning category. it does not require any external corpus compared to the previous fine-tuning debiasing approaches. pretrained models. in the experiments, we consider three popular masked language models: bert @xcite , albert @xcite and roberta @xcite . we implement bert, albert, and roberta using the huggingface transformers library @xcite . generated prompts 1 substitute, premier, united, became, liberal, major, acting, professional, technical, against, political 2 united domestic, substitute foreign, acting field, eventual united, professional domestic, athletic and 3 professional domestic real, bulgarian domestic assisted, former united free, united former inside 4 eventual united reading and, former united choice for, professional domestic central victoria 5 united former feature right and, former united choice for new, eventual united reading and biased prompts. we present some examples of the generated biased prompts in ## discussion prompts have been an effective tool in probing the internal knowledge relations of language models @xcite , and they can also reflect the stereotypes encompassed in plms @xcite . ideally, when prompted with different demographic targets and potential stereotype words, a fair language model's generated predictions should be equally likely. our method shows that, from the other direction, imposing fairness constraints on the prompting results can effectively promote the fairness of a language model. we also observe a trade-off between efficiency and equity: tuning with more training steps, more prompts and more target words leads to a fairer model (which can even make the seat score very close to 0), however, it comes at the price of harming the language modeling ability. over-tuning may harm the internal language patterns. it is important to strike a balance between efficiency and equity with appropriate fine-tuning. also, in order not to break the desirable connections between targets and attributes, carefully selecting the target words and stereotyped attribute words is crucial. however, acquiring such word lists is difficult and depends on the downstream applications. some prior work establishes word lists based on theories, concepts, and methods from psychology and other social science literature @xcite . however, such stereotyped word lists are usually lim-ited, are often contextualized, and offer limited coverage. moreover, word lists about other protected groups, such as the groups related to education, literacy, or income, or even intersectional biases @xcite , are still missing. one promising method to acquire such word lists is to probe related words from a pre-trained language model, for example, "the man/woman has a job as [mask]" yields job titles that reflect the stereotypes. we leave such probing-based stereotype word-list generation as an important and open future direction. ## conclusion in this work, we propose auto-debias, a framework and method for automatically mitigating the biases and stereotypes encoded in plms. compared to previous efforts that rely on external corpora to obtain context-dependent word embeddings, our approach automatically searches for biased prompts in the plms. therefore, our approach is effective, efficient, and is perhaps also more objective than prior methods that rely heavily on manually crafted lists of stereotype words. experimental results on standard benchmarks show that auto-debias reduces gender and race biases more effectively than prior efforts. moreover, the debiased models also maintain good language modeling capability. bias in nlp systems can stem from different aspects such as training data, pretrained embeddings, or through amplification when finetuning the machine learning models. we believe this work contributes to the emerging literature that sheds light on practical and effective debiasing techniques.