An Empirical Survey of the Effectiveness of Debiasing Techniques for Pre-trained Language Models (2022)

Abstract: Recent work has shown pre-trained language models capture social biases from the large amounts of text they are trained on. This has attracted attention to developing techniques that mitigate such biases. In this work, we perform an empirical survey of five recently proposed bias mitigation techniques: Counterfactual Data Augmentation (CDA), Dropout, Iterative Nullspace Projection, Self-Debias, and SentenceDebias. We quantify the effectiveness of each technique using three intrinsic bias benchmarks while also measuring the impact of these techniques on a model’s language modeling ability, as well as its performance on downstream NLU tasks. We experimentally find that: (1) Self-Debias is the strongest debiasing technique, obtaining improved scores on all bias benchmarks; (2) Current debiasing techniques perform less consistently when mitigating non-gender biases; And (3) improvements on bias benchmarks such as StereoSet and CrowS-Pairs by using debiasing strategies are often accompanied by a decrease in language modeling ability, making it difficult to determine whether the bias mitigation was effective.

## introduction large pre-trained language models have proven effective across a variety of tasks in natural language processing, often obtaining state of the art performance @xcite @xcite . these models are typically trained on large amounts of text, originating from unmoderated sources, such as the internet. while the performance of these pre-trained models is remarkable, recent work has shown that they capture social biases from the data they are trained on @xcite @xcite @xcite . because of these findings, an increasing amount of research has focused on developing techniques to mitigate these biases @xcite @xcite @xcite . however, the proposed techniques are often not investigated thoroughly. for instance, much work focuses only on mitigating gender bias despite pre-trained language models being plagued by other social biases (e.g., racial or religious bias). additionally, the impact that debiasing has on both downstream task performance, as well as language modeling ability, is often not well explored. in this paper, we perform an empirical survey of the effectiveness of five recently proposed debiasing techniques for pre-trained language models: foot_1 counterfactual data augmentation (cda; @xcite , dropout @xcite , iterative nullspace projection (inlp; @xcite , self-debias @xcite , and sentencedebias @xcite . following the taxonomy described by @xcite , our work studies the effectiveness of these techniques in mitigating representational biases from pre-trained language models. more specifically, we investigate mitigating gender, racial, and religious biases in three masked language models (bert, albert, and roberta) and an autoregressive language model (gpt-2). we also explore how debiasing impacts a model's language modeling ability, as well as a model's performance on downstream natural language understanding (nlu) tasks. concretely, our paper aims to answer the following research questions: q1 which technique is most effective in mitigating bias? q2 do these techniques worsen a model's language modeling ability? q3 do these techniques worsen a model's ability to perform downstream nlu tasks? to answer q1 ( §4), we evaluate debiased models against three intrinsic bias benchmarks: the sentence encoder association test (seat; @xcite , stereoset @xcite , and crowdsourced stereotype pairs (crows-pairs; @xcite . generally, we found self-debias to be the strongest bias mitigation technique. to answer q2 ( §5) and q3 ( §6), we evaluate debiased models against wikitext-2 @xcite and the general language understanding evaluation (glue; wang and cho 2019) benchmark. we found debiasing tends to worsen a model's language modeling ability. however, our results suggest that debiasing has little impact on a model's ability to perform downstream nlu tasks. ## techniques for measuring bias we begin by describing the three intrinsic bias benchmarks we use to evaluate our debiasing techniques. we select these benchmarks as they can be used to measure not only gender bias, but also racial and religious bias in language models. ## debiasing techniques below, we describe the five debiasing techniques we evaluate in this work. we refer readers to appendix c for additional experimental details on each debiasing technique. ## which technique is most effective in mitigating bias? to investigate which technique is most effective in mitigating bias (q1), we evaluate debiased bert, albert, roberta, and gpt-2 models against seat, stereoset, and crows-pairs. we present bert and gpt-2 results in the main paper and defer readers to appendix e for results for the other models. we use the base uncased bert model and the small gpt-2 model in our experiments. ## how does debiasing impact language modeling? to investigate how debiasing impacts language modeling (q2), we measure perplexities before and after debiasing each of our models on wikitext-2 @xcite . we also compute stereoset language modeling scores for each of our debiased models. we discuss our findings below. in to investigate how debiasing impacts performance on downstream nlu tasks (q3), we evaluate our gender debiased models against the glue benchmark after fine-tuning them. we report the results for bert and gpt-2 in ## discussion and limitations below, we discuss our findings for each research question we investigated in this work. we also discuss some of the limitations of our study. q1: which technique is most effective in mitigating bias? we found self-debias to be the strongest debiasing technique. self-debias not only consistently reduced gender bias, but also appeared effective in mitigating racial and religious bias across all four studied pre-trained language models. critically, self-debias also had minimal impact on a model's language modeling ability. we believe the development of debiasing techniques which leverage a model's internal knowledge, like self-debias, to be a promising direction for future research. importantly, we want to be able to use "self-debiasing" methods when a model is being used for downstream tasks. q2: do these techniques worsen a model's language modeling ability? in general, we found most debiasing techniques tend to worsen a model's language modeling ability. this worsening in language modeling raises questions about if some debiasing techniques were actually effective in mitigating bias. furthermore, when you couple this with the already noisy nature of the bias benchmarks used in our work @xcite it becomes even more difficult to determine which bias mitigation techniques are effective. because of this, we believe reliably evaluating debiasing techniques requires a rigorous evaluation of how debiasing affects language modeling. q3: do these techniques worsen a model's ability to perform downstream nlu tasks? we found the debiasing techniques did not damage a model's ability to learn to perform downstream nlu tasks-a finding in alignment with other recent work @xcite . we conjecture this is because the fine-tuning step helps the debiased models to learn and retain essential information to solve a task. limitations. we describe three of the main limitations of our work below. 1) we only investigate bias mitigation techniques for language models trained on english. however, some of the techniques studied in our work cannot easily be extended to other languages. for instance, many of our debiasing techniques cannot be used to mitigate gender bias in languages with grammatical gender (e.g., french). foot_5 2) our work is skewed towards north american social biases. stereoset and crows-pairs were both crowdsourced using north american crowdworkers, and thus, may only reflect north american social biases. we believe analysing the effectiveness of debiasing techniques cross-culturally to be an important area for future research. furthermore, all of the bias benchmarks used in this work have only positive predictive power. for example, a perfect stereotype score of 50% on stereoset does not indicate that a model is unbiased. ## conclusion to the best of our knowledge, we have performed the first large scale evaluation of multiple debiasing techniques for pre-trained language models. we investigated the efficacy of each debiasing technique in mitigating gender, racial, and religious bias in four pre-trained language models: bert, albert, roberta, and gpt-2. we used three intrinsic bias benchmarks to evaluate the effectiveness of each debiasing technique in mitigating bias and also investigated how debiasing impacts language modeling and downstream nlu task performance. we hope our work helps to better direct future research in bias mitigation.