Debiasing Pre-Trained Language Models via Efficient Fine-Tuning (2022)

Abstract: An explosion in the popularity of transformer-based language models (such as GPT-3, BERT, RoBERTa, and ALBERT) has opened the doors to new machine learning applications involving language modeling, text generation, and more. However, recent scrutiny reveals that these language models contain inherent biases towards certain demographics reflected in their training data. While research has tried mitigating this problem, existing approaches either fail to remove the bias completely, degrade performance (“catastrophic forgetting”), or are costly to execute. This work examines how to reduce gender bias in a GPT-2 language model by fine-tuning less than 1% of its parameters. Through quantitative benchmarks, we show that this is a viable way to reduce prejudice in pre-trained language models while remaining cost-effective at scale.

## introduction transformer-based language models such as gpt-2 @xcite , @xcite , bert @xcite , roberta @xcite , and albert @xcite have propelled advances in natural language processing (nlp) for tasks including language modeling, text generation, and more @xcite . while these powerful language models pick up useful patterns such as english grammar and syntax, they also learn harmful and nuanced information. analysis by @xcite reveals that gpt-2 will reveal gendered, racial, and religious stereotypes. thus, practitioners must ensure that their language models benefit all people fairly before deploying them into the real world. in recent work, @xcite demonstrate that fine-tuning gpt-3 on a curated dataset will mitigate biased output. however, their approach requires fine-tuning the entire model, which has a few fundamental limitations. first, training a large language model such as gpt-2 or gpt-3 from scratch takes considerable time, costs on the order of millions of dollars, and emits hundreds of tons of co 2 into the environment @xcite . second, fine-tuning all parameters may significantly drop the language modeling performance due to "catastrophic forgetting": the phenomenon when an ai model unlearns old knowledge when trained with additional information @xcite . we propose a novel approach to modify a gpt-2 language model that overcomes the aforementioned limitations. in particular, our approach is inspired by @xcite , who adapt an existing gpt-2 model (trained on english text) to completely different task modalities such as image classification. they froze over 99% of the model's trainable parameters (namely the attention and feedforward layers, which do the bulk of the computation) while only modifying the layer norm parameters, positional embeddings, and applying a linear transformation to the input and output layer. a natural question arises-if it is possible to adapt a language model to completely different tasks and modalities in such an efficient way, then is it possible to mitigate language model prejudice through similar means? this paper makes the following contributions: first, we show that fine-tuning less than 1% of the gpt-2 language model can reduce prejudice on quantitative benchmarks. second, we publicly release our fine-tuned model on github 1 and provide a live demo on hugging face spaces to qualitatively compare our model output side-by-side with the original gpt-2 output. 2 2 related work bias issues in machine learning unfair behaviors have been found in many machine learning and artificial intelligence applications, including facial recognition (raji and buolamwini, 2019), recommendation systems @xcite , and speech recognition @xcite . one major source of bias comes from training datasets that render models to behave negatively towards underrepresented groups @xcite . for example, @xcite found that im-agenet @xcite and the open images dataset @xcite disproportionately represented people from north america and europe. to mitigate biased behaviors in machine learning models, researchers have proposed methods targeting different tasks and domains, such as classification @xcite , regression @xcite , and adversarial learning @xcite . ## results see we hypothesize that the slight average increase in the lms can be attributed to the model better fitting the task itself; i.e., the curated dataset more closely resembles the stereoset cat prompts compared to the heterogeneous repository from which gpt-2 was originally trained @xcite . the stereoset ss decrease signifies that the models correctly balance the word distributions away from traditional stereotypes. overall, this leads to an icat increase of about 2.73% by training only a relatively small portion of the model. roughly a third of the fine-tuning dataset comes from winobias @xcite , which focuses on gender and profession bias, which may explain why the stereoset gender and profession categories observed particularly good results. for stereoset intrasentence gender, the top-performing model (ln) observed a 2.59 point decrease in its ss, which is a 4.14% improvement from baseline leading to an icat increase of 4.31 (6.19%). the top-performing overall model was the ln + wpe model, which we fine-tuned on only 0.66% of the original gpt-2 parameters ( ## conclusion before successfully deploying these powerful language models in real-world applications, society must take steps to ensure that it does not marginal-