Improving Gender Fairness of Pre-Trained Language Models without Catastrophic Forgetting (2023)

Abstract: Existing studies addressing gender bias of pre-trained language models, usually build a small gender-neutral data set and conduct a second phase pre-training on the model with such data. However, given the limited size and concentrated focus of the gender-neutral data, catastrophic forgetting would occur during second-phase pre-training. Forgetting information in the original training data may damage the model’s downstream performance by a large margin. In this work, we empirically show that catastrophic forgetting occurs in such methods by evaluating them with general NLP tasks in GLUE. Then, we propose a new method, GEnder Equality Prompt (GEEP), to improve gender fairness of pre-trained models with less forgetting. GEEP freezes the pre-trained model and learns gender-related prompts with gender-neutral data. Empirical results show that GEEP not only achieves SOTA performances on gender fairness tasks, but also forgets less and performs better on GLUE by a large margin.

## introduction pre-trained language models, e.g., bert @xcite and roberta @xcite , have shown competitive performance in a wide variety of nlp downstream applications. however, such models are often prone to exhibit gender bias @xcite @xcite , due to their large scale unsupervised training data from the web @xcite . gender bias refers to unbalanced model behaviors with respect to a specific gender @xcite . among various gender-biased behaviours of pre-trained models, bias on professions is the most prominent and wellstudied @xcite @xcite . for example, in coreference resolution tasks, a pre-trained model would predict female pronoun and names for professions like "nurse" and "housekeeper", while predict male pronouns for "computer programmer" or "doctor" @xcite . the pre-trained models also wouldn't prefer genderneutral pronouns actively, which is unfair to other gender identities beyond males/females @xcite . given the large model size and tremendous time complexity for language model pre-training, training a gender-neutral model from scratch with manually filtered data seems impossible for most organizations. due to this limitation, existing studies usually build a relatively small gender-neutral data set (for example building a data set that have more balanced gender pronouns for profession names), and conduct second phase pre-training on the pretrained model with such data @xcite . however, given the limited size of the gender-neutral data and its potential distributional mismatch with the original pre-training data, catastrophic forgetting can occur during second-phase pre-training of such methods. catastrophic forgetting @xcite is a long-standing problem which illustrates the tendency of a neural network to forget previously learned information upon learning new information. when it comes to further training a pretrained model, using the small gender-neutral data to update the entire massive model could make the model forget the diverse information from the original pre-training data, which damages the model's downstream performance by a large margin. in this paper, we first empirically verify that further updating a pre-trained model (such as roberta @xcite ) with manually-built gender-neutral data can cause catastrophic forgetting. we follow existing work and build our profession-related gender-neutral data set by filtering out wikipedia sentences mentioning professions and swapping their gender related pronouns. we find that although our gender-neutral data is from wikipedia which is part of roberta's pre-1 1249 training data, the model's performance on downstream tasks in glue @xcite still drops with a considerable margin after secondphase pre-training, due to the smaller size and more concentrated focus of the gender-neutral data. therefore, we propose a new method, gender equality prompt (geep), to alleviate gender bias of pre-trained models without catastrophic forgetting. specifically, inspired by recent prompt-tuning methods @xcite for fine-tuning large pre-trained models, geep freezes the entire model, adds and updates new word embeddings of professions as gender equality prompts, instead of updating all model parameters at second-phase pretraining as previous methods. since all the pretrained parameters are frozen during further training, diverse information from the original training data preserved in the pre-trained parameters is not erased. therefore forgetting can be alleviated to large extent. moreover, since the embeddings of professions are re-initialized when debiasing training starts, gender bias from previous data that is embedded in such representations is already removed before second-phase pre-training. therefore, geep also improves gender fairness of the model more effectively with much fewer iterations. empirical results show that geep not only achieves state-of-the-art performances with fewer iterations on various gender fairness tasks such as pronoun coreference resolution, but also forgets less and achieves better results on glue tasks. ## related work compared with the existing work focusing on quantifying and alleviating gender bias @xcite @xcite @xcite @xcite @xcite in standard word embedding models, such as word2vec @xcite and glove @xcite , gender bias in large pre-trained language models seems less studied. recent work on gender fairness of pre-trained language models, such as elmo @xcite and bert @xcite , mostly focus on showing and measuring the gender bias embedded in such models @xcite . these studies propose metrics to quantify gender bias in pre-trained language models @xcite @xcite . in our work, we employ such methods to evaluate geep and baseline methods on improving gender fairness. existing works focusing on mitigating gender bias of pre-trained models usually collect and build gender-neutral data on their own and conduct a second phase pre-training on the released pre-trained model @xcite @xcite . in this work, we demonstrate empirically that the performance of the debiased model on general downstream tasks such as glue, still drops by a considerable margin after such second-phase pre-training. then, given this phenomenon, we propose geep to alleviate gender bias in pre-trained models without forgetting. ## improving gender fairness without forgetting in this section, we first describe the gender-neutral collection method we adopt from existing methods and the forgetting issue in such methods. then we describe the proposed method gender equality prompt (geep). ## experiments in this section, we present the results of geep and its baselines to show that geep achieves state-ofthe-art performances on gender fairness tasks while alleviating the forgetting issue of the baselines. ## closing remarks in this paper, we proposed geep to improve gender fairness of pre-trained language models with less catastrophic forgetting. for a fair comparison to existing work under the current gender fairness literature, we mainly conduct experiments with profession-related gender neutral data because profession-related gender bias is relatively more well studied so far. the good empirical results indicates it is worth to try applying geep to other more challenging and under-explored aspects of gender fairness, which would be our future work. 4 1252 ## 1258 we would also like to note that geep-without-gn outperforms sppa-without-gn as well and achieve similar glue score as roberta. this indicates that geep can also alleviate forgetting introduced by data subset selection effectively when there is not gender-neutralizing procedure is taken.