Mitigating Gender Bias in Distilled Language Models via Counterfactual Role Reversal (2022)

Abstract: Language models excel at generating coherent text, and model compression techniques such as knowledge distillation have enabled their use in resource-constrained settings. However, these models can be biased in multiple ways, including the unfounded association of male and female genders with gender-neutral professions. Therefore, knowledge distillation without any fairness constraints may preserve or exaggerate the teacher model’s biases onto the distilled model. To this end, we present a novel approach to mitigate gender disparity in text generation by learning a fair model during knowledge distillation. We propose two modifications to the base knowledge distillation based on counterfactual role reversal—modifying teacher probabilities and augmenting the training set. We evaluate gender polarity across professions in open-ended text generated from the resulting distilled and finetuned GPT–2 models and demonstrate a substantial reduction in gender disparity with only a minor compromise in utility. Finally, we observe that language models that reduce gender polarity in language generation do not improve embedding fairness or downstream classification fairness.

## introduction the ever-increasing size of language models (lms) have increased their energy and compute requirements, making them impractical for many real-time resource-constrained applications such as personal assistants deployed on edge devices. to address this issue, various approaches have been proposed to compress or distill these large models (e.g., @xcite ; @xcite ; @xcite ). however, distillation techniques are designed to mimic the uncompressed lm (i.e., teacher model). thus, the societal biases encoded in the teacher he works in a hospital as a prompt . . . doctor, treating the elderly with a variety, and by all accounts does an excellent work of medicine. gpt-2 . . . physician and helps a lot of the patients. ## related work large lms embody societal biases that could result in harms such as misinformation, stereotype propagation, and disparate resource allocation @xcite . multiple studies have shown that lms are biased in producing outputs with negative connotations such as toxicity @xcite @xcite and negative regard @xcite towards minority populations. others have shown that lms encode prevalent gender biases, such as one gender being more associated with a particular class of professions. such biases can be revealed via contextual embedding tests @xcite , stereotype tests @xcite , and evaluation of generated texts @xcite . few works have also shown that lm can be biased towards ideologies, e.g., @xcite . approaches to mitigate bias in lms can be broadly summarized as: (a) training or finetuning on a balanced dataset @xcite ), (b) attaching prefix at inference or training time @xcite , and (c) using a bias or attribute classifier (e.g., toxicity classifier) to control fairness in text generation @xcite @xcite . while all these debiasing approaches can be used to mitigate bias in an lm after it is distilled, no prior work aims to directly debias and distill in a single step. furthermore, the majority of existing approaches focus on reducing toxic text generation @xcite @xcite @xcite . different from existing works, we present an approach for fair knowledge distillation that aims to mitigate gender bias in text generated from the distilled models. our approach is inspired by the counterfactual notion of fairness @xcite and introduces two modifications to the standard distillation: (a) counterfactual data augmentation, and (b) using modified teacher probabilities. counterfactual fairness and related notions have been previously used for bias mitigation in hate speech detection @xcite , word embeddings @xcite @xcite , and coreference resolution @xcite tasks. ours is the first work that uses counterfactual knowledge to achieve equitability in text generation during distillation. our method is also applicable when the student model or architecture is the same as the teacher model, and we have demonstrated it via experiments. ## notion of language model fairness we focus on mitigating gender bias in open-ended language generation from an lm. the bias is mea-sured by assessing the tendency of the lm to associate a specific set of professions to a specific gender, e.g., healthcare professions to female and engineering professions to male. as discussed in @xcite , such societal biases may cause a negative representational impact by propagating stereotypes, misrepresentations, or denigrations of social groups. we consider only binary gender in this paper as lms often do not encode sufficient representation of non-binary gender context, restricting a meaningful analysis @xcite . we use a related counterfactual notion of fairness, commonly studied in the nlp fairness literature, to motivate our fair distillation approach in sec. 4. the counterfactual notion of fairness @xcite adjudges a model fair if it generates similar predictions before and after swapping the sensitive features in the input. ## fair knowledge distillation via counterfactual role reversal in typical knowledge distillation, a smaller student model, imitating the behavior of the large teacher model, is obtained by using additional training signals from the target probabilities output by the teacher model. let {x 1 . . . x m } denote sequence of text tokens in a training sample, @xmath0. . . x t-1 } denotes sequence of tokens prior to t and boldface denote random variables. lms such as gpt-2 model probability distribution of next token p (x t |@xmath1, i.e., x @xmath2. distillation loss is then defined as follows: this loss consists of two terms: (a) the crossentropy (ce) between the predicted next token probability and the observed token, and (b) the kldivergence between the output probabilities from the teacher (p teacher ) and the student (p θ ) models. the kl-divergence term provides a stronger training signal to the student, leading to more accurate and faster learning @xcite . knowledge distillation (eq. ( 1 )) will also transfer societal biases while transferring information from the teacher model. to address this problem, we propose to infuse the bias mitigation strategy with knowledge distillation to obtain a less biased and compact model. our bias mitigating strategy is based on the intuition that given a sequence such as 'she works as a' and its counterfactual 'he works as a', a fair lm should generate similar texts. we materialize this intuition by encouraging student lm to learn similar distribution of probabilities for a sequence of tokens and its counterfactual. to this end, we propose two modifications to the base distillation strategy: (a) using counterfactual role reversal to modify token probabilities of the teacher model; and (b) using counterfactual role reversed data for model distillation. we study these two modifications independently and in various combinations foot_1 . ## gender fairness on other tasks it is often expected that different fairness measures designed for different but related tasks would be correlated. however, recently @xcite found that fairness measures for static word embeddings and downstream tasks do not correlate. to this end, we study if our fair text generation models improve fairness on other tasks. ## discussion and limitations mitigating disparity across races. we conducted preliminary experiments to test if the proposed approach can be extended to different race groups. similar to @xcite , we consider race bias manifested via people's names and race-specific tokens across four races common in the us: african, european or white, hispanic & latino, and asian. we construct a many-to-many mapping that maps words referring to a given race to words referring to the other races for the counterfactual generation. the rest of the method remains the same as sec. 4. for fairness evaluation, we use race prompts from bold and regard classifier from @xcite , which evaluates whether the person in the text is portrayed as being 'highly thought of.' results show that the lms obtained with the proposed approach were less biased in treating different races similarly, indicating that the proposed approach can be extended to other nonbinary groups. however, the improvements were not as significant as gender bias mitigation, leaving plenty of scope for improvement left for future work. we describe the results and experiments in more detail in appendix c. ## conclusion we proposed techniques to use counterfactual information during knowledge distillation to mitigate gender bias in lms. in experiments, we show that this approach improves fairness in text generation, but it does not simultaneously enhance fairness on lm embedding and downstream classification task. lms have become the swiss army knife of nlp because modeling next word probabilities can learn versatile models that are effective on many tasks. it was surprising that reducing gender disparity in text generation had little effect on other downstream tasks. this finding underscores the importance of evaluating lm fairness along multiple metrics and tasks.