Don’t Just Clean It, Proxy Clean It: Mitigating Bias by Proxy in Pre-Trained Models (2022)

Abstract: Transformer-based pre-trained models are known to encode societal biases not only in their contextual representations, but also in downstream predictions when fine-tuned on task-specific data.We present D-Bias, an approach that selectively eliminates stereotypical associations (e.g, co-occurrence statistics) at fine-tuning, such that the model doesn’t learn to excessively rely on those signals.D-Bias attenuates biases from both identity words and frequently co-occurring proxies, which we select using pointwise mutual information.We apply D-Bias to a) occupation classification, and b) toxicity classification and find that our approach substantially reduces downstream biases (e.g. by > 60% in toxicity classification, for identities that are most frequently flagged as toxic on online platforms).In addition, we show that D-Bias dramatically improves upon scrubbing, i.e., removing only the identity words in question.We also demonstrate that D-Bias easily extends to multiple identities, and achieves competitive performance with two recently proposed debiasing approaches: R-LACE and INLP.

## introduction one of the most dominant paradigms in natural language processing is the fine-tuning of large language models for downstream tasks @xcite . transformer-based language models pre-trained on massive volumes of text achieve state-of-the-art performance on a wide spectrum of tasks, when fine-tuned on task-specific data @xcite . a major concern of this paradigm, however, is that pre-trained models can learn societal biases from both the pretraining and fine-tuning data. these biases can be expressed in two ways -in a) the contextualized representations of the model themselves, i.e., intrinsic biases @xcite @xcite , and b) its downstream predictions after fine-tuning, i.e., extrinsic or allocational harms @xcite @xcite . there has been extensive research into mitigating bias in pre-trained language models @xcite . prior work has primarily focused on debiasing contextual representations upstream, i.e., before the pre-trained model is fine-tuned @xcite @xcite . however, recent work on the bias-transfer hypothesis foot_0 @xcite reveals that most of the variation in downstream biases can be explained by biased associations in the fine-tuning dataset. this work also shows that variations in upstream bias have little impact on downstream disparities. these results emphasize the importance of debiasing interventions that target biases foot_1 introduced during fine-tuning from taskspecific data. however, @xcite also show that simply scrubbing identity terms foot_2 from the downstream/ fine-tuning data does not work for bias mitigation. scrubbing the fine-tuning data is effective in reducing bias only when the model is not pre-trained (i.e., a model with randomly initialized weights). this finding emphasizes that while an intervention targeted at the fine-tuning data is crucial, pre-training can complicate downstream-focused debiasing interventions. we hypothesize that the ineffectiveness of scrubbing the fine-tuning data on a pre-trained model is because pre-trained models are more sensitive to the bias by proxy problem, at fine-tuning. this problem arises when words that co-occur with an identity term function as proxies for that identity term and therefore, act as an additional source of bias @xcite . typically, a pre-trained model predicts masked tokens or spans from their contexts. during pre-training, sometimes an identity term is masked and the model learns to predict it from any proxy words that happen to appear in the identity's context. similarly, sometimes an identity word is part of the context for predicting a masked proxy word. in this way, we suspect that the pre-training process entangles identity terms with words that could serve as a proxy at fine-tuning. in contrast, a model that is not pre-trained has randomly initialized weights, and therefore, there is no initial entangling of identity words with their proxies. this can explain the effectiveness of simply scrubbing the identity terms, in the absence of pre-training. based on the bias by proxy intuition, to address downstream biases, we present dropout bias associations (d-bias), an approach which selectively drops out or ignores bias-associated words and proxies at fine-tuning, in addition to scrubbing identity words. specifically, d-bias aims to identify words, in the vocabulary of the finetuning data, that are most likely to act as proxies for identity words in the context of a particular downstream task. we identify such proxies based on co-occurrence frequencies using pointwise mutual information @xcite . during fine-tuning, d-bias replaces identity words and a set of the most relevant proxies with [mask] (details in section 2), preventing the model from relying on bias associations and also preventing it from inferring those associations from the proxies. as a result, our approach is simple and relatively straightforward to implement. moreover, it can readily and effectively be extended to multiple identity groups. we experiment with d-bias and other variations of word dropout on two downstream tasks: (multi-class) occupation classification of online biographies @xcite and toxicity classification on wikipedia talk page comments @xcite . we demonstrate that our debiasing approaches are consistently effective in reducing downstream biases compared to fine-tuning without word dropout. improvements are especially dramatic with d-bias on toxicity classification. for certain identity groups that are more vulnerable to unfair censorship on online forums, such as gay and homosexual, d-bias reduces classifier false positive rates (fpr) @xmath0. in contrast, scrubbing these identity terms does not decrease the fpr for these groups. in occupation classification, all of our proposed methods outperform two state-of-the-art debiasing approaches r-lace @xcite and inlp @xcite , in terms of reducing the true positive rate (tpr) gaps between biographies of men and women across occupations. ## methods our hypothesis is centered on addressing proxy behavior in the downstream data, in order to prevent the model from learning or inferring societal stereotypes. for a given identity word, we define proxies as a set of words which frequently co-occur with the identity word, in the fine-tuning vocabulary. we describe our approach to compute these proxy words more formally in section 2.1. our proposed approach is straightforward and aims to find sets of the most relevant proxies corresponding to identity words. accordingly, it consists of three major components. the first step is to analyze the fine-tuning data to identify proxy words that are likely to be associated with identity terms. we describe our approach for identifying proxies in section 2.1. the second step is to employ word dropout strategies, building on those word associations. more precisely, we define word dropout as replacing a word with the [mask] token. 4 our word dropout strategies are based on specific heuristics that utilize information on proxies, to make decisions on whether a word in the fine-tuning vocabulary should be dropped out. we present these strategies in section 2.2. the third step is the actual fine-tuning of the pre-trained model, on a version of the fine-tuning data in which identity words and relevant proxies have been dropped out (e.g., using one of the dropout strategies). we begin with a description of the downstream tasks, in order to refer to concrete examples. ## related work prior work on de-biasing pre-trained language models largely focus on upstream mitigation of intrinsic biases @xcite @xcite . however, recent findings @xcite suggest that debiasing effects from upstream mitigation may not hold downstream, motivating the need for interventions focused on the context of downstream tasks. @xcite @xcite ) explore de-biasing contextual representations by identifying and subtracting away, a linear subspace from the embeddings space. these approaches aim to identify one or several sets of linear subspaces that most accurately describe biased artifacts, such as gender biases. these approaches work with task-specific data, by adapting a linear layer on top of the debiased representations (post fine-tuning, after a bias subspace is subtracted from the representations). however, these approaches largely focus on binary attributes for bias and can often be computationally intensive. these can either require massive text corpora to construct templates for bias subspace generation @xcite or use contrastive learning for debiasing @xcite , also relying on massive external text corpora for creating augmented examples, or require iterative optimization to identify large sets of linear subspaces @xcite ). in contrast, our approach is relatively straightforward to implement, does not make linearity assumptions, and easily extends to multiple identities. ## conclusions in this paper, we show, in the context of two classification tasks, that eliminating proxies for identity words, in the fine-tuning data can substantially reduce downstream biases. our findings underscore the importance of targeted and context-specific debiasing approaches, with a focus on attenuating stereotypical associations in the fine-tuning data.