On Transferability of Bias Mitigation Effects in Language Model Fine-Tuning (2021)

Abstract: Fine-tuned language models have been shown to exhibit biases against protected groups in a host of modeling tasks such as text classification and coreference resolution. Previous works focus on detecting these biases, reducing bias in data representations, and using auxiliary training objectives to mitigate bias during fine-tuning. Although these techniques achieve bias reduction for the task and domain at hand, the effects of bias mitigation may not directly transfer to new tasks, requiring additional data collection and customized annotation of sensitive attributes, and re-evaluation of appropriate fairness metrics. We explore the feasibility and benefits of upstream bias mitigation (UBM) for reducing bias on downstream tasks, by first applying bias mitigation to an upstream model through fine-tuning and subsequently using it for downstream fine-tuning. We find, in extensive experiments across hate speech detection, toxicity detection and coreference resolution tasks over various bias factors, that the effects of UBM are indeed transferable to new downstream tasks or domains via fine-tuning, creating less biased downstream models than directly fine-tuning on the downstream task or transferring from a vanilla upstream model. Though challenges remain, we show that UBM promises more efficient and accessible bias mitigation in LM fine-tuning.

## introduction the practice of fine-tuning pretrained language models (ptlms or lms), such as bert @xcite , has improved prediction performance in a wide range of nlp tasks. however, finetuned lms may exhibit biases against certain protected groups (e.g., gender and ethnic minorities), as models may learn to associate certain features with positive or negative labels spuriously @xcite , or propagate bias encoded in ptlms to downstream classifiers @xcite . among many examples, @xcite demonstrates gender-bias in the pronoun resolution task when models are trained using bert embeddings, and @xcite shows that hate speech classifiers finetuned from bert result in more frequent false positive predictions for certain group identifier mentions (e.g., "muslim", "black"). approaches for bias mitigation are mostly applied during fine-tuning to reduce bias in a specific downstream task or dataset @xcite @xcite (see fig. 1 (a)). for example, data augmentation approaches reduce the influence of spurious features in the original dataset @xcite @xcite , and adversarial learning approaches generate debiased data representations that are exclusive to the downstream model @xcite . these techniques act on biases particular to the given dataset, domain, or task, and require new bias mitigation when switching to a new downstream task or dataset. this can require auxiliary training objectives, the definition of task-specific fairness met-rics, the annotation of bias attributes (e.g., identifying african american vernacular english), and the collection of users' demographic data. these drawbacks make bias mitigation inaccessible to the growing community, fine-tuning lms to new datasets and tasks. in contrast, we investigate initially mitigating bias while fine-tuning an "upstream" model in one or more upstream datasets, and subsequently achieving reduced bias when fine-tuning for downstream applications (fig. 1 (d )), so that bias mitigation is no longer required in downstream training. similar to transfer learning for enhancing predictive performance in common setups @xcite , we suggest that lms that undergo bias mitigation acquire inductive bias that is helpful for reducing harmful biases when fine-tuned on new domains and tasks. in four tasks with known bias factors -hate speech detection, toxicity detection, occupation prediction from short bios, and coreference resolution -we explore whether upstream bias mitigation of a lm followed by downstream fine-tuning reduces bias for the downstream model. though previous work has addressed biases in frozen ptlm or word embeddings @xcite @xcite @xcite , for example by measuring associations between gender and occupations in an embedding space, they do not study their effect on downstream classifiers (fig. 1 (b) ), while some of them study the effects while keeping the embeddings frozen @xcite @xcite . bias in these frozen representations can also be directly corrected by removing associations between feature and sensitive attributes @xcite (fig. 1 (c )), but this does not allow predictions to be generated for new data. our experiments address the following research questions: (a) whether mitigating a single bias factor in the upstream stage is maintained when finetuning on new examples from the same domain and task, (b) whether transfer is viable when the downstream domains and tasks are different from the upstream model, and (c) whether we can address multiple kinds of bias with a single upstream model. we perform these experiments under a generic transfer learning framework, noted as upstream bias mitigation (ubm) for downstream fine-tuning for convenience, which consists of two stages: first, in the upstream bias mitigation stage, a lm is fine-tuned with bias mitigation objectives on one or several "upstream" tasks, and subsequently the classification layer is re-initialized; then, in the downstream fine-tuning stage the encoder from the upstream model, jointly with the new classification layer, are again fine-tuned on a downstream task without additional bias mitigation steps. using six datasets with previously recognized bias factors, our analysis show overall positive results for the questions above; still, there are challenges remaining to stabilize the results of bias mitigation in challenging setups, e.g., the multi-bias factor setting. ## exploring the transferability of bias mitigation effects we consider biases against protected groups in classifiers fined-tuned from lms. in our present analysis, bias is defined as disparate model performance on different subsets of data which are associated with different demographic groups (e.g., instances that mention or are generated by different social groups) @xcite . our evaluation of bias aligns with the definition of equalized odds and equal opportunities @xcite in previous works of fairness in machine learning. here, we first outline our experimental setup for exploring the transferability of bias mitigation effects, in which we detail the process of applying ubm and pose three key research questions (section 2.1). we follow by introducing the bias factors studied and the corresponding classification tasks and datasets (section 2.2), and our evaluation protocols and metrics (section 2.3). ## method here, we detail the particular bias mitigation algorithms used for implementing ubm, as well as the other baselines used for verifying the transferability of bias mitigation effects. ## results in this section, we present the results of ubm in three settings following the order in sec. 2.1: transferring to the same data distribution, transferring to different data distributions, transferring from an upstream model with bias mitigation for multiple bias factors. we follow these main analyses with an investigation of the impact of freezing encoder weights before downstream fine-tuning, and lastly with a brief exploration of how ubm's positive results are achieved. ## related works here we review approaches that inform the present work (techniques for bias mitigation) and are related to the basic idea of ubm. ## conclusion we observe that the effects of bias mitigation are indeed transferable in fine-tuning lms. future works in fine-tuning lms can use ubm in order to easily apply the positive effects of bias mitigation methods to new domains and tasks without customized bias mitigation processes or access to sensitive user information. though ubm does not rival directly mitigating bias on the downstream task, it is more efficient and accessible. future works can develop the effectiveness of ubm beyond the default scenarios in this paper, and potentially apply it to tasks and settings beyond hate speech, toxicity classification, occupation prediction, and coreference resolution in english corpora.