Bigger Data or Fairer Data? Augmenting BERT via Active Sampling for Educational Text Classification (2022)

Abstract: Pretrained Language Models (PLMs), though popular, have been diagnosed to encode bias against protected groups in the representations they learn, which may harm the prediction fairness of downstream models. Given that such bias is believed to be related to the amount of demographic information carried in the learned representations, this study aimed to quantify the awareness that a PLM (i.e., BERT) has regarding people’s protected attributes and augment BERT to improve prediction fairness of downstream models by inhibiting this awareness. Specifically, we developed a method to dynamically sample data to continue the pretraining of BERT and enable it to generate representations carrying minimal demographic information, which can be directly used as input to downstream models for fairer predictions. By experimenting on the task of classifying educational forum posts and measuring fairness between students of different gender or first-language backgrounds, we showed that, compared to a baseline without any additional pretraining, our method improved not only fairness (with a maximum improvement of 52.33%) but also accuracy (with a maximum improvement of 2.53%). Our method can be generalized to any PLM and demographic attributes. All the codes used in this study can be accessed via https://github.com/lsha49/FairBERT_deploy .

## introduction pretrained language models (plms) have been increasingly applied to tackle various nlp tasks in recent years @xcite @xcite @xcite . along with the wide application of plms is growing concerns about the bias encoded in the representations generated by these plms @xcite . for instance, @xcite demonstrated that stereotypical associations were encoded in plms when tackling a pronoun resolution task. more importantly, such bias has been demonstrated to be harmful to the prediction fairness of downstream models, i.e., there exists a consistent gap between the prediction accuracy for people of different protected attributes. for example, @xcite showed that the gender-related bias embedded in plms could be propagated to downstream classification tasks in medical scenarios. as a remedy, researchers have endeavored to develop techniques to debias plms. these techniques, more often than not, focused on the finetuning stage when using a plm, e.g., correcting the bias hidden behind the learned representations by removing associations between embedding features and protected attributes during the fine-tuning process or using a protected-attribute-balanced dataset to fine-tune a plm @xcite . it is worth noting that, though assuming that the bias contained in a plm is associated with the amount of demographic information carried in the learned representations, these debiasing techniques oftentimes failed to (i) explicitly quantify the capability of a plm in revealing people's demographic attributes or (ii) depict the relationship between the amount of demographic information contained in the learned representations and the prediction fairness in downstream tasks. inspired by the studies which demonstrated the benefits of using additional task-specific data to continue to pretrain a language model and boost prediction accuracy @xcite @xcite @xcite , in this study, we focused on continuing the pretraining of a language model (i.e., bert @xcite ) with carefully-selected data so as to reduce the amount of demographic information contained in the learned representations and subsequently enhance the downstream predic-tion models in terms of both accuracy and fairness. our rationale is essentially in line with those held in @xcite , i.e., the bias carried in a plm can be potentially reduced by inhibiting a plm's awareness of people's protected attributes. formally, this study was guided by the following research questions: ## discussion and conclusion to debias bert by directly reducing the information related to protected attributes in the learned representations, this study developed a dynamic and fair sampling method to select data to perform additional pretraining to bert, which is capable of significantly inhibiting bert's awareness of protected attributes and subsequently improved both the prediction fairness and accuracy in the downstream task. here, we further elaborated on our study's practical insights and implications and dis-cussed the limitations to be addressed in the future. implications. firstly and most importantly, our study corroborated the findings of previous studies @xcite , i.e., prediction fairness in a downstream task can be greatly enhanced by reducing a plm's awareness of sensitive protected attributes, e.g., the amount of information related to such protected attributes in the learned representations. second, as demonstrated in limitations. firstly, the effectiveness of the proposed fair data sampling method was only validated based on bert and one dataset in the field of education. future studies are needed to replicate the study using other plms or datasets to further validate the presented findings. secondly, we focused on debiasing bert in terms of two protected attributes (i.e., first-language backgrounds and gender) separately. future work may further investigate methods to debias a plm by considering other types of protected attributes or simultaneously taking multiple of them into consideration. lastly, we only experimented with a limited set of values for the parameters used in the proposed method, e.g., @xmath0. in the future, it would be worthwhile to develop methods to automatically determine the best values for such parameters.