Demographic-Aware Language Model Fine-tuning as a Bias Mitigation Technique (2022)

Abstract: BERT-like language models (LMs), when exposed to large unstructured datasets, are known to learn and sometimes even amplify the biases present in such data. These biases generally reflect social stereotypes with respect to gender, race, age, and others. In this paper, we analyze the variations in gender and racial biases in BERT, a large pre-trained LM, when exposed to different demographic groups. Specifically, we investigate the effect of fine-tuning BERT on text authored by historically disadvantaged demographic groups in comparison to that by advantaged groups. We show that simply by fine-tuning BERT-like LMs on text authored by certain demographic groups can result in the mitigation of social biases in these LMs against various target groups.

## introduction bias is defined as any kind of preference or prejudice of an individual or group, towards another individual or group @xcite . the underlying traits of one's demographic group shape one's thoughts and worldviews @xcite , and therefore may surface in one's language preferences and biases in the day-to-day life. for example, the word admit is more often associated with hospital by indian bloggers, whereas american bloggers associate it with guilt @xcite . most prior work in bias mitigation has largely taken the "one-size-fits-all" approach, with most models being agnostic to the language of the speakers behind the language @xcite @xcite . in this paper, we draw inspiration from previous research that showed the effect of demographic information on nlp tasks, such as word embeddings @xcite , word associations @xcite , empathy prediction @xcite , varied model performance of demographic-aware models @xcite . we hypothesize that biases toward or against a specific group vary based on the demographic lens through which the world is viewed, and analyzing the social biases of various demographic groups from their language use can help uncover their characteristics. we believe such an understanding can move us beyond "one-size-fitsall" models, while at the same time developing demographic-aware bias mitigation techniques. the advent of large pre-trained transformerbased @xcite language models, such as bert @xcite and gpt @xcite , have revolutionized nlp techniques for several downstream tasks @xcite @xcite . however, along with their high performances, they can also inherit the various social biases that may be present in the large unstructured datasets they are exposed to @xcite @xcite . in this paper, we focus on gender (male, female) and racial (african american, european american) biases, and address two main research questions: (1) how do gender and racial biases encoded in bert vary when exposed to language authored by different demographic groups? (2) how do biases in demographic-specific bert models compare to those in vanilla bert? the paper makes two main contributions. first, we present an empirical analysis of gender and racial biases encoded in bert when it is trained on datasets authored by various demographic groups, and show that the biases, as measured by the sentence encoder association test @xcite , vary across different speaker groups under consideration. second, we compare the biases in demographic-specific bert models with those in vanilla bert, and examine the extent to which the biases are either amplified or reduced in bert upon exposure to language authored by specific demographic groups. to our knowledge, this is the first work that explores the effect of speaker demographic information on biases in bert-like language models. ## effect of speaker demographics on biases in bert to investigate the effect of speaker demographic information on biases encoded in bert, we build variants of the pre-trained bert model that are exposed to the language of various demographic groups. we consider the resulting change in the model biases to be a result of the underlying group's bias. specifically, we use the pre-trained bert model and fine-tune it on datasets authored by different demographic groups for masked language modelling (mlm). foot_0 datasets. we use several datasets to fine-tune bert. first, we use gender-aware datasets, to measure gender bias: (i) blogs collected from google blogger @xcite , and (ii) reddit comments @xcite . the former consists of a large set of blog posts collected from google blogger from 1999 to 2016, where the gender information of the bloggers is self-provided in their profiles. the latter consists of publicly available reddit comments from 2007 to 2015; since reddit users do not have profiles with personal information fields that could be scraped, @xcite extracted the demographic attributes of users from self-identification in their text. gender was extracted by searching for statements referring to oneself as a 'boy', 'man', 'male', 'guy', for male (e.g., 'i am a male'), or 'girl', 'woman', 'female', 'gal', for female (e.g., 'i am a female'). we use 50k examples for each gender, randomly sampled from these datasets for fine-tuning bert. second, we use location-aware datasets, covering africa, asia, europe, north america, and oceania, to measure the racial bias held in these regions: (i) geowac, a geographically-balanced gigaword corpus, that consists of web pages from the common crawl @xcite , and (ii) reddit comments @xcite ). 2 in geowac, the language samples are geo-located using country-specific top-level domains; e.g., a web page under the .ca domain is assumed to have originated from canada @xcite . 3 this dataset consists of gigaword corpora for 48 languages, with the english corpus spanning across more than 150 countries. we consider top three countries per region with the highest number of examples, and select around 94k examples (the minimum number of examples for any country) from each of them ( for the reddit dataset, similar to the case in gender, @xcite segregated the comments region-wise based on the usage of phrases such as 'i am from' and 'i live in'. this dataset consists of eight regions, namely africa, asia, canada, europe, mexico, oceania, south america, united kingdom, and united states ( we only consider examples with @xmath0. for each gender and region, we perform fine-tuning five times on each dataset, by randomly sampling the required number of examples for each fold, and use 90:10 ratio to obtain training and validation splits. we report the results averaged on the five folds. further implementation details are provided in appendix. evaluation. social biases are typically measuring using the word embedding association test (weat) @xcite . weat imitates the human implicit association test @xcite for word embeddings, by measuring bias via the association between two sets of target concepts and two sets of attributes. for example, to measure gender bias with respect to career/family, which is a common historical gender bias @xcite , it uses target words such as female, woman and male, man, and attributes such as executive, office and home, children. the bias is determined by the difference between the relative similarity of the target concepts to the two sets of attributes: greater ## results and discussion tables 1 and 2 show the seat effect sizes of bert, and its fine-tuned gender-specific and race-specific variants respectively, averaged over five folds (with absolute values taken for bert and each fold of its variants). while the effect sizes can be positive or negative, we present their absolute values as they indicate the severity of the models' bias toward or against any given group. ## conclusions in this paper, we analyzed gender and racial bias in bert when it is fine-tuned on datasets authored by different demographic groups. we found that 5 given that bert fine-tuning can be unstable due to the randomness in data shuffling and initialization @xcite , there may be slight variations in some of the results if the same experiments are re-run with the same set of hyperparameters and data splits. our aim in this paper is to only highlight the variations in the biases in bert when exposed to language authored by different demographic groups, and bring to attention that sometimes this could lead to bias mitigation in it. bert when exposed to female language exhibits lower gender bias than when it is exposed to male language as measured by the seat effect size with respect to career/family attributes. for european american/african american racial bias, we observed that with one dataset, bert exposed to african language exhibits lower bias, while on another dataset, bert exposed to north american language results in lower bias. we also found that simply fine-tuning bert on mlm tasks with data authored by specific demographic groups can result in bias mitigation in bert, indicating that depending on the lens through which the world is viewed, biases can be lowered in large pre-trained lms. based on these initial findings, we believe further research is warranted in this direction of bias mitigation using demographic data and demographicaware bias mitigation methods.