Unmasking Contextual Stereotypes: Measuring and Mitigating BERT ’s Gender Bias (2020)

Abstract: Contextualized word embeddings have been replacing standard embeddings as the representational knowledge source of choice in NLP systems. Since a variety of biases have previously been found in standard word embeddings, it is crucial to assess biases encoded in their replacements as well. Focusing on BERT (Devlin et al., 2018), we measure gender bias by studying associations between gender-denoting target words and names of professions in English and German, comparing the findings with real-world workforce statistics. We mitigate bias by fine-tuning BERT on the GAP corpus (Webster et al., 2018), after applying Counterfactual Data Substitution (CDS) (Maudslay et al., 2019). We show that our method of measuring bias is appropriate for languages such as English, but not for languages with a rich morphology and gender-marking, such as German. Our results highlight the importance of investigating bias and mitigation techniques cross-linguistically,especially in view of the current emphasis on large-scale, multilingual language models.

## introduction the biases present in the large masses of language data that are used to train natural language processing (nlp) models naturally leak into nlp systems. these systematic biases can have real-life consequences when such systems are e.g. used to rank the resumes of possible candidates for a vacancy in order to aid the hiring decision @xcite ). if, for example, a model does not associate female terms with engineering professions, because these do not often co-occur in the same context in the training corpus, then the system is likely to rank male candidates for an engineering position higher than equally qualified female candidates. as nlp applications reach more and more users directly @xcite , bias in nlp and as well as resulting societal implications, have become an area of research @xcite . the acl conference includes a workshop on ethics in nlp since 2017 @xcite and one that specifically addresses gender bias since 2019 @xcite . the present work contributes to promoting fairness in nlp by exploring methods to measure and mitigate gender bias in bert @xcite , a contextualized word embedding model. its widespread and quick adoption by the research community as the backbone for a variety of tasks calls for an assessment of possible biases encoded in it. ## background and previous work approaches to gender bias in contextualized word embeddings borrow techniques originally developed for standard embedding models. however, they need to rely on sentence contexts since contextualized word representations are conditioned on the sentence the word occurs in. previous research uses either templates @xcite or sentences randomly sampled from a corpus @xcite . @xcite adapt the word embedding association test (weat) @xcite to pooled sentence representations, resulting in the seat (sentence encoder association test). however, the authors express concerns about the validity of this method. @xcite analyze the gender subspace following @xcite , and also classify the vectors of occupation words that occur in the same context with male and female pronouns, using coreference resolution as an extrinsic measure of gender bias. they mitigate bias via counterfactual data augmentation (cda) and neutralization. 1 results show that cda was more effective. @xcite measure gender bias by projection onto the gender direction @xcite as well as clustering and classification, following @xcite . results from these adapted methods show that contextualized embeddings encode biases just like standard word embeddings @xcite . instead of adapting bias measuring methods from standard word embeddings, @xcite exploit the masked language model (mlm), native to bert @xcite . unlike elmo @xcite or gpt-2 @xcite , bert learns contextualized word representations using a masked language modelling objective @xcite , making the model bi-directional. crucially, this makes it possible to obtain the probability of a single token in a sentence. @xcite use the mlm to estimate the probability of a masked, gendered target word being associated with an attribute word in a sentence. this method was shown to capture differences in association across the categories covered by @xcite in an interpretable way. one of the problems of current research in nlp is that most work focuses on english @xcite . methods developed to study gender bias in english do not translate well to languages that have grammatical gender, since grammatical gender can have a veiling effect on the semantics of a word. for example, @xcite find that words with the same grammatical gender were regarded as more similar, as opposed to words that have similar meanings. this can occur, for instance, because gender agreement between articles and adjectives (corbett, 2013) renders the contexts in which nouns with the same grammatical gender occur more similar. @xcite also found that due to this grammatical gender bias, the debiasing method of @xcite was ineffective on italian and german word embeddings. (2019) propose to use cda as a debiasing method for gendermarking languages, because it is a pre-processing method and as such independent from the resulting vectors. the researchers measure gender bias extrinsically by using a neural language model, following @xcite . ## data in line with previous research @xcite @xcite , we measure gender bias in bert using sentence templates. for this purpose we create the bias evaluation corpus with professions (bec-pro), containing english and german sentences built from templates (section 3.2). we also use two previously existing corpora, which are described in section 3.1. ## results ## discussion and conclusions the goal of this work is to measure and mitigate gender bias in english and german bert models @xcite . for measuring gender bias, we use a method first proposed by @xcite : word probabilities taken from the bert language model are used to calculate association bias between a gender-denoting target word and an attribute word, such as a profession. our success in making gender bias in the english bert model visible supports the establishment of the method as a unified metric. moreover, we create the bec-pro (bias evaluation corpus with professions), a template-based corpus set in a professional context, which includes professions from three different statistical groups as well as several male and female person words. with this corpus, which we make available to the community, we contribute to streamlining the visualization of gender bias in other contextualized word embedding models. for mitigating gender bias, we first apply cds @xcite to the gap corpus @xcite and then fine-tune the english bert language model on this corpus. we confirm @xcite finding that cda, or cds in this case, is useful for mitigating gender bias in the english bert model. using professions based on workforce statistics allows for a comparison of bias in the bert language model with real-world data. we find that the english bert language model reflects the real-world bias of the male-and female-typical profession groups through positive pro-typical associations and negative anti-typical associations before fine-tuning. after fine-tuning, we observe a reduction in association only for female person words and female-typical professions, but there is an increase in association in both anti-typical settings. this lends support to the effectiveness of our bias mitigation method. however, we also observe that female person words have higher absolute pre-association values in both the pro-and anti-typical settings, and also show greater changes in post-association. one possible reason could be bert's male bias, which has been previously investigated by @xcite . male person terms seem to have a more stable position in bert, which could cause their probabilities in the model to not vary much depending on the context and make them less susceptible to change through fine-tuning. another explanation for female terms being more affected by fine-tuning could be that the gap corpus contained somewhat more female pronouns and nouns, but especially first names, after cds. fine-tuning on a corpus with a slight surplus of female person words and first names could have made the likelihood of these terms more sensitive to change. in the balanced profession group before fine-tuning, we observe that the bert language model does not only encode biases that reflect real-world data, but also those that are based on stereotypes. despite the fact that all of the balanced professions have an approximately even distribution of male and female employees in the u.s. (bureau of labor statistics, 2020), there is a significantly lower, negative association for female person words before fine-tuning. this signifies that women's visibility in these professions is inhibited, i.e. that women are seen as less likely to carry out such a profession. in general, the associations in the balanced profession group behave similarly to those in the male-typical profession group. thus, unless a profession is typically carried out by women, such as the professions kindergarten teacher or nurse, the default 'worker' is culturally seen as male. our results moreover show that a method that works well for english is not necessarily transferable to other languages. since german is a gender-marking language, the agreement between the grammatical gender of the person word and the profession influences the associations. still, the consistently higher associations of female person words compared to male person words illustrate the linguistic markedness of feminine word forms, as opposed to the default masculine forms, in german. furthermore, the fact that english and german both belong to the germanic language family @xcite highlights that (genetic) linguistic relatedness does not predict the cross-linguistic success of a method. especially for a relatively new model such as bert, developing language-specific methods to assess its limitations is crucial to prevent bias propagation to downstream applications in the language concerned. our lack of success in transferring the method to german emphasizes the need for more typological variety in nlp research as well as language-specific solutions @xcite . naturally, there are a number of limitations of this work. we specifically focus on two, here. firstly, we work with only one very specific english bert model, namely the uncased bert base . there are many more contextualized word embedding models besides bert, such as gpt-2 @xcite or @xcite . moreover, there have been various developments and enhancements of the initial bert model, such as distilbert @xcite , albert @xcite , or roberta @xcite . therefore, future work could focus on gender bias in a variety of models and investigate whether there are common patterns. secondly, the present work extensively relies on choices made by the researchers, due to the templatebased method of measuring bias. on one hand, the method is dependent on curated lists of person words and profession terms, which already introduce human bias @xcite . we tried to partially counteract this bias by basing the choice of professions on recent labor statistics. on the other hand, the words in the templates themselves influence the target likelihood, because word representations in bert are dependent on the entire sentence context @xcite . therefore, future research could include a broader variety of sentences, which could also be randomly sampled.