Don’t Forget About Pronouns: Removing Gender Bias in Language Models Without Losing Factual Gender Information (2022)

Abstract: The representations in large language models contain multiple types of gender information. We focus on two types of such signals in English texts: factual gender information, which is a grammatical or semantic property, and gender bias, which is the correlation between a word and specific gender. We can disentangle the model’s embeddings and identify components encoding both types of information with probing. We aim to diminish the stereotypical bias in the representations while preserving the factual gender signal. Our filtering method shows that it is possible to decrease the bias of gender-neutral profession names without significant deterioration of language modeling capabilities. The findings can be applied to language generation to mitigate reliance on stereotypes while preserving gender agreement in coreferences.

## introduction neural networks are successfully applied in natural language processing. while they achieve stateof-the-art results on various tasks, their decision process is not yet fully explained @xcite . it is often the case that neural networks base their prediction on spurious correlations learned from large uncurated datasets. an example of such a spurious tendency is gender bias. even the state-of-theart models tend to counterfactually associate some words with a specific gender @xcite . the representations of profession names tend to be closely connected with the stereotypical gender of their holders. when the model encounters the word "nurse", it will tend to use female pronouns ("she", "her") when referring to this person in the generated text. this tendency is reversed for words such as "doctor", "professor", or "programmer", which are male-biased. figure 1 : a schema is presenting the distinction between gender bias of nouns and factual (i.e., grammatical) gender in pronouns. we want to transform the representations to mitigate the former and preserve the latter. it means that the neural model is not reliable enough to be applied in high-stakes language processing tasks such as connecting job offers to applicants' cvs @xcite . if the underlying model was biased, the high-paying jobs, which are stereotypically associated with men, could be inaccessible for female candidates. when we decide to use language models for that purpose, the key challenge is to ensure that their predictions are fair. the recent works on the topics aimed to diminish the role of gender bias by feeding examples of unbiased text and training the network (de vassimon @xcite or transforming the representations of the neural networks post-hoc (without additional training) @xcite . however, those works relied on the notion that to de-bias representation, most gender signal needs to be eliminated. it is not always the case, pronouns and a few other words (e.g.:"king" -"queen"; "boy" -"girl") have factual information about gender. a few works identified gendered words and exempted them from @xcite . in contrast to these approaches, we focus on contextual word embeddings. in contextual representations, we want to preserve the factual gender information for genderneutral words when it is indicated by context, e.g., personal pronoun. this sort of information needs to be maintained in the representations. in language modeling, the network needs to be consistent about the gender of a person if it was revealed earlier in the text. the model's ability to encode factual gender information is crucial for that purpose. we propose a method for disentangling the factual gender information and gender bias encoded in the representations. we hypothesise that semantic gender information (from pronouns) is encoded in the network distinctly from the stereotypical bias of gender-neutral words (figure 1 ). we apply an orthogonal probe, which proved to be useful, e.g., in separating lexical and syntactic information encoded in the neural model @xcite . then we filter out the bias subspace from the embedding space and keep the subspace encoding factual gender information. we show that this method performs well in both desired properties: decreasing the network's reliance on bias while retaining knowledge about factual gender. ## methods we aim to remove the influence of gender-biased words while keeping the information about factual gender in the sentence given by pronouns. we focus on interactions of gender bias and factual gender information in coreference cues of the following form: in english, we can expect to obtain the factual gender from the pronoun. revealing one of the words in coreference link should impact the prediction of the other. therefore we can name two causal associations: in our method, we will primarily focus on two ways bias and factual gender interact. for genderneutral nouns (in association c i ), the effect on predicting masked pronouns would be primarily correlated with their gender bias. at the same time, the second association is desirable, as it reveals factual gender information and can improve the masked token prediction of a gendered word. we define two conditional probability distributions corresponding to those causal associations: where y is a token predicted in the position of pronoun and noun, respectively; x is the context for masked language modeling. b and f are bias and factual gender factors, respectively. we model the bias factor by using a gender-neutral biased noun. below we present examples for introducing female and male bias: foot_2 example 1: we aim to diminish the role of bias in the prediction of pronouns of a specific gender. on the other hand, the gender indicated in pronouns can be useful in the prediction of a gendered noun. mathematically speaking, we want to drop the conditionality on bias factor in p i from eq. ( 1 ), while keeping the conditionality on gender factor in p ii . to decrease the effect of gender signal from the words other than pronoun and noun, we introduce a baseline, where both pronoun and noun tokens are masked: example 3: ## experiments and results we examine the representation of two bert models (base-cased: 12 layers, 768 embedding size; and large-cased: 24 layers, 1024 embedding size, @xcite ), and electra (base-generator: 12 layers, 256 embedding size @xcite ). all the models are transformer encoders trained on the masked language modeling objective. ## related work in recent years, much focus was put on evaluating and countering bias in language representations or word embeddings. @xcite observed the distribution of word2vec embeddings @xcite encode gender bias. they tried to diminish its role by projecting the embeddings along the so-called gender direction, which separates gendered words such as he and she. they measure the bias as cosine similarity between an embedding and the gender direction. @xcite propose a method to diminish differentiation of word representations in the gender dimension during training of the glove embeddings @xcite . nevertheless, the following analysis of @xcite argued that these approaches remove bias only partially and showed that bias is encoded in the multidimensional subspace of the embedding space. the issue can be resolved by projecting in multiple dimensions to further nullify the role of gender in the representations @xcite . dropping all the gender-related information, e.g., the distinction between feminine and masculine pronouns can be detrimental to gender-sensitive applications. @xcite proposed a de-biasing algorithm that preserves gendered information in gendered words. unlike the approaches above, we work with contextual embeddings of language models. @xcite investigated bias in the representation of the contextual model (gpt-2, @xcite ). they used causal mediation analysis to identify components of the model responsible for encoding bias. @xcite and @xcite propose a method of evaluating bias (including gender) with counterfactual test examples, to some extent similar to our prompts. @xcite and @xcite employ prompts similar to ours to evaluate the gender bias of professional words in language models. the latter work also aims to identify and remove gender subspace in the model. in contrast to our approach, they do not guard factual gender signal. recently, @xcite summarized the research on the evaluation and mitigation of gender bias in the survey of 304 papers. ## conclusions we propose a new insight into gender information in contextual language representations. in debiasing, we focus on the trade-off between removing stereotypical bias while preserving the semantic and grammatical information about the gender of a word from its context. our evaluation of gender bias showed that three analyzed masked language models (bert large, bert based, and elec-tra) are biased and skewed toward predicting male gender for profession names. to mitigate this issue, we disentangle stereotypical bias from factual gender information. our filtering method can remove the former to some extent and preserve the latter. as a result, we decrease the bias in predictions of language models without significant deterioration of their performance in masked language modeling task. jieyu zhao, tianlu wang, mark yatskar, vicente ordonez, and kai-wei chang. 2018a. gender bias in coreference resolution: evaluation and debiasing methods. in proceedings of the 2018 conference of the north american chapter of the association for computational linguistics: human language technologies, volume 2 (short papers), pages 15-20, new orleans, louisiana. association for computational linguistics. jieyu zhao, yichao zhou, zeyu li, wei wang, and kai-wei chang. 2018b. learning gender-neutral word embeddings. in proceedings of the 2018 conference on empirical methods in natural language processing, pages 4847-4853, brussels, belgium. association for computational linguistics.