Multi-Dimensional Gender Bias Classification (2020)

Abstract: Machine learning models are trained to find patterns in data. NLP models can inadvertently learn socially undesirable patterns when training on gender biased text. In this work, we propose a novel, general framework that decomposes gender bias in text along several pragmatic and semantic dimensions: bias from the gender of the person being spoken about, bias from the gender of the person being spoken to, and bias from the gender of the speaker. Using this fine-grained framework, we automatically annotate eight large scale datasets with gender information. In addition, we collect a new, crowdsourced evaluation benchmark. Distinguishing between gender bias along multiple dimensions enables us to train better and more fine-grained gender bias classifiers. We show our classifiers are valuable for a variety of applications, like controlling for gender bias in generative models, detecting gender bias in arbitrary text, and classifying text as offensive based on its genderedness.

## introduction language is a primary means by which people communicate, express their identities, and categorize themselves and others both explicitly and implicitly. such social information is present in the words we write and, consequently, in the text we use to train our nlp models. in particular, models often can unwittingly learn negative associations about protected groups present in their training data and propagate them. in particular, nlp models often learn to replicate unwanted gender biases present in society @xcite @xcite @xcite @xcite . since unwanted gender biases can affect downstream applications-sometimes even * joint first authors. leading to poor user experiences-understanding and mitigating gender bias is an important step towards making nlp tools and models safer and more equitable. in this paper, we provide a finegrained framework for that purpose, analyze the presence of gender bias in models and data, and empower others by releasing tools that can be further applied to numerous text-based use-cases. while many works have explored methods for removing gender bias from text @xcite @xcite , no extant work on classifying gender or removing gender bias has foregrounded the fact that we use language, at least in part, to collaboratively and socially construct our gender identities. we propose a pragmatic and semantic framework for measuring bias along three dimensions that is sensitive to conversational and performative aspects of gender, as illustrated in figure 1 . recognizing these dimensions is important, because gender along each dimension can affect text differently, for example, by modifying word choice or imposing different preferences on sentence structure. decomposing gender into separate dimensions also allows for better identification of gender bias, which subsequently enables us to train a suite of classifiers for detecting different kinds of gender we use a part-of-speech tagger @xcite , and computed p (word | gender)/p (word) for words that appear more than 500 times. ## related work gender affects myriad aspects of nlp, including corpora, tasks, algorithms, and systems @xcite @xcite . for example, statistical gender biases are rampant in word embeddings @xcite @xcite @xcite @xcite @xcite @xcite @xcite )-including multilingual ones @xcite @xcite -and affect a wide range of downstream tasks including coreference resolution @xcite @xcite , part-ofspeech and dependency parsing @xcite , language modeling @xcite , appropriate turn-taking classification @xcite , relation extraction @xcite , identification of offensive content @xcite , and machine translation @xcite . for dialogue, gender biases in training corpora have been found to be amplified in machine learning models @xcite @xcite . while many of the works cited above proposed methods of mitigating the unwanted effects of gender on text, hall 2020 ) in particular relied on counterfactual data to alter the training distribution to offset gender-based statistical imbalances (see §4.2 for more discussion of training set imbalances). also relevant is @xcite , which introduced a parallel style corpus and showed gains on style-transfer across binary genders. most relevant to this work, @xcite proposed a framework for modeling pragmatic aspects of many social biases in text. our work and theirs focus on complementary aspects of a larger goal-namely, making nlp safe and inclusive for everyone-but the two approaches differ in several ways. we treat statistical gender bias in human or model generated text specifically, and in detail. @xcite proposed a different but compatible perspective, and aimed to situate gender bias within the broader landscape of negative stereotypes in social media text, an approach that can make parallels apparent across different kinds of harmful text. moreover, they considered different pragmatic dimensions than we do: they targeted negatively stereotyped commonsense implications in arguably innocuous statements, whereas we investigate pragmatic dimensions that straightforwardly map to conversational roles (i.e., topics, addressees, and creators of text). finally, when investigating gender biases, one cannot ignore the intersectionality of gender identities, i.e., when gender non-additively interacts with other identity characteristics. negative gender stereotyping is known to be alternatively weakened or reinforced by the presence of social attributes like dialect @xcite , class (degaetano-ortlieb, 2018) and race @xcite . these differences have been found to affect gender classification in images @xcite , and also in sentences encoders @xcite . we acknowledge that these are crucial considerations, and intend to incorporate them in future work. for a thorough survey and a critical discussion of best practices for researching social "biases" in nlp, including and beyond gender, see @xcite . ## dimensions of gender bias gender permeates language differently depending on the conversational role played by the people using that language (see figure 1 ). we decompose gender bias along multiple dimensions: bias when speaking about someone, bias when speaking to someone, and bias from speaking as someone. this framework enables both finer-grained understanding of bias and better classification of gender's effects on text from multiple domains. definition of gender we annotate gender with four potential values: masculine, feminine, neutral and unknown. we take neutral to contain characters with either non-binary gender identity, or an identity which is unspecified for gender by definition (e.g. a talking tree). we include an unknown category for when the gender is genuinely not known. speaking about: gender of the topic. it's well known that we change how we speak about others depending on who they are @xcite , and what their gender identity is @xcite . for example, adjectives which describe women have been shown to differ from those used to describe men in numerous situations @xcite @xcite , as do verbs that take nouns referring to men as opposed to women @xcite . speaking to: gender of the addressee. people often adjust their speech based on who they are speaking with-their addressee(s)-to show solidarity with their audience or to express social dis-tance @xcite @xcite @xcite . we expect the addressee's gender to affect, for example, how a man might communicate with another man about hair styles or beard hygiene. this exchange would probably differ if the man was communicating instead with a woman about the same topic. speaking as: gender of the speaker. people react to content differently depending on who created it. like race, gender is often described as a "fundamental" category for self-identification and selfdescription @xcite , with men, women, and non-binary people differing in how they actively create their own gender identities @xcite . who someone is speaking as strongly affects what they may say and how they say it, down to the level of their choices of adjectives and verbs in self-descriptions @xcite . ## creating gender classifiers in an ideal world, we would expect little difference between texts describing men, women, and people with other gender identities. a machine learning model, then, would be unable to pick up on statistical differences in gendered language (i.e., statistical gender bias), because such differences would not exist. however, gender-based distributional differences do exist in current-day text ( previous work on gender bias classification has been predominantly single-task-often supervised on the task of analogy-and relied mainly on word lists, that are binarily gendered @xcite @xcite )-sometimes also explicitly @xcite based approaches provided a solid start, they ultimately prove insufficient. first, they conflate different conversational dimensions of gender bias, and are therefore unable to detect the subtle, but very well-described, pragmatic differences of interest here. second, most existing gendered word lists are limited to explicitly binarily gendered words (e.g., mom vs. dad). not only is binary gender completely inadequate for the task, but excluding statistically gendered words is problematic-because they are also strong anchors of gender stereotypes @xcite . instead, we develop classifiers that decompose gender bias over sentences into semantic and/or pragmatic dimensions (about/to/as), including gender information that (i) falls outside the malefemale binary, (ii) can be contextually determined, and (iii) is statistically as opposed to explicitly gendered. in the subsequent sections, we provide details regarding the annotation of data, and details for training these classifiers. ## applications we demonstrate the broad utility of our multitask classifier by applying it to three different downstream applications. first, we show that we can use the classifier to control the genderedness of generated text. next, we demonstrate its utility in biased text detection by applying it to wikipedia to find the most gendered biographies. finally, we evaluate our classifier on offensive text to explore the interplay between offensive text and gender. ## conclusion we propose a general framework for analyzing gender bias along three dimensions: (1) gender of the person being spoken about (about), ( 2 ) gender of the addressee (to), and (3) gender of the speaker (as). we annotate eight large existing datasets and create an evaluation dataset for the task of detect bias along each of these dimensions. we train classifiers (single and multitask) that demonstrate their broad utility by displaying strong performance in controlling bias in dialogue, detecting genderedness in text such as wikipedia, and highlighting gender differences in offensive text.