Type B Reflexivization as an Unambiguous Testbed for Multilingual Multi-Task Gender Bias (2020)

Abstract: The one-sided focus on English in previous studies of gender bias in NLP misses out on opportunities in other languages: English challenge datasets such as GAP and WinoGender highlight model preferences that are “hallucinatory”, e.g., disambiguating gender-ambiguous occurrences of ‘doctor’ as male doctors. We show that for languages with type B reflexivization, e.g., Swedish and Russian, we can construct multi-task challenge datasets for detecting gender bias that lead to unambiguously wrong model predictions: In these languages, the direct translation of ‘the doctor removed his mask’ is not ambiguous between a coreferential reading and a disjoint reading. Instead, the coreferential reading requires a non-gendered pronoun, and the gendered, possessive pronouns are anti-reflexive. We present a multilingual, multi-task challenge dataset, which spans four languages and four NLP tasks and focuses only on this phenomenon. We find evidence for gender bias across all task-language combinations and correlate model bias with national labor market statistics.

## introduction a reflexive pronoun is an anaphor that requires a c-commanding antecedent within its binding domain (chomsky, 1991). @xcite in languages with type b reflexivization (heine, 2005), the referent of a reflexive possessive pronoun has to be the subject of the clause, while non-reflexive possessive pronouns (so-called anti-reflexives) trigger an interpretation where its referent is not the subject; see we focus on the subset of those languages in which anti-reflexive possessive pronouns are gen- ## the anti-reflexive bias challenge the anti-reflexive bias challenge (abc) dataset is designed to force humans and models to align with either widespread gender assumptions or hard grammatical rules. note, again, that this is in sharp contrast with other gender bias challenge datasets, where gender biases lead to biases in semantic disambiguation, but do not interact with grammatical constraints. our approach is similar to previous work in other respects: similarly to rudinger et al. (2018) and other recent challenge datasets, abc relies on hand-written templates, which are used to generate examples in conjunction with lists of occupations. we make use of the 60 occupations listed in caliskan et al. (2017) containing statistics about gender distributions across professions, taken from the u.s. bureau of labor statistics. specifically, we generate a base set of 4,560 sentences from 38 templates, two tenses (present and past), and 60 occupations. the 38 templates vary the position of the pronouns, e.g.: in our language modelling experiments, we predict the pronoun in question. for nli and coreference, we introduce three variations of each datapoint (possessive masculine, possessive feminine (anti-reflexive) pronouns and the non-gendered reflexive pronoun). this leads to a total of 13,680 examples for each language. for nli, we use these as premises and add possible entailments to our templates. see examples (1) and (2). for machine translation, we use the english versions of examples (3) and (4) as source sentences, with feminine and masculine third-person pronouns. this leads to 9,120 translation problems. native speakers manually verified and corrected all templates and sample examples for all tasks. appendix a shows examples from the four tasks in the four languages. we discuss each task in detail below. nli examples (1) and (2) illustrate the entailment phenomenon that we are interested in. reflexive possessive pronouns are coreferential with their subjects, which leads to the interpretation that the book is on the surgeon's table. anti-reflexive pronouns, on the other hand, prevent this reading and leads to an interpretation that a new discourse entity -another person -exists and that the book is located on that person's table. the general form of our inference examples is as follows: we will primarily be interested in the rate at which state-of-the-art nli models (wrongly) predict examples of the form in example (5) to be cases of entailment, and how this depends on whether the possessive pronoun pron.poss is masculine or feminine. to generate examples of this form, we translate one prototype example and then identify the variables in the output example. we also make sure to check that there are no morpho-syntactic dependencies, e.g., agreement, between these variables. we then generate all possible examples and have native speakers manually verify the correctness of samples of the generated examples. machine translation for machine translation, we are interested in the way that gender assumptions play a role in the resolution of the gendered possessive pronoun in the source language. as an example, when translating the phrase the doctor put the book on her table, an english-danish translation system would likely generate one of the following two options, a reflexive reading and an anti-reflexive one: while abc focuses on translating from english, it holds that similarly, if we translate the danish sentence mekanikeren har brug for sine.refl vaerktøjer til at arbejde, which uses a genderneutral reflexive possessive pronoun sine, into english, the model will have to choose between two possible, correct translations: the machine translation section of the abc dataset consists of translations from english sentences with gendered possessive pronouns into one of the four target languages (danish, russian, swedish, and chinese). for a single occupation on the list, this would correspond to two english sentences (masculine and feminine possessive pronoun) per template. we quantify to what extent models translate english source sentences with possessive masculine or feminine pronouns into target sentences with reflexive pronouns. @xcite coreference resolution for coreference resolution, we generate variants of our templates in the four target languages with each of the gendered anti-reflexives and the reflexive pronoun. that is , for a sentence such as: in examples ( 12 ) and ( 13 ), the use of antireflexive pronouns hans or the femine anti-reflexive hendes means the shoes placed in the closet belong to someone other than the firefighter. in our coreference resolution experiments, we are thus interested in how often models wrongly link the anti-reflexive pronouns (hans/hendes) to the occupation. such predictions violate grammatical constraints and are clear examples of gender assumptions overwriting morpho-syntactic evidence. language modelling for language modeling, we are interested in how likely the models are to predict gendered anti-reflexive possessive pronoun when the original sentence contains a reflexive pronoun. in: we compute the sentence perplexity replacing the reflexive pronoun sine with a feminine @xcite in the context of examples such as example ( 9 ) and ( 10 ), using an anti-reflexive pronoun in the target translation may seem more like a hallucination than violating grammatical constraints, and we acknowledge that in machine translation, as well as in language modeling, the difference concerning existing gender bias challenge datasets is less pronounced than with nli and coreference resolution. nevertheless, note that the model not only hallucinates a gender attribution, but also co-referentiality, making it relatively simple to construct semantically impossible examples, e.g., the mechanic needs his tools, but not his own tools. furthermore, introducing a new referent without evidence also violates pragmatic economy principles (grosz et al., 1995; gardent and webber, 2001). google translate incorrectly translates into a sentence with two reflexive pronouns (violating the semantic principle of bivalence). in this work, we are focused on highlighting a linguistic phenomenon that is useful for diagnosing gender bias, therefore we do not focus on an extensive comparison of model architectures; further work would be required to examine more models. we are interested in the gender associations that existing models make. because of this, we take offthe-shelf translation models and language models. as there were not any state-of-the-art models already pre-trained for coreference in the languages of interest, we train a state-of-the-art architecture for coreference resolution on languages where we could obtain data. to be able to evaluate nli models on the target languages, we fine-tune a pretrained model for this task. as previously found in (rudinger et al., 2018), gender biases in models tended to correlate with labor statistics of the percentage of female in each occupation according to bureau of labor statistics 2015 foot_6 released with caliskan et al. (2017). we correlate our findings with these statistics as well as national statistics. nli nli is originally a three-way classification task. given two sentences; a premise and a hypothesis, the system classifies the relation between them as entailment, contradiction, or neutral. since abc is only intended for diagnosing gender bias in off-the-shelf models, and not for training models, we only consider the entailment relation. if the premise contains a reflexive pronoun, the true class is entailment, and if the premise contains a masculine or feminine pronoun it is not entailment. xnli (conneau et al., 2018) is a manual translation of the english nli data into 15 languages. chinese and russian are among them and we benchmark the model on the xnli test set. singh et al. (2019) extend the xnli train set to a wider set of languages, including danish and swedish but there is not test set for benchmarking. we use cross-lingual language model pre-training (xli) (conneau and lample, 2019), i.e., we fine-tune on english nli training data. for chinese and russian, we use a publicly available implementation foot_7 of the xlm-15 model (conneau and lample, 2019) and fine-tune it using a batch size of 4 and a learning rate of 0.000005 for 35 epochs, which led to the best performance on the xnli development set. for danish and swedish, we use the xlm-100 model, which we fine-tune for 28 epochs. machine translation for machine translation, we evaluate models for @xmath0, russian, swedish, chinese} to assess how often they predict the non-gendered reflexive possessive pronouns when the source possessive pronoun is masculine versus feminine. for all languages, we report the performance of google translate. additionally, for the languages where an off-the-shelf, near-stateof-the-art system was publicly available, we also report performance. for chinese, we use the pretrained models provided by sennrich et al. (2017) foot_8 (e-wmt). for russian, we use the winner system of wmt19 (ng et al., 2019), which is provided as part of the fairseq toolkit (f-wmt). @xcite coreference resolution for coreference resolution, we are interested in whether the system violates grammatical rules by placing an anti-reflexive possesive pronoun in a cluster. we train coreference resolution models for chinese and russian using the model and code of joshi et al. (2019). for chinese, we use the chinese version of ontonotes as our training data, which is made up of about 1800 documents for training. for russian, we use the rucor corpus (ju et al., 2014), which is small, containing only 181 documents total, but has been used to train coreference models for russian before (ju et al., 2014; sysoev et al., 2017). the task consists of predicting the spans that make up a coreference cluster. we train the model using the hyperparameters specified in the source code @xcite . we use a maximum segment length of 128. see appendix §b for statistics of the coreference resolution datasets used for training. while we do not have coreference resolution systems we can evalu-ate for danish and swedish, we include challenge examples for these languages that can be used to detect bias in future systems for these languages. language modeling for our language modeling experiments, we use the pre-trained bert masked language modeling architecture (devlin et al., 2019). we turn pronoun prediction into a cloze task (taylor, 1953) where the pronoun is masked and then the probabilities of each possible alternative taken to compute the sentence-level perplexity. we use chinese bert (for chinese) and multilingual bert for russian, danish, and swedish. @xcite the overall perplexities of these models on our challenge examples, are low; again, this is because of the simple vocabulary and constructions used in the examples. we nevertheless see strong gender bias in the language models, especially for danish and chinese. ## results our evaluation results are found in nli for nli, the xlm models generally overpredict entailment for anti-reflexive pronouns. the models perform well on benchmark data, e.g., 0.742 on the chinese xnli test set, but much worse (0.330) on our challenge examples. for chinese and danish, the models perform slightly better on sentences with masculine anti-reflexive pronouns, whereas they perform slightly better on sentences with feminine anti-reflexives in russian and swedish. for all four languages, we see significant negative correlations between relative error increase on sentences with feminine pronouns and the ratio of women in corresponding occupations; see §5 for a discussion of the statistics. this suggests that the very poor performance numbers on sentences with anti-reflexive pronouns is, in part, the result of gender bias. ## analysis: biased statistics? we used occupations from caliskan et al. (2017) in creating our template data; this database also includes u.s. occupation statistics. in our results in our danish labor market statistics come from larsen et al. (2016), as well as statistics denmark foot_12 and bevaegelsesregisteret, @xcite which is a national database over authorised health staff. some numbers (paramedic, scientist and receptionist) are based on graduation statistics. the russian labor market statistics were mostly obtained from the federal state statistic service. @xcite for occupations not contained on this website we obtained the numbers from separate sources such as the center of fire statistics (cfs) of international association of fire and rescue services (ctif) @xcite and the organisation for economic cooperation and development's statistics website @xcite . we obtain most of our swedish labor market statistics from statistics sweden (scb). @xcite we use the most recent statistic from 2017, which considers people aged 16-64 (eriksson and nguyen, 2019). for clerk and worker, we found labor market statistics in scb (2018). for medical jobs, we used member statistics by swedish medical association (slf) from 2016. @xcite finally, we obtain statistics for china from national bureau of statistics ( 2004 ), which is based on census data from 2000. @xcite while labor statistics correlate strongly across countries ( ## related work the abc dataset is not first to focus on pronouns and gender bias. the ud english-pronouns 23 (munro, 2020), a manually constructed, genderbalanced benchmark of english sentences with pronouns, was motivated by the observation that the genitive pronoun hers only occurs three times in the english universal dependencies (nivre et al., 2016). the gendered, ambiguous pronoun (gap) dataset (webster et al., 2018) is a coreference resolution dataset of human-annotated ambiguous pronoun-name examples from english wikipedia. prates et al. (2018) constructed a translation challenge dataset of simple sentences in gender-neutral languages such as hungarian and yoruba and english target sentences such as he/she is an engineer to estimate gender biases in machine translation. both these challenge datasets focus on gender hallucinations, not unambiguous errors induced by gender bias. some of our examples share similarities with the english winogender schema (rudinger et al., 2018). consider the following minimal pair of winograd schema taken from their paper: in the winograd schema, the context, i.e., the second clause, is supposed to disambiguate the pronoun on semantic grounds. in example (16), the pronoun refers to the paramedic, because the patient is unlikely to know whether cpr is too late. in example (17), the pronoun refers to the patient, because it is impossible to perform cpr if you are dead. our examples, in contrast, do not disambiguate pronouns on semantic grounds and @xcite universaldependencies.org/ this is why we are interested in reflexive possessive pronouns: they always refer to the subject, and their anti-reflexive counterparts never do, so there is no grammatical ambiguity. the disadvantage with semantic disambiguation, we argue, is that it ultimately becomes a subjective competition of belief biases. it is generally impossible to perform cpr if you are dead, but special cases exist: this construction, however, is less interesting than the reflexive possessive pronominal construction, since in this case, pronouns are always coreferential with the object position, regardless of the pronoun. in sum, the abc challenge dataset is, to the best of our knowledge, the first dataset to focus on cases where gender bias leads to unambiguous errors; it is also the first multilingual, multi-task gender bias challenge dataset, and the first to focus on anti-reflexive pronouns. ## conclusion in this work we have introduced the anti-reflexive bias challenge (abc) dataset for multilingual, multi-task gender bias detection, the first of its kind, including four languages and four tasks: machine translation, natural language inference, coreference resolution and masked language modeling. the abc dataset focuses on a specific linguistic phenomenon that does not occur in english but is found in languages with type b reflexivization: namely, anti-reflexive gendered pronouns. this phenomenon is shown to be useful for exposing unambiguous gender bias, because it quantifies to what extent gender bias leads to prediction errors, in contrast to just unwarranted disambiguations ('hallucinations'). the problem of anti-reflexive gendered pronouns has, to the best of our knowledge, not received attention before in the nlp literature, which tends to focus heavily on english (bender and friedman, 2018). our evaluations of state-of-the-art models across the four tasks generally reveal significant gender biases leading to false predictions. additionally, we find that for some tasks, these associations are more in line with national labor market gender statistics than with u.s. statistics, revealing another way that anglocentric biases can prohibit the detection of gender biases in our models.