W ino P ron: Revisiting E nglish W inogender Schemas for Consistency, Coverage, and Grammatical Case (2024)

Abstract: While measuring bias and robustness in coreference resolution are important goals, such measurements are only as good as the tools we use to measure them. Winogender Schemas (Rudinger et al., 2018) are an influential dataset proposed to evaluate gender bias in coreference resolution, but a closer look reveals issues with the data that compromise its use for reliable evaluation, including treating different pronominal forms as equivalent, violations of template constraints, and typographical errors. We identify these issues and fix them, contributing a new dataset: WinoPron. Using WinoPron, we evaluate two state-of-the-art supervised coreference resolution systems, SpanBERT, and five sizes of FLAN-T5, and demonstrate that accusative pronouns are harder to resolve for all models. We also propose a new method to evaluate pronominal bias in coreference resolution that goes beyond the binary. With this method, we also show that bias characteristics vary not just across pronoun sets (e.g., he vs. she ), but also across surface forms of those sets (e.g., him vs. his ).

## introduction third-person pronouns (he, she, they, etc.) help us refer to people in conversation. since they mark referential gender in english, gender bias affects how coreference resolution systems map these pronouns to people. @xcite demonstrated this by introducing winogender schemas, a challenge dataset to evaluate occupational gender bias in coreference resolution systems. the dataset has become popular due to its careful construction; it has been translated to other languages @xcite and used in framings beyond coreference resolution, e.g., to evaluate natural language inferences @xcite and intrinsic bias in language models @xcite . however, a closer look at the dataset reveals weaknesses that compromise its use for reliable fix #1: add 2 missing grammatical cases (1a) the counselor told the patient that he/she/they/xe had a professional duty to report certain issues. winopron (ours) @xcite the counselor disclosed to the patient that he/she/they was professionally mandated to report certain issues. winogender schemas @xcite fix #3: ensure templates support all pronouns evaluation (see figure 1 ), which we hypothesize would affect both performance and bias evaluation. in this paper, we identify issues with the original dataset and fix them to create a new dataset we call winopron ( §3). 1 we then empirically show how our fixes affect coreference resolution system performance ( §4) as well as bias ( §5), with a novel method we propose to evaluate pronominal bias in coreference resolution that goes beyond the binary and focuses on linguistic rather than social gender (cao and @xcite . ## background: winogender schemas winogender schemas @xcite are a widely-used dataset consisting of paired sentence templates in english, with slots for two human entities (an occupation and a participant), and a third person singular pronoun. as figure 2 shows, the second part of each template disambiguates which of the two entities the pronoun uniquely refers to, similar to winograd schemas @xcite . changing the pronoun (e.g., from his to her) maintains the coreference, allowing us to measure whether coreference resolution systems are worse at resolving certain pronouns to certain entities. @xcite use the gendered associations of these pronouns to show that gender bias affects coreference resolution performance. the entities consist of 60 occupation-participant pairs (e.g., accountant is paired with taxpayer). a pair of templates is created for each occupationparticipant pair, resulting in a total of 120 unique templates. the template pairs are designed to be parallel until the pronoun, such that only the ending can be used to disambiguate how to resolve the pronoun: it should resolve to the occupation in one template, and to the participant in the other. each template can be instantiated with three pronoun sets (he, she, and singular they), for a total of 120 x 3 = 360 sentences for evaluation. ## winopron dataset although winogender schemas are established in the coreference resolution literature, we find issues with the dataset that compromise its use for reliable evaluation (see figure 1 for examples). we first motivate these issues and our fixes, and then describe how we create and systematically validate our new dataset, winopron. we mostly reuse the occupation-participant pairs from winogender schemas (see appendix a for the full list of pairings), but add 240 templates to cover missing grammatical cases, for a total of 360 templates. we also include a neopronoun set (xe/xem/xyr), giving us 360 templates x 4 pronoun @xmath0,440 sentences for evaluation. ## performance and consistency to demonstrate the effects of our changes, we evaluate performance and consistency on winopron with a range of models with different levels of training for coreference resolution. ## pronominal bias so far, we have focused on coreference resolution performance and consistency and found that accusative forms and less frequent pronoun sets are harder, and models are mostly non-robust to pronominal variation and antecedent disambiguation. however, we have not established the extent to which models fail because they simply cannot perform the task, or if they are over-resolving a pronoun to a particular antecedent due to biased associations between them. thus, we aim to disentangle performance and bias in this section. winogender schemas were originally proposed to measure gender bias in coreference resolution by using pronouns (a form of lexical gender) as a proxy for social gender. @xcite then correlate incorrect resolution of english masculine and feminine pronouns with occupational statistics from the usa. by conflating lexical and social gender (see cao and daumé @xcite for a critical discussion), their analysis is subject to the same limitations as their data: treating different grammatical cases of the same pronoun as equivalent, and focusing only on he and she. we thus propose a new method for evaluating pronominal bias in coreference resolution, correcting for these issues, and we then apply our method to investigate bias in spanbert models on winopron. ## discussion by systematically identifying and fixing issues with winogender schemas @xcite , we create a new dataset, winopron, and find that: second, we echo prior calls for fairness researchers to attend to the differences between social gender and terms that index it (cao and @xcite , to include more diversity in pronouns @xcite @xcite , and to move towards richer operationalizations of gender @xcite and bias @xcite . specifically, future work on bias in coreference resolution should treat pronominal bias as distinct from (social) gender bias, defend how and why pronouns are mapped to social gender, and move beyond binary, oppositional methods of evaluation. lastly, as our work is a case study in how careful data curation and operationalization affects claims about system performance and bias, we emphasize the need for thoughtful data work @xcite , and encourage the use of automatic checks when feasible, as in our work. ## related work besides @xcite , there are a number of papers that tackle gender bias in coreference resolution, all of which differ from ours. similar to winogender schemas, winobias @xcite proposes winograd-like schemas that focus on occupations to evaluate gender bias in coreference resolution. however, winobias only covers he and she, rather than our coverage of all english pronoun sets by design. in addition, like winogender, wino-bias also treats pronouns in all grammatical cases the same way. winonb schemas @xcite evaluate how coreference resolution systems handle singular they and plural they with similar schemas. beyond these constructed schemas, there also exist datasets of challenging sentences found "in the wild," such as bug @xcite , @xcite , and gi-coref (cao and @xcite . however, as these natural datasets are not carefully constructed like winograd-like schemas, pronouns cannot be swapped in dataset instances and still be assumed to be grammatical or coherent. our work is also one among several papers that investigate datasets for problems including low quality or noisy data @xcite @xcite @xcite , contamination @xcite , and issues with conceptualization and operationalization of bias @xcite @xcite @xcite . we cover many of these areas, but do not control for dataset artifacts, which we explain further in our limitations section. ## conclusion we demonstrate a number of issues with the wellknown winogender schemas dataset, which we fix in our new, expanded winopron dataset. in addition, we propose a novel way to evaluate pronominal bias in coreference resolution that goes beyond the binary and focuses on lexical gender. with our new dataset, we evaluate both supervised coreference resolution systems and language models, and find that the grammatical case of pronouns affects model performance and bias, and that bias varies widely across models, pronoun sets and grammatical cases. our work demonstrates that measurements of bias and robustness are only as good as the datasets and metrics we use to measure them, and we call for careful attention when developing future resources for evaluating bias and coreference resolution, with attention to grammatical case, more careful operationalizations of bias, and greater diversity in the pronouns we consider.