Recognition of They/Them as Singular Personal Pronouns in Coreference Resolution (2022)

Abstract: As using they/them as personal pronouns becomes increasingly common in English, it is important that coreference resolution systems work as well for individuals who use personal “they” as they do for those who use gendered personal pronouns. We introduce a new benchmark for coreference resolution systems which evaluates singular personal “they” recognition. Using these WinoNB schemas, we evaluate a number of publicly available coreference resolution systems and confirm their bias toward resolving “they” pronouns as plural.

## introduction while singular "they" has been widely used in the english-speaking world as a personal pronoun among nonbinary and other members of the lgbtqia+ community for many years, it has taken time for this usage to be accepted by cisnormative society at large. in particular, professional institutions' acceptance of singular personal "they" has been much slower, but seen some uptake in recent years. the american psychological association began accepting singular "they" as a "self-identified pronoun" in 2019, 1 the same year merriam-webster added "nonbinary they" to their dictionary and made it their word of the year. 2 as a result, english language models trained exclusively on sources such as old newspaper articles are likely not exposed to examples of singular "they" usage, in either the personal or generic setting. even models trained on more recent data may miss this usage entirely if they rely on text that was subjected to conservative style guides. this means even state-of-the-art coreference resolution systems may not have recognition of singular "they." in this work, we examine whether existing coreference resolutions systems are able to correctly 1 via apa style 2 via merriam-webster (1a) the paramedic performed cpr on the passenger even though they knew it was too late. (1b) the paramedic performed cpr on the passenger even though they were already dead. (2a) the paramedics tried to help riley even though they knew it was too late. (2b) the paramedics tried to help riley even though they were already dead. resolve cases of singular personal "they". we find that when given a choice between resolving "they" correctly to a singular, named entity or to a group, current systems overwhelmingly choose to resolve "they" as plural or even choose not to resolve the pronoun at all. we also investigate these systems' recognition of singular generic "they" and find that this is a much more easily recognized use-case for some models. overall, we find that models which have been trained on more contemporary or stylistically varied text that may contain examples of singular "they" to have the best performance in both the personal and generic case. while failing to correctly resolve uses of singular personal "they" may feel like a trivial case to those who do not use it as their personal pronoun, this can cause a number of allocational and representational harms @xcite @xcite . for example, a difference in coreference resolutions system performance between nonbinary people who use they/them pronouns and their peers who use binary personal pronouns would constitute a representational harm. this difference in performance can lead to allocational harms when coreference resolution is used in down-stream tasks such as ranking authors based on citation counts in the bodies of texts @xcite . ## relevant datasets to evaluate coreference resolution systems' understanding of singular personal "they", we follow existing work @xcite that uses winograd schemas to test for gender bias in such systems. the winograd schema challenge (wsc) dataset consists of pairs of sentences in which there are two possible referents for a pronoun @xcite . based on a small edit, the pronoun in each sentence in a pair resolves to the opposite referent. these schemas' resolutions are designed to be obvious to a human reader but require deeper knowledge of the given situation for a model to understand them. beyond the original wsc schemas, @xcite provide a set of definite pronoun resolution schemas (dpr) that focus on complex cases of definite pronouns that require world knowledge. multiple winograd-style datasets exist to benchmark gender bias in coreference resolution systems. winogender schemas @xcite ) use an occupation and a participant as their two possible referents. the correct resolution is clear from commonsense knowledge about what the person with the given occupation should be doing in the scenario. for example, in figure 1 , we know it doesn't make sense for the occupation referent "the paramedic" to be dead. we can see in the pair of paramedic examples how, by editing the circumstance (i.e., someone is already dead vs someone knows it is too late), we can change the correct resolution. these schemas are used to confirm that coreference resolution systems are more likely to choose interpretations that match with occupational gender stereotypes instead of the scenario. winobias schemas @xcite are constructed similarly, though both possible referents are an occupation. these schemas are also split between cases in which the correct resolution can be found using purely syntactic information and cases that contain no syntactic clues, instead requiring deeper knowledge about the circumstance. @xcite used these schemas to show that resolution systems will continue to make interpretations based on gender stereotypes, even when the correct answer can be chosen using only syntactic information. beyond the winograd-style datasets we use in this work, there are a number of other coreference resolution datasets that focus on gender bias. gap @xcite consists of naturallyoccurring ambiguous pronouns. it is balanced between male and female referents, but does not include instances of gender-neutral usage. cao and daumé iii (2020) investigate coreference resolution systems' ability to resolve gender neutral pronouns. their work introduces two datasets: map and gicoref. map removes social gender cues such as gendered pronouns and semantically gendered nouns from the gap dataset. they find that these changes dramatically decrease the accuracy of coreference systems. the gicoref dataset consists of naturally occurring text with examples of pronoun usage that are less common in prior datasets such as personal singular "they", neopronouns, and switching pronouns throughout a document. they find that coreference systems still have opportunity for improvement on this dataset, especially in the case of neopronouns. while their datasets contain ambiguous cases of singular personal "they", they do not explicitly test for coreferece resolution systems' understanding of the singular vs plural personal case. our work is complementary to cao and daumé iii (2020) as we focus on controlled experiments with constructed schemas ( §3) rather than uncontrolled but naturally occurring text. ## winonb schemas winogender and winobias schemas do not consider understanding of singular personal vs plural "they". while winogender schemas do include sentences using singular "they", their two possible resolutions were both to individual referents. this means the tested systems had no choice to resolve "they" as a plural pronoun. additionally, neither the occupation nor the participant referent in winogender schemas is a specific, named person. we would like to consider understanding of singular "they" when used as a personal pronoun. to do this, we modify appropriate schemas by hand from the winogender, winobias, wsc, and dpr datasets to create "winonb schemas". to modify winogender schemas into winonb schemas, we begin by changing the occupation referent to be a group of people and the participant to be a named individual. we will not use occupations to test for gender bias as @xcite did, but instead use them to make the scenario and the correct resolution more clear. in example 2a of figure 1 , it is sensible for the people with medical training to "know it was too late", but this could be more ambiguous if neither riley nor the group explained to have medical training. we further edit the schemas as necessary to ensure that the resulting sentences make sense. for instance, in figure 1 , we can see that it wouldn't have made sense for multiple paramedics to "perform cpr". instead, we have the paramedics "try to help" the individual. we also had many cases in which using "the [occupation]s" was confusing as it sounded as though a group of people with the occupation were acting (or being acted upon) together. for instance, in "the physicians warned riley that they needed to get more rest," it sounds as though a group of physicians are all giving riley advice at once. in these cases, we change "the physicians" out for "several physicians" so the sentence can be interpreted as riley getting the advice from different physicians on multiple occasions. we apply this same methodology to the wino-bias, wsc, and dpr datasets. we exclude examples with non-human referents (such as "the dog chased the cat, which ran up a tree. it waited at the {bottom/top}") and examples in which the scenario will not make sense with a singular and plural referent. this left 4077 templates that can be filled with individuals' names. the split between cases in which the pronoun should be interpreted as singular or plural was not perfectly even (with one more plural case than singular) as some datasets provided more than two possible predicates for a handful of scenarios. as our analysis will consider accuracy on the set of plural and singular cases individually, this slight imbalance will not affect the result. the authors performed the edits to turn the existing winogender, winobias, wsc, and dpr schemas into winonb schemas manually. a random subset of 100 of the templates (25 from each source) were verified by a fluent english speaker who resolved the pronouns with 96% accuracy. ## results and discussion we evaluate a representative but not exhaustive set of five coreference resolution systems on our winonb schemas. first, we use clark and manning (2016)'s deep reinforcement learning system, which we will refer to as c&m. we call @xcite 's model with attention-based span representation refinement end-to-end. both of these models were trained on the conll-2012 dataset @xcite which consists of coreference resolution problems in ontonotes 5.0 @xcite . ontonotes contains text from sources such as newswire and magazines. we will also evaluate hugging face's model, which we call c&m++, which builds on c&m @xcite . c&m++ was also trained on ontonotes. finally, we will evaluate bert @xcite and span-bert @xcite , whose original pre-training was done on the bookscorpus @xcite and english wikipedia with finetuning on ontonotes. 4.1 singular personal "they" vs plural "they" first, we consider how well each of our models performs on the examples using singular personal vs plural "they". as we can see in figures 2a and 2b , the difference in performance is stark. on average, the models are 94.8% less likely to correctly resolve a winonb schema that uses singular personal "they" than one that uses plural "they". the different datasets did not exhibit a consistent ordering of difficulty across models. using a friedman test @xcite , we find that the ranking of the accuracies of each model over all datasets are not significantly different at the @xmath0.05 level. however, using paired t-tests, we find that some models have significantly different performances (@xmath1.05) on individual schema sources. we find that bert-base (which achieved about 3× the average accuracy on the singular personal case) has significantly better performance on three of the datasets, and its results are not significantly different from the other models on the dataset in which it underperforms. the three non-bertbased models were all trained on ontonotes 5.0, a dataset that is unlikely to contain instances of singular personal "they". sampling 100 sentences from ontonotes 5.0 that contain a they/them pronoun, we found no cases of singular personal "they". 3 while the bert models were fine-tuned 3 we did find cases where of singular "they" referring to an instiution such as "kraft foods". in many cases, the number of the pronoun was ambiguous given without larger context of the full document. as the ontonotes entries from web data consist only of single sentences @xcite , some of these examples' number could not be determined, but there was no evidence to suggest that any were intended to be on ontonotes, their pre-training on the bookscorpus and wikipedia may have contained singular personal "they" with wikipedia officially allowing and encouraging the use of singular "they" since at least 2017. 4 from this, we may speculate that the bert models' relative success in handling singular personal "they" comes from exposure to the concept during pre-training. while span-bert would have received similar exposure during pre-training, span-bert-base achieved a significantly lower accuracy than bertbase on most winonb datasets. since @xcite reported that span-bert outperforms bert on ontonotes, we speculate that span-bert's lower winonb accuracy comes from optimizing for ontonotes, which may require a deprioritization of any knowledge of singular personal "they" gleaned from pre-training. bert's increased accuracy may also have its drawbacks. while figure 2a focuses solely on how many pronouns were correctly resolved as singular, many of the mistaken examples were either resolved to an unrelated entity or received no resolution at all. bert-base failed to resolve 5.15% of singular personal "they" examples to the group or the individual. this rate is more than double the average rate of such failures over all models. ## conclusion we have introduced "winonb schemas", a set of pronoun resolution pairs that test recognition of singular personal, singular generic, and plural "they" in english coreference resolution. these schemas are adapted from four existing sets of winograd schemas. testing on five publicly available off-theshelf coreference models, we demonstrated that current models largely do not interpret "they" as a singular personal pronoun, though they are more likely to accept singular generic "they". we infer that this is due to popular training datasets largely containing text from times and settings in which singular personal "they" is unlikely to have been used. we find that bert models, which may have seen cases of singular personal "they" during pretraining, are most able to solve winonb schemas. winonb schemas can only demonstrate the existence of bias against nonbinary people, not its absence. our methodology relies on "they" having both singular and plural usages, so it cannot be used to test for understanding of neopronouns such as xe/xem/xyr. a coreference resolution sys-tem can handle winonb schemas well and still perform poorly for members of the nonbinary community who use neopronouns or switch between multiple sets of pronouns. still, these schemas can serve as a jumping-off point for creating and evaluating future models that better serve the nonbinary community. yukun zhu, ryan kiros, rich zemel, ruslan salakhutdinov, raquel urtasun, antonio torralba, and sanja fidler. 2015. aligning books and movies: towards story-like visual explanations by watching movies and reading books. in proceedings of the ieee international conference on computer vision, pages 19-27.