The S lay QA benchmark of social reasoning: testing gender-inclusive generalization with neopronouns (2024)

Abstract: We introduce SlayQA, a novel benchmark data set designed to evaluate language models’ ability to handle gender-inclusive language, specifically the use of neopronouns, in a question-answering setting. Derived from the Social IQa data set, SlayQA modifies context-question-answer triples to include gender-neutral pronouns, creating a significant linguistic distribution shift in comparison to common pre-training corpora like C4 or Dolma. Our results show that state-of-the-art language models struggle with the challenge, exhibiting small, but noticeable performance drops when answering question containing neopronouns compared to those without.

## introduction currently, the recognition of the importance of inclusivity and representation in nlp is growing @xcite @xcite . traditional data sets often reflect and perpetuate binary gender norms, which can marginalize non-binary and gender nonconforming individuals and cause harm @xcite . this lack of inclusivity highlights a critical need for resources that better represent the full spectrum of gender identities. one aspect of gender-inclusive language use that is gaining more and more acceptance is the usage of neopronouns like xe/xyr or ze/zir. neopronouns are novel pronouns that people who do not identify themselves as belonging to the polar extremes of the gender spectrum can choose to use for reference to themselves instead of the classical, gendered pronouns: he/him and she/her. current benchmarks that assess this kind of linguistic inclusivity mostly focus on the generation of correct neopronouns in context or similar tasks @xcite . on the other hand, resources that simply implement established lm benchmarks in a more inclusive way are rare. to address this gap, we present slayqa: social linguistics analytics yielding queer agents, a novel benchmark set derived from the existing social iqa (siqa) data set @xcite . it contains a situation description (the context), social reasoning questions and three prospective answers, where all context-question-answer pairs include at least two acts of pronoun-based reference and gender-neutral pronouns. because slayqa systematically replaces established, gendered pronouns with gender-affirming neopronouns, it is a more inclusive data set that better reflects the diversity of human identities. although neo-pronouns are commonly rated less grammatical than their established counterparts @xcite , they are beginning to be adopted in various social circles. here, the key word is beginning -they are still infrequent in discourse and and also in common pretraining corpora like c4 @xcite , dolma @xcite , and redpajama-data-1t (together computer, 2023). as such, our pronoun-altered benchmark marks a significant pronoun distribution shift in comparison to these pretraining corpora. consequentially, slayqa helps to assess how well language models are able to generalize to novel linguistic structures. we argue that our benchmark introduces a significant distribution shift between the pretraining corpora and the evaluation data with regard to pronouns. to assess this proposed distribution shift, we determine the frequencies of established and neopronouns in these corpora through the ngram lookup function of what's in my big data? (wimbd) @xcite -if the neopronouns occur less frequently in pretraining corpora than established pronouns, then our neopronoun benchmark introduces a drastic distribution shift in its pronoun distribution compared to these corpora. we adapt our list of pronouns from the seminal study by @xcite . for the sake of brevity, we do not include further gender-affirming pronoun variations like nounself, emojiself, numberself or nameself pronouns @xcite . the frequencies for subject pronouns in both corpora are found in although no n-gram frequency results are available for redpajama-data-1t, we assume that the underlying distribution should be mostly equal to the two examined corpora -all three corpora are mainly based on commoncrawl web dumps, so it is reasonable to expect a large lexical overlap between them. a final indicator for the different pronoun distributions can be found in ## slayqa in the generalisation taxonomy motivation slayqa is both cognitively and fairness-motivated. humans are generally able to use neopronouns correctly and productively. consequently, if language models are indeed good models of human language (usage), they should not struggle with social reasoning that includes neopronouns. additionally, the inclusion and correct processing of neopronouns also relates to fairness of language technologies -they should be applicable to all potential users, even in the light of a changing linguistic and societal landscape (cf. also related work in section 2). generalisation type slayqa assesses whether language models can interpret novel, highly infrequent pronoun forms in social reasoning contexts. this is a test for compositional generalisation, as neopronouns are systematic, productively used, substitutive with regard to the referents they replace, and localist in the sense of only depending on context, question and answer sentences. as such, our benchmark fulfils the criteria essential for compositional generalisation, as laid out by @xcite . shift type our benchmark constitutes a covariate shift. we assume that social reasoning of the kind that siqa tests is somehow implicitly, if not explicitly, included in the pre-training data. by changing the pronouns in the complete data sets (context, questions and answers), we do not alter the nature of the task or the correctness patterns of the answers. while the test distribution now differs more strongly from the training distribution: p(x tst ) ̸ = p(x tr ), the conditional probabilities still stay the same: p(y tst |x tst ) = p(y tr |x tr ). ## discussion and conclusion we created slayqa as a more inclusive benchmark for evaluating question answering and social reasoning in lms. a key motivation was to test how well these models can generalize in this domain, particularly under the significant distribution shift between pre-training and test data that we created by replacing gendered pronouns with neopronouns that occur much less frequently in the investigated pre-training corpora. the results indicate that we have succeeded in our objectives. the scores on slayqa are consistently lower than those on the parallel noslayqa data set, which suggests that the models struggle more with the challenges slayqa presents. interestingly, however, the scores on slayqa are not always below our random selection siqa data. this finding is intriguing because we expected questions requiring the tracking of two coreference chains to be more challenging than those without such demands. it is quite possible that questions without two coreference chains introduce different, perhaps equally complex, challenges. moreover it should also be noted that the relatively small differences between models could be due to training noisefor an even more comprehensive evaluation of neopronouns' influence, several comparable models that differ in their random initializations would be needed. as these are not readily available and costly to train, we have to leave this direction to future work. from a model-centric standpoint, the largest model (llama-3.1-8b) consistently outperforms the smaller models, even though it was drastically quantized to much lower number precision. among the smaller models, the performance differences are minimal, with no substantial gap between the 1b olmo and 2b redpajama models. additionally, there are no significant differences between the base redpajama model and those fine-tuned for instruction-following or conversational tasks, which is surprising given the assumption that finetuning should improve performance on question answering tasks compared to vanilla models. although we were not able to evaluate larger models, our accuracies do not deviate drastically from comparable zero-shot evaluations for much larger models. in the the llama-1 paper, @xcite report scores between 48.5% for llama-1-7b and 52.3% for llama-1-65b. for llama-2 @xcite , scores align as well (48.3% for the 7b model, 50.7% for the 70b model). even the largest llama-3 model with 405b parameters only achieves 53.7% on siqa, as reported in @xcite . judging from these meagre scaling effects, we assume that evaluations of larger models on slayqa should not deviate drastically. while we decided to employ a zero-shot evaluation approach for comparability, it would also be interesting to see how models fare in multi-shot reasoning or fine-tuning contexts. the allenai leader board for siqa 5 reports the best fine-tuned model with a score of 84.31%. furthermore, prompting has started to replace more technical evaluation ap-proaches (although it remains debated, see @xcite ) -as such it would be also interesting to see how commercial and open models work in siqa in prompting settings with chain-of-thought or different reasoning approaches. future research similar to slayqa should definitely aim to include even more novel and linguistically interesting forms, e.g. the aforementioned nounself, emojiself, numberself or nameself pronouns @xcite . their unique structure and usage should be even rarer than neopronouns and could pose even more veritable challenges to the generalization capabilities of modern lms. additionally, the slayqa paradigm could be expanded to other benchmarks that test different capabilities. for example, it would be interesting whether performance on the grammatical benchmark blimp @xcite deteriorates with the inclusion of neopronouns. finally, the influence of different language modeling choices, e.g. tokenization, deserves further scrutiny. as our tested models did not drastically differ in subword tokenization for the tested (neo)pronouns, we cannot draw definite conclusions. evaluation on a wider range of models could illuminate this further.