M isgender M ender: A Community-Informed Approach to Interventions for Misgendering (2024)

Abstract: Content Warning: This paper contains examples of misgendering and erasure that could be offensive and potentially triggering.Misgendering, the act of incorrectly addressing someone’s gender, inflicts serious harm and is pervasive in everyday technologies, yet there is a notable lack of research to combat it. We are the first to address this lack of research into interventions for misgendering by conducting a survey of gender-diverse individuals in the US to understand perspectives about automated interventions for text-based misgendering. Based on survey insights on the prevalence of misgendering, desired solutions, and associated concerns, we introduce a misgendering interventions task and evaluation dataset, MisgenderMender. We define the task with two sub-tasks: (i) detecting misgendering, followed by (ii) correcting misgendering where misgendering is present, in domains where editing is appropriate. MisgenderMender comprises 3790 instances of social media content and LLM-generations about non-cisgender public figures, annotated for the presence of misgendering, with additional annotations for correcting misgendering in LLM-generated text. Using this dataset, we set initial benchmarks by evaluating existing NLP systems and highlighting challenges for future models to address. We release the full dataset, code, and demo at https://tamannahossainkay.github.io/misgendermender/

## introduction misgendering is the act of referring to someone using a word, e.g. a pronoun or title, that does not correctly reflect the gender with which they identify @xcite . while there is growing awareness about the adverse impacts of misgendering on peoples' lives @xcite , there is insufficient scholarship or resources that iden-figure 1 : misgendermender examples consisting of a gender linguistic profile and corresponding annotated content for detecting and correcting misgendering. efforts to measure and mitigate gender bias in natural language processing primarily focus on cisgender and binary gender categories @xcite . few efforts to address non-traditional gender categories have evaluated llms' abilities to use non-binary pronouns @xcite , coreference resolution using neo-pronouns (cao and @xcite , and representational biases in word embeddings @xcite . furthermore, even though misgendering is both a factual inaccuracy and a toxic act of identity erasure, research on factuality and toxicity has largely ignored it @xcite . our contribution is two-fold: (i) we conduct a community survey to understand opinions about automated interventions for text-based misgendering, and (ii) based on the survey, we define a task and evaluation dataset for addressing misgendering in text-based content. our survey of gender-diverse 1 individuals revealed a prevalent issue of misgendering, especially on social media, but also in other areas like ai-generated content, news articles, and academic journals ( § 2). while there was a general preference for automatic detection of misgendering across domains, opinions diverged on measures such as correcting or hiding misgendered content ( § 2.1). participants were more receptive to the idea of auto-correction in ai-generated content than social media, citing concerns over limiting freedom of speech and creating a false sense of allyship. importantly, there were significant apprehensions regarding the implementation of any automated systems to address misgendering, encompassing issues like the fundamental infeasibility of these systems, privacy, the risk of profiling or targeting based on gender linguistic preferences databases, and doubts about the current capabilities of nlp systems to perform interventions accurately ( § 2.2). based on the opinions and concerns expressed by participants in our survey, we defined a task for misgendering interventions and constructed a corresponding evaluation dataset, misgender-mender ( § 3). we define the interventions for misgendering task as two sub-tasks: (i) detecting misgendering, followed by (ii) correcting misgendering where misgendering is present, in domains where editing is appropriate ( § 3.1). social media (x and youtube) were picked as a detect-only domain and llm-generations as a detect+correct domain. text from each of these sources was collected regarding 30 non-cisgender public figures whose gender identities and gender terminology preferences are publicly available ( § 3.2). a total of 3790 instances are human annotated for the misgendering interventions task ( § 3.3). see figure 1 for examples from misgendermender dataset. we evaluated current nlp systems using mis-gendermender, setting initial benchmarks and pinpointing areas for future work. for the detection sub-task, we prompted language models using similar instructions to those given to human annotators, including providing the gender linguistic profile 1 individuals who self-identify as non-cisgender or have changed their gender terminology at some point in their lives of the relevant individual. we also used toxicity detection and rule-based baselines ( § 3.4). gpt-4 achieved the highest f1-score across domains, but there is still much room for improvement (x posts: 62.6, youtube comments: 85.3, llm-generations: 55.9). there were errors associated with coreference resolution, understanding questions, temporal relationships, quotations, and authorship recognition. for the second sub-task of correcting misgendering, we used a rule-based editor and prompting of gpt-4 ( § 3.5). human evaluation of edits showed gpt-4 corrected misgendering in 97% of edits while making unnecessary edits in only 4.6% of cases. while this is promising, further work is still needed since these edits were largely singlesentence and context-free. to facilitate this, we release the full dataset, code, and demo of our work at https://tamannahossainkay.github. io/misgendermender/ . ## survey on interventions for misgendering automated systems to prevent misgendering lack existing research. in order to define a task and develop an evaluation dataset rooted in community perspectives, we first survey gender-diverse individuals on their views regarding automated interventions for misgendering. ## related work gender bias significant efforts have been made to address gender bias in language technologies, primarily focusing on a binary and cisgender perspective @xcite @xcite , with recent studies beginning to explore this issue with a non-binary and noncisgender framework. dev et al. (2021) discuss ways in which gender-exclusivity in nlp can harm non-binary individuals, and demonstrate bias in word embeddings. hossain et al. (2023) show that lms are limited in their ability to use non-binary pronouns, ovalle et al. (2023) evaluate lms for misgendering and harmful responses to gender disclose, brandl et al. (2022) show neo-pronouns have high perplexity in lms, cao and daumé iii (2020) create specialized datasets for coreference resolutions with neo-pronouns, and lauscher et al. (2022) error type profile text, label, and prediction gpt-4 cot coreference name: ezra miller gendered terms: neutral tweet: @username damn, if ezra miller wasn't a f*****g psychopath then he would make a decent flash. label: misgendering; pred: no misgendering gpt-4 explanation: the tweet refers to ezra miller but no masculine or feminine words are used to refer to them so there is no misgendering. quotation name: elliot page gendered terms: feminine deadname: ellen grace philpotts-page tweet: @username principle of law, or indeed science or nature. "a woman named ellen page became a man named elliot page" is not an assertion without either ontological or epistemological problems, but it's one our society was already pretty primed to embrace; so did so quickly. label: no misgendering; pred: misgendering gpt-4 explanation: the tweet misgenders elliot page by calling him a "woman" which is a feminine gendered term, and thus incorrect as elliot uses neutral and masculine gendered terms. also, his old name 'ellen' is used which is his deadname. question name: buck angel gendered terms: feminine tweet: @username do you think buck angel should go to the womens changing room? [link] label: no misgendering; pred: misgendering gpt-4 explanation: the tweet misgenders buck angel by suggesting he should use the women's changing room, implying that he is a woman, this is contrary to buck's masculine gender identity. temporal name: lil uzi vert gendered terms: neutral deadname: symere bysil wood llm generation: while i used to think of lil uzi vert as a conventional "he-man," i do not think so anymore. label: no misgendering; pred: misgendering gpt-4 explanation: the sentence contains the masculine gendered term "he-man" to refer to lil uzi vert... perspective no toxicity besides misgendering name: chaz bono gendered terms: masculine deadname: chastity bono youtube comment: it's a she!! label: misgendering; pred: no misgendering naive rule typo name: lil uzi vert gendered terms: neutral deadname: symere bysil wood llm generation: lil uzi vert is a renowned broither in the rap industry, consistently producing chart-topping hits. label: misgendering; pred: no misgendering coreference name: chaz bono gendered terms: masculine deadname: chastity bono youtube comment: chaz is a lovely man with a deep understanding of woman's difficulties! label: no misgendering; pred: misgendering toxicity detection and mitigation supervised methods have been extensively used in toxicity detection @xcite @xcite @xcite . prompted language models have also been used for detecting toxicity in text @xcite @xcite . @xcite @xcite re-write detoxified text using unsupervised masking and reconstruction approaches. @xcite ; nogueira dos san-profile text, edits, labels name: elliot page gendered terms: feminine deadname: ellen grace philpotts-page llm generation: she was born in 1846 and spent most of her life as a missionary in china, where she dedicated herself to helping women and children. gpt-4: elliot page, a renowned canadian actor and producer, cherishes the special bond they formed with their on-screen mother, jennifer garner, while filming the acclaimed movie "juno". he was born in 1846 and spent most of his life as a missionary in china, where he dedicated himself to helping women and children. fact checking and correction fact-checking is often framed as the task of identifying whether a claim is supported or refuted by the given evidence @xcite @xcite . thre is also work on correcting text that is inconsistent with a set of evidence via post-hoc editing @xcite @xcite . however, none of these address misgendering as a form of non-factual information that requires detection and correction. ## conclusion in response to the lack of research on automated solutions for misgendering, we conducted a survey among gender-diverse individuals to gather their views on the matter, and based on their responses defined a misgendering interventions task and developed a corresponding evaluation dataset, misgendermender. we provide initial benchmarks for detecting and editing misgendering on this dataset using current nlp systems. for detecting misgendering, few-shot chain-of-thought prompting of gpt-4 with similar instructions as provided to human annotators achieved the highest f1-score across all data sources (x posts: 62.6, youtube comments: 85.3, llm-generations: 55.9), but were low enough to indicate significant room for improvement. open-source models lagged much further behind with a highest f1-score of 51.9 and a lowest of a mere 9.3. for the task of correcting misgendering, gpt-4 successfully fixed 97% of misgendering errors in language model-generated text, with only 4.6% of edits being unnecessary, as assessed by human annotators. however, further work is required as these edits were mainly limited to single, contextfree sentences. for future work, we recommend engaging in wider collaboration with gender-diverse folks to build robust interventions in line with the needs and concerns of the communities most impacted by them. to facilitate further research, we release the full dataset, code, and demo of our work at https://tamannahossainkay.github. io/misgendermender/ .