Tokenization Matters: Navigating Data-Scarce Tokenization for Gender Inclusive Language Technologies (2024)

Abstract: Gender-inclusive NLP research has documented the harmful limitations of gender binary-centric large language models (LLM), such as the inability to correctly use gender-diverse English neopronouns (e.g., xe, zir, fae). While data scarcity is a known culprit, the precise mechanisms through which scarcity affects this behavior remain underexplored. We discover LLM misgendering is significantly influenced by Byte-Pair Encoding (BPE) tokenization, the tokenizer powering many popular LLMs. Unlike binary pronouns, BPE overfragments neopronouns, a direct consequence of data scarcity during tokenizer training. This disparate tokenization mirrors tokenizer limitations observed in multilingual and low-resource NLP, unlocking new misgendering mitigation strategies. We propose two techniques: (1) pronoun tokenization parity, a method to enforce consistent tokenization across gendered pronouns, and (2) utilizing pre-existing LLM pronoun knowledge to improve neopronoun proficiency. Our proposed methods outperform finetuning with standard BPE, improving neopronoun accuracy from 14.1% to 58.4%. Our paper is the first to link LLM misgendering to tokenization and deficient neopronoun grammar, indicating that LLMs unable to correctly treat neopronouns as pronouns are more prone to misgender.

## introduction gender bias in nlp has been extensively studied for binary gender, however mitigating harmful biases for underrepresented gender minorities remains an active area of research @xcite bpe prioritizes frequently occurring terms during llm vocabulary creation. infrequent neopronouns result in fragmented tokenization. ## background gender-inclusive nlp gender bias has been studied across several nlp contexts, including machine translation @xcite , coreference resolution @xcite , and named entity recognition @xcite . works like @xcite and others have found that choice of word segmentation exacerbates gender biases in machine translation. recent works expand gender bias evaluations to harms unique to non-normative gender communities within llms @xcite @xcite @xcite . @xcite examine non-binary gender bias in static and contextual language representations, highlighting how data limitations affect these embeddings. similarly, @xcite explore misgendering and harmful responses related to gender disclosure using their tango framework, pointing to challenges in neopronoun consistency, possibly due to data scarcity. @xcite corroborate these findings with an in-context-learning evaluation and analyses into llm pretraining corpus statistics. despite exploring various in-context learning strategies, they find persistent gaps between binary pronoun and neopronoun misgendering. these studies collectively emphasize data scarcity's impact on neopronouns, though questions remain regarding how data scarcity shapes neopronoun representations and subsequent llm pronoun consistency. in this study, we investigate the pivotal role of bpe tokenization due to its critical relationships to pretraining corpora and subsequent llm vocabulary construction. ## tracing llm misgendering to grammatical deficiencies this section presents a series of metrics to evaluate llm misgendering from the standpoint of pronoun proficiency. we perform baseline evaluations on out-of-the-box gpt-neo-x based models and provide an overview of our evaluation scheme in figure 2 . ## experimental setup we provide an overview of our experimental setup in figure 3 . we conduct carefully controlled experiments across two finetuning paradigms using open-source llms that vary in model size and neopronoun data scarcity. in the first set of experiments, we employ ptp in a standard full finetuning paradigm. in the second experiment, we introduce lexical finetuning and variants with ptp. we perform these experiments across binary pronouns and the neopronoun family xe. we center xe for several reasons: xe ranks among the most widely adopted non-binary pronouns (gender census, 2023). non-binary pronouns also exhibit diverse linguistic variations, spanning from closed to open word class forms @xcite . this diversity requires a nuanced yet flexible approach. by focusing on the xe pronoun family, we showcase the effectiveness of ptp while providing a generalizable framework for researchers to build upon for studying non-binary pronouns within their respective linguistic contexts. ## conclusion in this work, we discover how disparate bpe tokenization across gendered pronouns, a consequence of data infrequency in training corpora, is associated with a model's degraded ability to adhere to pronoun morphosyntax. this deficiency is highly correlated with an llm's propensity to misgender data-scarce neopronouns. parallels to low-resource multilingual nlp efforts in addressing tokenizer limitations help inform novel approaches to mitigating english neopronoun misgendering. we find that employing vocabulary amelioration with pronoun tokenization parity along with a monolingual twist on lexical finetuning improve llm neopronoun consistency and grammatical proficiency over traditional finetuning settings with standard bpe tokenization. as bpe is just one of many subword tokenization algorithms, our work opens new avenues for exploring this phenomenon under various subword tokenization algorithms and in multilingual settings. nonetheless, these challenges ultimately arise from larger issues surrounding data availability and limitations of greedy (i.e., context-free) tokenization techniques. addressing these foundational issues in future work is essential for sustainably developing inclusive llms and preventing social harm.