Gender Identity in Pretrained Language Models: An Inclusive Approach to Data Creation and Probing (2024)

Abstract: Pretrained language models (PLMs) have been shown to encode binary gender information of text authors, raising the risk of skewed representations and downstream harms. This effect is yet to be examined for transgender and non-binary identities, whose frequent marginalization may exacerbate harmful system behaviors. Addressing this gap, we first create TRANsCRIPT, a corpus of YouTube transcripts from transgender, cisgender, and non-binary speakers. Using this dataset, we probe various PLMs to assess if they encode the gender identity information, examining both frozen and fine-tuned representations as well as representations for inputs with author-specific words removed. Our findings reveal that PLM representations encode information for all gender identities but to different extents. The divergence is most pronounced for cis women and non-binary individuals, underscoring the critical need for gender-inclusive approaches to NLP systems.

## introduction gender identity -an individual's sense of self, reflected in their experience and perception of their gender -is closely connected to language. sociolinguistic research shows that speakers intentionally use language to construct their gender identity @xcite ; such linguistic practices evoke links with identity when perceived by others @xcite . crucially for nlp, this means that gender information is inherent in all linguistic data and may be inadvertently learned by computational systems. in a stark illustration, @xcite conduct a probing study to show that pretrained language models (plms) encode binary gender in their representations. but are transgender and non-* equal contribution. binary identities encoded to the same extent? 1 this question is vital because gender information may have a skewed distribution in the data, which may be exacerbated through pretraining and propagated to downstream tasks. plm-based systems tend to make more errors for texts authored by underrepresented groups, particularly in tasks such as predicting psychometric characteristics and personality traits @xcite . these biases may result in allocation harms and have significant realworld consequences @xcite . regarding gender identity, harmful system behaviors are likely to disproportionately affect transgender and non-binary people since they are generally underrepresented @xcite . this issue reflects a broader need for gender-inclusive approaches to language, centering the experiences of trans and non-binary people @xcite . we adopt precisely such an approach to investigate gender information in plms from an inclusive perspective. we create transcript, a corpus of texts by transgender, cisgender, and non-binary speakers (see figure 1 for an overview). it contains transcripts of youtube videos whose creators were sampled based on public sociodemographic information from wikidata (see section 8 for an explanation of the data licensing). we use our corpus to replicate the probing methodology by @xcite on a broader range of identities, and extend it to account for interactions with topic @xcite . we pose the following research questions: rq1 do plm representations encode gender identity beyond the male-female binary? rq2 does the encoded information reflect author identity rather than gender? rq3 is gender identity encoded differently after fine-tuning on an explicit label? our approach uses various plms to encode segments of transcripts and then trains a probing classifier to predict the gender identity of the transcript's author. in this setting, gender classification is used as a proxy task to examine plm representations: above-chance performance is indicative of encoded gender information, and relatively higher performance for a given class suggests a stronger bias towards it. we provide the following contributions: another line of work focuses on biased gender representations learned from training data @xcite . @xcite analyze corpora in terms of topics specific to masculine, feminine, and non-binary gender. genders are treated differently in all datasets, pointing to a misrepresentation of disadvantaged groups and a risk for the biases to transfer to plms. @xcite investigate bert's predictions of pronouns for non-binary individuals, noting its tendency to misgender even when given additional context. cao and daumé @xcite observe that nlp research tends to make strong binary assumptions around gender identity, ignoring the existence of trans and non-binary individuals. this overall highlights the need to better understand the limitations of nlp models with respect to marginalized communities. probing plms. the knowledge encoded in plms' internal representations is generally analyzed using probing techniques @xcite @xcite . a probe is usually a simple classifier trained to predict a property of interest on a model's frozen representations @xcite . its performance is taken to indicate the information encoded in the representations. however, probing classifiers may memorize patterns in the data which are unrelated to representational properties @xcite . therefore, alternative probing techniques are based on an information-theoretic approach, such as the minimum description length (mdl) probe @xcite . different techniques have been applied to probe syntactic structures @xcite , lexical semantics @xcite , and factual knowledge @xcite . in directly relevant work, @xcite assess sociodemographic knowledge: age and binary gender. they use a probing classifier and an mdl probe to analyze plms of different sizes, comparing frozen and fine-tuned representations across layers. they find that sociodemographic information is encoded, but also note the binary framing of gender as a limitation due to a lack of genderinclusive datasets. it is to be determined if trans and non-binary identity is encoded in the same way. gender-inclusive datasets. existing datasets generally treat gender as a binary variable, leading the studies that use them to adopt the same framing @xcite @xcite . one exception is gicoref, a dataset aimed at analyzing coreference resolution systems (cao and @xcite . it includes neopronouns (e.g. ze/hir, xe/xem) and articles about non-binary individuals. from a sociolinguistic perspective, zimman and hayworth (2020a) create the translivecorpus using posts from four online communities aimed at transgender and non-binary people. these examples incorporate gender-inclusive data based on the topic or target audience; we are unaware of similarly inclusive datasets with author-level information. ## transcript corpus in order to investigate the encoding of gender in plm representations from an inclusive perspective, we require texts produced by authors of different gender identities. faced with a lack of such corpora, we construct transcript: the transgender non-binary cisgender transcripts corpus. due to recent restrictions on sources of sociodemographically enriched text such as x and reddit @xcite , we devise an alternative approach (figure 1 ). we use wikidata to sample english-speaking, famous youtubers across gender identities, collect transcripts of their youtube videos, and then filter them to ensure data quality. youtube transcripts are a unique domain, comprising segments of prepared speech (e.g., interviews) as well as spontaneous conversations, opening possibilities for analyzing various language variations, such as regional origin @xcite . we now describe our pipeline in more detail and present a topic analysis of the collected content. ## probing methodology we replicate the probing methodology used by @xcite , who investigated binary gender information encoded in plms. to ensure consistency, we reimplement all their experimental steps, including the probing methods and the selection of plms. the only variable that differs in our experiments is the dataset used for the analysis. note more generally that our method sometimes involves direct comparisons of trans and cis gender identities. we stress that we do not imply a fundamental distinction between these groups, but rather use this perspective for a controlled assessment of disparities in model behavior. ## gender identity in plms we now turn to the primary goal of this work -analyzing whether plms capture information about authors' gender identities. before addressing our three main research questions, we replicate @xcite binary classification setup with our dataset. these experiments evaluate the positioning of transprob in relation to other domains analyzed by the authors and determine if our probing results are consistent with theirs. ## discussion and conclusion in this paper, we adopted an inclusive perspective to analyze whether gender identity is encoded in the representations of plms. our study began with the creation of transcript, which, to the best of our knowledge, is the first dataset to include language samples from individuals across five gender identities. the proposed data collection pipeline and topic analysis contribute to ongoing research on language use in trans and queer communities.