Leveraging Large Language Models to Measure Gender Representation Bias in Gendered Language Corpora (2025)

Abstract: Large language models (LLMs) often inherit and amplify social biases embedded in their training data. A prominent social bias is gender bias. In this regard, prior work has mainly focused on gender stereotyping bias – the association of specific roles or traits with a particular gender – in English and on evaluating gender bias in model embeddings or generated outputs. In contrast, gender representation bias – the unequal frequency of references to individuals of different genders – in the training corpora has received less attention. Yet such imbalances in the training data constitute an upstream source of bias that can propagate and intensify throughout the entire model lifecycle. To fill this gap, we propose a novel LLM-based method to detect and quantify gender representation bias in LLM training data in gendered languages, where grammatical gender challenges the applicability of methods developed for English. By leveraging the LLMs’ contextual understanding, our approach automatically identifies and classifies person-referencing words in gendered language corpora. Applied to four Spanish-English benchmarks and five Valencian corpora, our method reveals substantial male-dominant imbalances. We show that such biases in training data affect model outputs, but can surprisingly be mitigated leveraging small-scale training on datasets that are biased towards the opposite gender. Our findings highlight the need for corpus-level gender bias analysis in multilingual NLP. We make our code and data publicly available.

## introduction in recent years, the presence of social biases in machine learning models @xcite has gained significant attention due to their potential to perpetuate and amplify existing inequalities, impacting areas of great consequence in people's lives, 1 https://github.com/ellisalicante/grb-corpora llm identify all nouns and pronouns in the sentence and for each of them, determine 1) whether it refers to a person (p) or not (n), and 2) its grammatical gender: masculine (m) or feminine (f). such as hiring practices @xcite , law enforcement @xcite , healthcare @xcite , and everyday digital interactions. among various forms of bias, gender bias, i.e., the systematic preference or prejudice toward one gender versus others, is particularly concerning because it affects roughly half of the global population and has pervasive effects across different sectors of society. this concern is amplified in the area of natural language processing (nlp), particularly given the fast and wide adoption of large language models (llms). an important source of gender bias in these models is the training data which is typically obtained from sources such as books, websites, and social media, often containing biases that reflect societal prejudices and stereotypes. it has been found that biases in the training data are not only learned and perpetuated but even amplified by the models @xcite . text can exhibit different types of gender bias, including stereotyping bias @xcite , i.e., associating certain roles or traits with a specific gender, representation bias @xcite , i.e., ignoring or under-representing one gender, and semantic bias @xcite , i.e., using language that subtly devalues one gender over another. in this paper, we focus on an under-studied challenge: the existence of gender representation bias in the language corpora that are used to train llms. furthermore, we focus on gendered languages, i.e., languages that exhibit a grammatical gender. existing methods, developed for english, are often not applicable to detecting and measuring gender representation bias in gendered languages despite their prevalence in the world -it is estimated that 38 % of the world's population speaks a language with grammatical gender @xcite . to that end, we propose a novel and robust method to quantify gender representation bias in text corpora and apply it in two gendered languages: spanish and valencian. an overview of the method is shown in figure 1 . as a central component of our method, we leverage the contextual understanding capabilities of llms by prompting them to identify and classify nouns and pronouns in a given text by their reference to persons and their grammatical gender. to empirically support the motivation of our method, we also show how bias propagates from data to llm outputs through continual pretraining and how training on small datasets biased toward the opposite gender equalizes the gender imbalance in the model outputs. bias statement this paper investigates gender representation bias in text collections used as training corpora for llms, specifically in gendered languages such as spanish and valencian. we define gender representation bias as the unequal frequency of human references of different genders in textual data with respect to their prevalence in the population @xcite . this bias constitutes a form of representational harm: if one gendertypically male -is systematically overrepresented in the data, it can lead models to underrepresent or ignore the existence and perspectives of other genders in their outputs. this misrepresentation affects various downstream applications of llms, from machine translation to conversational agents, by reinforcing the invisibility of underrepresented genders and normalizing a skewed worldview. ## related work there is a growing body of literature on gender biases in nlp systems, which has been summarized in several surveys @xcite . in nlp, gender bias can take multiple forms. among these, gender representation bias refers to an imbalance in the frequency or proportionality of references to individuals of different genders within a given text. it is orthogonal to gender stereotyping, which involves associations between gender and specific traits, roles, or occupations. for example, if a corpus includes five mentions of men as doctors and only one mention of a woman as a doctor, there is no gender stereotyping involved, but there is a gender representation bias. however, if a text only includes five mentions of men as doctors and five mentions of women as nurses, there is no gender representation bias yet there is a gender stereotype regarding professions. interestingly, a relation between gender stereotyping bias and gender representation bias has been reported in a recent study @xcite , underscoring the importance of studying various forms of gender bias. from a language perspective, most existing research about biases in nlp has focused on english. as one of the prominent examples, @xcite introduce the bias in open-ended language generation dataset (bold), which benchmarks social biases across five domains: profession, gender, race, religion, and political ideology, using english text generation prompts. however, languages differ widely in how they encode gender, which has important implications for how gender bias may surface in nlp systems across languages. for instance, @xcite quantify gender bias in multilingual language models focusing on biases directed towards politicians, revealing how gender biases can vary in multilingual contexts and across culturally diverse datasets. languages can be broadly categorized into three types based on how they encode gender: grammatical gender languages, natural gender languages, and genderless languages @xcite . in grammatical gender languages, also called gendered languages, such as spanish, french, or czech, all nouns are assigned a grammatical gender -typically masculine, feminine, and sometimes neuter. the gender of person-referencing nouns in these languages often aligns with the gender of the referent. in contrast, natural gender languages, such as english or swedish, feature mostly genderneutral nouns, and gender distinctions are typically expressed through pronouns (e.g., he, she). in genderless languages, such as turkish or finnish, neither personal nouns nor pronouns encode gender; gender distinctions, when relevant, are conveyed through context or explicitly gendered lexical items (e.g., father, woman). the way gender is encoded in a language has been linked to levels of gender equality in the societies where those languages are spoken @xcite . research suggests that countries where gendered languages are spoken tend to exhibit lower levels of gender equality compared to countries with other grammatical gender systems @xcite . this correlation may reflect how the linguistic visibility of gender asymmetries parallels or reinforces broader societal gender inequalities. masculine terms are often considered the default in many gendered languages, which can implicitly prioritize male entities or perspectives. numerous studies have shown that these imbalances can significantly influence model behavior in downstream tasks, including machine translation and sentiment analysis, leading to skewed model predictions that can disadvantage one gender over another @xcite @xcite . studies by @xcite and @xcite demonstrate that biases present in training corpora can directly influence model outputs, perpetuating gender stereotypes and imbalances in downstream tasks. therefore, detecting and addressing gender imbalances in corpora is an important element to mitigate bias. it requires developing bias measurement methods that account for language-specific characteristics, as traditional methods used for english fail to accurately measure gender representation bias in gendered languages @xcite . ## methodology first, we describe a gender polarity method that has been proposed to measure gender-specific terms in english texts. next, we present a novel gender representation bias quantification method leveraging the llms' natural language comprehension power to accommodate the complexities of gendered languages. ## measuring gender representation bias in this section, we present our experimental setup and results. first, we describe the datasets on which we apply the proposed method. next, we validate our approach on an annotated dataset. finally, we report the bias evaluation results for all datasets. ## bias propagation in model outputs while the primary aim of this paper is to quantify gender representation bias in training corpora, it is also crucial to understand how biased corpora can shape the behavior of llms. to that end, we conduct a set of continual pretraining experiments to demonstrate how llm training on deliberately male-or female-biased corpora can manifest in a model's generated text. ## discussion the results of our study have significant implications for the field of nlp, particularly in the understanding and mitigation of gender representation bias in gendered and low-resource languages. below, we discuss the main findings of our research. 1. llms are an effective tool to measure gender representation bias in gendered corpora. unlike traditional approaches, our method leverages the natural language comprehension power of high-end llms to identify and classify gendered language elements within complex linguistic frameworks. this allows for a deeper understanding of gender usage in text, beyond simple word matching or limited part-of-speech tagging. ## conclusion we have presented a novel methodology for measuring gender representation bias in gendered text corpora using large language models. the validation experiments confirm the method's applicability to both well-resourced (spanish) and low-resource (valencian) languages. through experiments with spanish and valencian datasets, we reveal a substantial male dominance in both languages. we have also empirically shown how these biases can be propagated in downstream applications: in continual pretraining experiments, we observed that even a short training on male-biased, balanced, or female-biased corpora can significantly shift the ratio of male-to-female references in the generated text. while our current focus is on representation bias -in particular, the underrepresentation of a certain gender -the proposed methodology is a building block toward more comprehensive approaches that include contextual or semantic biases (e.g., stereotypical associations). by identifying these biases at the dataset level, our framework paves the way for targeted interventions, including rebalancing strategies or gender-inclusive rewriting. future work will explore more nuanced forms of gender bias and incorporate additional languages, including those with more complex grammatical systems or different cultural norms, further advancing the broader goal of equitable nlp systems.