Exploring Gender Bias in Large Language Models: An In-depth Dive into the G erman Language (2025)

Abstract: In recent years, various methods have been proposed to evaluate gender bias in large language models (LLMs). A key challenge lies in the transferability of bias measurement methods initially developed for the English language when applied to other languages. This work aims to contribute to this research strand by presenting five German datasets for gender bias evaluation in LLMs. The datasets are grounded in well-established concepts of gender bias and are accessible through multiple methodologies. Our findings, reported for eight multilingual LLM models, reveal unique challenges associated with gender bias in German, including the ambiguous interpretation of male occupational terms and the influence of seemingly neutral nouns on gender perception. This work contributes to the understanding of gender bias in LLMs across languages and underscores the necessity for tailored evaluation frameworks.

## introduction recent advancements in large language models (llms) have significantly enhanced text generation technology. yet, critical questions have been raised regarding fairness and the reflection and amplification of biases within these models, where gender bias has formed a prominent role. prior research has demonstrated biases exhibited by llms and other natural language processing (nlp) models in internal representations and external outputs: word embeddings encode stereotypes regarding gender @xcite @xcite @xcite , race @xcite @xcite , religion @xcite , disability @xcite and sexual orientation @xcite . these biases can be found in contextualised and context-free word embeddings, as well as in sentence embeddings @xcite . bias can also be found in the output of generative language models. for example, gpt-3 has been shown to (re)produce biased outputs concerning religion, specifically showing anti-muslim sentiment @xcite . further studies have identified social biases in models' generated text related to geographic location @xcite , race, sexuality, and gender @xcite @xcite . bias in llms can have different sources like biased training data, modelling approaches introducing bias or reproducing of existing historical or structural biases @xcite . various methodologies have been proposed to quantify different forms of social biases within nlp. however, many of these approaches have faced significant criticism, mainly concerning their lack of conceptual foundation for defining bias @xcite @xcite . furthermore, most existing research has been focused on bias evaluation of english-language datasets @xcite . given the deeply embedded nature of social group disparities, particularly in highly gendered languages, it is unlikely that english-language-only datasets can capture these biases across different linguistic contexts or languages. this work contributes to the existing body of research by developing and presenting five germanlanguage datasets designed for evaluating gender bias in llms. these datasets are grounded in welldefined concepts of gender bias and consider the relevant characteristics of the german language. moreover, we propose metrics for each dataset to facilitate bias analysis and provide empirical results derived from an evaluation of eight multi-lingual llms. our results show that all investigated models are prone to reproduce gender stereotypes in q&a tasks as well as in open text generation tasks. further, the models prefer generating personas of one gender over another. ## related work the evaluation of bias within nlp has earned considerable scholarly attention. traditional embedding-and probability-based methods have faced criticism due to their limited correlation with downstream biases manifested in text generated by llms @xcite @xcite . while output-based methods for bias evaluation highly depend on design choices @xcite and potentially suffer from additional bias when using auxiliary classifier models @xcite , they evaluate the text generated by llms and thus directly examine their downstream behavioural implications. bias evaluation metrics require specific datasets for retrieving embeddings and computing probabilities for generating outputs. the structural composition of the datasets varies with the evaluation method used. most datasets were designed for probability-based assessments, such as wino-bias @xcite , winogender @xcite , and stereoset @xcite , which evaluate gender-based word predictions. in contrast, counterfactual-based datasets like crows-pairs @xcite and redditbias @xcite support the comparison of probabilities attributed to gender-swapped sentences. for the output-based analysis of models, specific datasets are designed to provide inputs for llms. for instance, sentence completion datasets (e.g., honest @xcite , bold @xcite ) serve as a tool for generating text. this can be analysed with lexical @xcite , distribution-based @xcite , or classifier metrics @xcite . whereas, question-answering datasets (e.g., bbq @xcite , unqover @xcite ) can be used to test whether models exhibit reliance on gender stereotypes when answering ambiguous questions. however, existing datasets have been criticised regarding their poor construction, errors, and methodological flaws. @xcite identified major validity issues within datasets such as stereoset and crows-pairs and estimated that only between 0% and 6% of the samples of these datasets are valid for bias evaluation. parts of the datasets are wrong in terms of grammar or spelling, while for other parts, it is unclear how they relate to the types of bias supposedly evaluable with the datasets. therefore, ensuring dataset validity and coherence is crucial for reliable bias evaluation strategies. the prevalence of existing datasets for the evaluation of (gender) bias is in the english language @xcite . given that gender is more strongly embedded in the german language compared to english, translating english datasets becomes a non-trivial task. in german, every noun is assigned a grammatical gender (genus) which is only minimally related to concepts of biological sex or social gender. for example, "the person" would be translated as "die person" in german and has female grammatical gender while not specifying the natural gender of the person. still, most personal nouns contain information about the natural gender foot_0 of the person they refer to, which usually coincides with the grammatical gender of that noun @xcite . thus, where english datasets rely on gender-neutral phrases, for example for pronoun resolution, they can not be directly translated into german. making things more complex is the adversary concept of the "generic masculine", referring to masculine versions of personal nouns that may denote persons of any natural gender @xcite . although there is existing research on the evaluation of bias in german @xcite @xcite @xcite , we could only identify one extensive german dataset for text generation: the salt datasets of @xcite that were published simultaneously to our research work. there is a small overlap between the salt dataset and the datasets proposed in this work. both include instructions for llms to write a story about a person. however, @xcite assess the general quality of the output while we analyse the outputs concerning lexical overlap and gender distribution. both ap-proaches can be combined for an even more holistic bias evaluation. this study considers eight categories of gender bias in the evaluation of llms. the categorisation is based on the bias taxonomy proposed by @xcite , which follows insights from (socio-)linguistic and machine learning related research, including @xcite and @xcite . additionally, @xcite created a categorisation of sexist content based on psychological scales measuring sexism and related gender-based concepts. these categories overlap with and extend the bias taxonomy of @xcite . the categories are not mutually exclusive and often appear together: ## datasets the main contribution of this work are five german datasets for bias evaluation in llms. their creation process and contents are presented in this section. each of the datasets is linked to the analysis of specific gender bias categories, however this also depends on the metrics chosen to analyse the output generated with the datasets. in section 5 we propose metrics for each dataset and apply them to eight models. we create new datasets specific to the german language and draw on established english datasets when they are transferable. two of the datasets are partly translated from existing english datasets with the deepl api foot_1 . while most of the data is manually created, some parts are synthetically generated with few-shot prompting using chatgpt using gpt-4o foot_2 , by providing translated or manually created examples as few-shot examples. prompts used for synthetic data generation can be found in appendix a.1. all translated and synthetically generated prompts are manually verified and postedited by the authors to ensure the quality of the datasets. two datasets are q&a datasets, and three contain prompts for open text generation. examples for datasets are only given as english translations; original german versions can be found in ## experiments the new datasets can be used on llms, and the generated output can be analysed with a variety of methods, in particular the open text generation outputs. due to the different natures of the datasets, they have to be assessed with specific types of metrics. a few of these are described below. datasets and metrics are applied to eight models, and the results are reported. ## discussion the experiments reveal systematic gender biases across all eight tested llms, and show that the datasets and metrics successfully capture the different kinds of gender bias. performance on the gerbbq+ dataset demonstrates that ambiguity in inference tasks significantly impacts model accuracy and bias. models frequently relied on gender stereotypes when resolving ambiguous prompts, with notably lower accuracy and higher bias scores under these conditions. minor uncertainties regarding answer extraction remain and should be addressed in the future. the stereopersona and genderpersona datasets revealed that models reinforce gender stereotypes when generating personas. output generated with the genderpersona dataset is complex and possible additional metrics can be investigated in the future. additionally, the neu-tralpersona dataset revealed that each model has preferences for one gender when generating personas, albeit the preferred gender differed across models. least bias was found with the sexiststatements dataset, where models overall tended to exhibit low sexism scores. however, higher sexism was found when statements referred to men, indicating a lack of mitigation efforts when sexism is aimed at the historically advantaged group. during developing the persona datasets, as well as some results further revealed the intricacies of the german language when dealing with gender. great care has to be taken with regard to grammatical and natural gender: in the genderpersona dataset, male personal nouns can be interpreted as gender-neutral ("generic masculine"), which we addressed by specifying that a specific, fictional persona is meant. on the other hand, results of the neutralpersona dataset suggest that the grammatical gender of gender-neutral personal nouns (the person (feminine)/ the human (masculine)) influence the natural gender of personas generated. these issues have to be investigated further. finally, when asked to generate descriptions of personas without reference to gender (stereopersona, neutralpersona), outputs could overwhelmingly be classified as male or female, indicating that models prefer gender-binary language over gender-neutral or non-binary language. ## conclusion the herein proposed german datasets for gender bias evaluation in llms aim to address the notable deficiency in resources for assessing bias in the german language, as existing bias assessment tools and datasets have been primarily developed for english. as gender is deeply embedded in german grammar, the implementation of german-specific approaches is necessary for more precise evaluations. the five proposed datasets, their empirical application to various llms and the analysis using the proposed metrics show promising results. all models display a tendency for stereotypical representations over anti-stereotypical alternatives, as evidenced by the gerbbq+ and stereopersona datasets. thus, it is vital to explore a broader set of methods for output analysis while refining and validating the proposed techniques. finally, we believe that the introduction of these datasets provides a crucial foundation for future inquiries on bias evaluation in german llms as well as potentially serving as a benchmark for bias mitigation approaches.