Examining the Cultural Encoding of Gender Bias in LLM s for Low-Resourced A frican Languages (2025)

Abstract: Large Language Models (LLMs) are deployed in several aspects of everyday life. While the technology could have several benefits, like many socio-technical systems, it also encodes several biases. Trained on large, crawled datasets from the web, these models perpetuate stereotypes and regurgitate representational bias that is rampant in their training data. Languages encode gender in varying ways; some languages are grammatically gendered, while others do not. Bias in the languages themselves may also vary based on cultural, social, and religious contexts. In this paper, we investigate gender bias in LLMs by selecting two languages, Twi and Amharic. Twi is a non-gendered African language spoken in Ghana, while Amharic is a gendered language spoken in Ethiopia. Using these two languages on the two ends of the continent and their opposing grammatical gender system, we evaluate LLMs in three tasks: Machine Translation, Image Generation, and Sentence Completion. Our results give insights into the gender bias encoded in LLMs using two low-resourced languages and broaden the conversation on how culture and social structures play a role in disparate system performances.

## introduction large language models (llms) are increasingly integrated into everyday interactions such as search engines and digital assistants @xcite and in several domains, including education @xcite and healthcare @xcite . however, these models also embody several risks @xcite . trained on large, webcrawled datasets, the models perpetuate the biases that are embedded within their datasets @xcite . further, several design choices in the design and deployment of llms marginalize some communities @xcite . while there have been rapid advancements in evaluation benchmarking over the past decade, low-resource languages continue to be underrepresented in llm research, development, and evaluation @xcite . as llms increasingly interact with users dailyincluding in sensitive domains like healthcare, finance, and policing -we must understand the biases encoded in them, particularly against marginalized groups. the field of natural language processing (nlp) has been scrutinized for its anglocentric practices, which exclude the majority of the world's population @xcite . fortunately, there is an emerging corpus of multilingual research that includes developing models for low-resourced languages from pre-training models @xcite @xcite or via fine-tuning models @xcite @xcite . despite this progress, there is still little research examining how biases are encoded in llms for low-resourced languages, limiting efforts toward bias mitigation. llms are mainly trained on data crawled from the internet. however, several socio-economic barriers determine whose voices are represented on the internet @xcite . prior work shows that the majority of the content online comes from western countries like the united states @xcite , and that content on websites like wikipedia is predominantly contributed by male users @xcite . particularly looking at african communities, while overall access to the internet is improving, women are less likely to have access to the internet than men, resulting in a digital gender divide (, unicef). as a result, the voices of women are less likely to be represented on online platforms. additionally, prior work shows that datasets sourced from online platforms for low-resourced languages might a signifies how gender is represented in these two languages and b signifies how stereotypically these systems inhibit bias against a particular gender. twi is a gender-neutral language in terms of grammar, meaning it does not have gendered pronouns or verb conjugations that change based on gender. in contrast, amharic is a gendered language, where pronouns, verb forms, and even some nouns explicitly indicate gender. a significant amount of the bias evaluation done on llms has been focused in western contexts and on western constructs like race. several works have demonstrated gender, religious, cultural, and racial bias in llms @xcite @xcite @xcite . however, social structures vary across communities, making multilingual and multicultural evaluations difficult @xcite @xcite . prior works have revealed religious @xcite and caste @xcite discrimination embedded in llms looking at middle eastern and south asian contexts, respectively. however, little attention has been paid to african languages and communities, with evaluations mainly covering performance disparities (e.g @xcite . in this paper, we lean into the diversity of language and culture in african communities and evaluate gender bias encoded in llms. we select two unrelated languages: twi, a niger-congo language spoken in ghana, and amharic, an afro-semitic language spoken in ethiopia. we use three tasks: machine translation (mt), image generation, and sentence completion. we prepared prompts that draw on the cultural and social aspects of the communities that speak the two languages. mainly, we use cultured names and pronouns to probe gender bias in llms. building on the background provided, this study seeks to answer the following research questions; ## related works nlp research has predominantly focused on higherresourced languages like english, leaving out the majority of the world's languages @xcite . as research trends mainly focus on llms that are trained on large corpora, the language divide is furthered as very limited languages have enough datasets to train such models @xcite . as a result, the performance of llms in low-resourced languages is low across several tasks and domains @xcite . recent works have tried to increase the inclusion of african languages in large models by training models from scratch (e.g. tonja et al., 2024) or fine-tuning pre-trained models (e.g. @xcite @xcite . as models increasingly become multilingual, we must understand how bias is encoded in the models across cultures and languages. @xcite found that lowresourced languages can bypass guardrails imposed in llms in english, effectively jail-breaking mitigation strategies. a number of prior works have evaluated occupational bias from llms and textto-image generators, finding that these systems are likely to recommend or associate certain demographic groups with stereotypical jobs (e.g., woman as nurse, man as engineer) @xcite @xcite @xcite . work by @xcite , assessing gender and racial bias from gpt-4 in relation to the healthcare sector, also indicates a need for sector-specific bias mitigation. while research has recently emerged to understand how to mitigate occupational bias and reduce gendered correlations @xcite @xcite , there is still much more work needed to understand how these methods could apply to non-western contexts and non-gendered languages. in addition to exclusion from model development, low-resourced languages-and their communities-are also understudied in bias evaluation @xcite . many works have investigated the biases in llms across gender @xcite @xcite @xcite @xcite @xcite , racial @xcite , and socioeconomic @xcite angles, focused on western contexts. however, these axes of social identity are expressed differently across communities and cultures @xcite . for instance, the gender roles in one community differ from those in another community. additionally, communities may have a social axis that is specific to how they organize their social structures. @xcite look at the biases encoded in llms in terms of caste identity, which is a social axis important in the south asian context. @xcite and okolo (2023) have looked into cultural bias in image-generation models and found that image-generation models perpetuate stereotypes against africans. as @xcite state, multicultural evaluation is complicated by the several intersecting social identities that shape how bias manifests in communities. gender bias has been studied by several prior works, particularly in machine translation (e.g. @xcite @xcite . @xcite investigate gender bias in three low-resourced languages,including one of our focus languages-amharic-and report that machine translation (mt) systems exhibited gender bias in 72. 5% of the cases report that the mt systems exhibited gender bias in 72% of the cases, specifically when translating gender-neutral english source sentences. oppong (2023) and @xcite explore gender bias in machine translation for twi, demonstrating how translation systems can learn, reflect, and reproduce societal biases, particularly those that disadvantage women in african contexts. prior work has also created a benchmark dataset to evaluate machine translation systems for luganda @xcite . these studies highlight the nature of gender bias in low-resource machine translation systems and demonstrate the need for languagespecific evaluation frameworks. in this paper, we select two unrelated african languages (twi and amharic) to investigate the gender bias encoded in llms. we focus on three tasks for our investigation and prepare prompts informed by the cultural and social aspects of the communities that speak these languages. ## methodology in this section, we will first describe the languages our study focuses on (section 3.1). in section 3.2, we present our experimental design, including how we prepared the datasets (section 3.2.2), the models we used (section 3.2.1), our evaluation metrics (section 3.2.3), and the tasks we evaluated (section 3.2.4). ## discussion in this work, we looked into gender bias encoded in llms using two low-resourced languages as a case study. we evaluated three tasks: machine translation, image generation, and sentence completion. in answering our research questions, we find that llms like gpt-4o and chatgpt consistently favor the male gender in translation, image generation, 2 following @xcite , we refrain from reporting which ethnic groups and religions are associated with violent verbs to not further perpetuate stereotypes and harmful connotations. and sentence completion. with sentence completion, we find that the model relies on statistics and acknowledging stereotypes in favoring the male gender for certain stereotypical occupation roles. we also find that gender bias is more pronounced in amharic, which is a gendered language, as compared to twi, which is not a gendered language. through all our experiments, we find that the models' outputs are more likely to conform to the male gender (section 4). gender bias has extensively been studied in higher-resourced languages (e.g. @xcite @xcite ); yet, these issues persist when prompting models in low-resourced languages. as we adopt methods developed for higherresourced languages to our languages, we must also consider issues of bias that have, at the very least, been identified in higher-resourced languages. for instance, performing audits for training datasets before training our models and critically reflecting on the datasets we release using tools like datasheets for datasets @xcite . for twi, we observed that some twi-to-english machine translation predictions from gpt-4 recognize the pronoun 'o' (he/she) for both genders but still translate it inconsistently, a challenge that is prevalent in most twi machine translation systems. further, we find that prompting models in a lowresourced language do not necessarily guarantee outputs that are reflective of the culture the languages come from; even when we prompted in twi and amharic, the image generation output was reflective of the dominant european culture and stereotypical depictions of africans (section 4.4). this calls for the inclusion of cultural and linguistic diversity beyond adding languages in models. the african nlp community should not only focus on whether our languages are included in mainstream llms but also reflect on how the inclusion materializes in system performance. ## conclusion in this paper, we investigated how gender bias is encoded in llms by designing prompts in twi and amharic. we tested machine translation, image generation, and sentence completion tasks with gpt-4o and chatgpt. we find that llm outputs display bias against the female gender and that the gendered language amharic suffers more from the bias compared to the non-gendered language twi. we hope our paper gives insights to the africanlp community, low-resource nlp, and others particularly invested in the culturally grounded development and evaluation of language technologies, into the inequitable performance of llms for certain communities, and that this prompts discussions around what inclusion in mainstream nlp means for african languages and communities.