The power of Prompts: Evaluating and Mitigating Gender Bias in MT with LLM s (2024)

Abstract: This paper studies gender bias in machine translation through the lens of Large Language Models (LLMs). Four widely-used test sets are employed to benchmark various base LLMs, comparing their translation quality and gender bias against state-of-the-art Neural Machine Translation (NMT) models for English to Catalan (En → Ca) and English to Spanish (En → Es) translation directions. Our findings reveal pervasive gender bias across all models, with base LLMs exhibiting a higher degree of bias compared to NMT models. To combat this bias, we explore prompting engineering techniques applied to an instruction-tuned LLM. We identify a prompt structure that significantly reduces gender bias by up to 12% on the WinoMT evaluation dataset compared to more straightforward prompts. These results significantly reduce the gender bias accuracy gap between LLMs and traditional NMT systems.

## introduction within the domain of machine translation, gender bias is defined as the tendency of mt systems to produce translations that reflect or perpetuate gender stereotypes, inequalities, or assumptions based on cultural and societal biases @xcite . given that the presence of such bias can lead to harmful consequences for certain groups -either in representational (i.e., misrepresentation or underrepresentation of social groups and their identities) or allocational harms (i.e., allocation or withholding of opportunities or resources to certain groups) - @xcite @xcite , it becomes paramount to thoroughly investigate and mitigate its occurrence. nevertheless, addressing gender bias is a multi-faceted task. gender bias is a pervasive issue in all generative nlp models, and llms are no exception to this situation. llms have gained significant popularity in recent years and are being used for many nlp tasks, including machine translation. while gender bias in machine translation has been extensively studied for neural machine translation models, little attention has been paid to this type of bias in llms. this paper aims to address this gap by examining and trying to mitigate this bias in the translations generated by the llms. the aim of this work is twofold. first, a comprehensive benchmarking process is conducted to compare various base llms with some state-of-theart nmt models. the directions of the translations under study are @xmath0. distinct popular test sets such as @xcite , winomt @xcite , gold bug @xcite , and must-she @xcite are used to assess the translation quality and the gender bias of the models. following the benchmarking, an investigation into the effectiveness of prompts in mitigating this bias in llms is conducted. the purpose of this research is to determine whether well-designed prompts can serve as a useful strategy in addressing bias. while existing literature has explored various approaches to mitigating this bias in neural machine translation models @xcite @xcite , we specifically focus on the realm of llms, probing the role of prompts. in this phase of the study, an instruction-tuned llm is employed, and several prompt engineering techniques are experimented with, including few-shot @xcite @xcite , context-supplying, and chain of thought @xcite . the relevance of this work lies in several insightful findings. firstly, we demonstrate that base llms tend to lag behind nmt models in terms of translation capabilities and gender-bias scores. afterwards, through an extensive trial-and-error examination into prompting, we present a prompt that, when applied to an instructed llm, achieves impressive bias mitigation across gender-bias test sets, resulting in an increase of 12.4 and 11.7 in the respective catalan and spanish winomt scores. finally, we study how gender-bias mitigation through prompting impacts llms translation performance. the rest of the paper is organized as follows: section 3 reviews relevant research in the field. section 4 details the methodology, including the datasets, models, and evaluation metrics employed. section 5 focuses on the benchmarking, while section 6 explores the investigation into prompting to mitigate gender bias. section 7 presents the results. finally, section 8 provides a discussion and section 9 highlights the conclusions of this work. ## gender bias statement as previously stated, gender bias may lead to inequalities and harmful consequences. in the context of machine translation, we easily come up with two different motivations to consider this issue seriously. first, the presence of gender bias may affect the representation of genders in certain communities. on the other hand, the majority of users of a machine translation system may not be proficient in at least one of the languages involved in the translation. producing incorrect gender translations provides inaccurate information, misleading users who are trying to understand the original text from a translation, or causing them to convey a different meaning when relying on mt engines to communicate. the presence and extent of gender bias in machine translation can vary depending on the languages involved, as gender is manifested differently across languages @xcite . when translating from a language with fewer gender cues to a language with more explicit gender markings, the issue of gender bias can arise. this is precisely the case in our study: we translate from a language with notional gender (english) to languages with grammatical gender (catalan and spanish). in this context, certain professions may be stereotypically associated with certain genders. examples of this phenomenon are engineers, who are often translated as masculine, while nurses are translated as feminine @xcite . additionally, adjectives may be gendered as masculine or feminine based on these stereotypes, rather than relying on gender cues. gender pronouns may also be overlooked in favor of or against certain genders. let's consider a typical example (figure 1 ). in our study, we analyze gender bias in two distinct ways, which we will refer to as gender-bias tasks: gender coreference resolution and gender terms detection. in both tasks, models must utilize contextual gender information (i.e., gender cues) to accurately translate, providing the correct gender terms in the translation. ## related work large language models are advanced ai models designed to understand and generate language @xcite . these models typically employ a decoder-only architecture and are characterized by their enormous size, often containing billions of parameters @xcite @xcite . the scale and capacity of llms enable them to capture intricate linguistic nuances and handle a wide range of language-related tasks, despite not being explicitly trained for each specific task @xcite @xcite @xcite @xcite . the training process for llms typically consists of two steps. first, they undergo self-supervised pretraining using vast amounts of text data, which allows them to develop a general understanding of language (i.e., base llms). subsequently, they are fine-tuned on specific supervised tasks to specialize in various applications @xcite . one of the key features of llms is the prompting mechanism. a prompt serves as the input or activation signal provided to the model. through this input, we specify to the model the nlp task we want it to perform, such as translation in our case. by leveraging the ability to guide the model with prompts, instruction-tuned llms are created @xcite . these are base llms that have undergone additional fine-tuning using datasets of instructions, containing explicit instructions or prompts to enhance their performance on various tasks. instruction-tuning is a subsequent step that tailors the model's behavior and output according to specific instructions or guidelines @xcite @xcite . moving away from llms, we find neural machine translation models @xcite @xcite @xcite @xcite . these models represent the state-of-the-art in machine translation, consistently achieving the highest translation performance. they typically leverage an encoder-decoder transformer @xcite trained with parallel data in a supervised manner, intended solely for the task of translation. unlike llms, nmt models are relatively smaller in size and present unique challenges when it comes to scaling (e.g., bidirectional processing, the attention mechanism complexity...). however, besides their size, the main distinction between llms and nmt models lies in the prompting method. llms necessitate a prompt to operate, making them entirely dependent on context. this is precisely the aspect we aim to explore: whether we can use the prompting mechanism, absent in nmt models, to alleviate gender bias. to date, significant research has been conducted on the translation capabilities of llms, as extensively documented in the literature @xcite @xcite @xcite @xcite . furthermore, several efforts have been made to identify and address bias in llms @xcite @xcite . however, the exploration of gender bias in the realm of mt and llms remains relatively scarce. this encompasses @xcite , which sought to leverage llms for gender-specific translations, and @xcite , which experimented with @xmath1, revealing how gpt models perpetuate biases even when explicitly prompted to provide alternative translations. additionally, @xcite examines bias between english and languages that exclusively use gender-neutral pronouns, and @xcite demonstrate through extensive manual analysis the potential of gpt-4 to produce gender-neutral translations for @xmath2. ## gender bias mitigation through prompting after observing that llms exhibit more gender bias than nmt models, we found it necessary to address this bias in llms. consequently, we have chosen to leverage prompting, as it is a distinctive feature of these models. therefore, the second stage of our work involves conducting exploratory research in a trial-and-error manner, aiming to identify a prompt that effectively mitigates bias in llms. for this experiment, we have selected the instruction-tuned model llama-2-7b-chat since it is more robust to complex prompts than its base version. in addition, in this stage, we have decided to focus solely on the gender coreference resolution task. ideally, our goal is to narrow the gap in gender scores with respect to nmt models, as this would represent a significant breakthrough. the procedure goes as follows: initially, we develop a range of prompts based on strategies outlined in the literature, including few-shot prompting, context-supplying, and chain-of-thought instructions. to assess the impact of these prompts, we test them on winomt and obtain gender-bias scores for each prompt. thereafter, the prompt that demonstrates the most considerable reduction in bias on winomt, as indicated by numerical genderbias scores, is evaluated on the remaining test sets (gold bug, . by doing so, we want to determine: firstly, the prompt's generalizability across the remaining gender-bias test sets, and secondly, if it affects the overall machine translation capabilities of the llm. 99 @xmath3 ## results after testing our top-performing prompt on the remaining gender-bias test sets, gold bug and must-she, we observe a significant reduction in gender bias within those test sets too. these results are detailed in sections a and b of the appendices. subsequently, all the three tables 3 , 4 , and 5 demonstrate a remarkable improvement in gender-bias scores, significantly reducing the upper bound in each test set compared to the best nmt model. this places the llm on par with nmt models in terms of gender bias manifestation. for example, on the winomt test set, the model achieves the second-best position in @xmath4. in must-she, the mitigation is less pronounced as this test set also encompasses other gender-related tasks, unlike winomt and gold bug, which focus solely on gender coreference resolution. regarding the mt metrics, we observe a small loss compared to the baseline when testing on flores-200 ( ## discussion initially, we believed that reducing gender bias through prompting would possibly be straightforward. however, it was surprising to find that the model only began effectively mitigating the bias after implementing the chain-of-thought approach. in fact, the results presented in ## conclusions this work investigates gender bias in the translation outputs generated by various llms through two distinct approaches. firstly, by benchmarking three base models ( ǎguila-7b, flor-6.3b and llama-2-7b) using different gender-bias test sets and comparing the results with state-of-the-art nmt models (m2m-100-1.2b, nllb-200-1.3b, mt-aina-en-ca, and google translate). secondly, by experimenting with the prompting mechanism of an instruction-tuned llm (llama-2-7b-chat) and trying to mitigate its gender bias in the output. this study is done in the @xmath5. results reveal the presence of gender bias across all models, with base llms exhibiting more gender bias than nmt models. moreover, the performance of all models correlates with gender stereotypes. in the absence of gender cues in the source sentence, they tend to generate predominantly male terms, while female terms are generated primarily when encountering female-stereotypical content. to mitigate this bias, prompting engineering techniques have been implemented in an instruction-tuned llm. after curating and testing several prompts, one prompt was identified that resulted in a significant reduction in gender bias, achieving impressive gender scores. the prompt follows a simplified chain-of-thought approach with 5-shots relying on anti-stereotypical content and increased female representation. this prompt enables the instructed llm to perform competitively in terms of gender scores, achieving results comparable to nmt models and even surpassing some of them. however, it is observed that using this prompt leads to a slight loss in the translation quality.