Does Context Help Mitigate Gender Bias in Neural Machine Translation? (2024)

Abstract: Neural Machine Translation models tend to perpetuate gender bias present in their training data distribution. Context-aware models have been previously suggested as a means to mitigate this type of bias. In this work, we examine this claim by analysing in detail the translation of stereotypical professions in English to German, and translation with non-informative context in Basque to Spanish. Our results show that, although context-aware models can significantly enhance translation accuracy for feminine terms, they can still maintain or even amplify gender bias. These results highlight the need for more fine-grained approaches to bias mitigation in Neural Machine Translation.

## introduction neural machine translation (nmt) models tend to exhibit gender bias, originating from their training data @xcite . a typical example is the translation of gender-neutral professions in a language like english, into languages with differentiated feminine and masculine forms. in this case, nmt systems often produce translations that reflect genderstereotypical biases @xcite . beyond translation errors, bias perpetuation has a clear negative impact overall. several studies have addressed gender bias in nmt, highlighting various sources and manifestations of gender bias in nmt models. for example, @xcite and @xcite examined gender bias in commercial machine translation systems, revealing a systematic bias towards masculine translation. a variety of approaches have been explored to mitigate these effects, such as data augmentation by swapping gender-specific words @xcite , or the incorporation of gender tags in the input to guide the translation process @xcite . the use of context has also been studied as a potential solution, as context-aware models have been shown to significantly enhance translation quality for specific linguistic phenomena, including gender agreement @xcite . @xcite thus demonstrated that using artificial context specifying the expected gender could mitigate gender bias, while @xcite and @xcite suggested that real context could as well. however, a more detailed study is still warranted. in this work, we further explore the role of context in mitigating gender bias in nmt, by addressing the following question: does context always help mitigate bias in nmt or can it have bias perpetuation effects? to tackle this question, we studied two specific phenomena related to gender bias. first, we evaluated the performance of contextaware models in the translation of stereotypical professions from english into german and french, measuring translation accuracy on gender-based subsets of the data. our results in this case indicate that, although context-aware models lead to significantly more feminine forms, this was achieved mainly for professions that are stereotypically viewed as feminine, thus with limited bias mitigation or an actual increase in gender bias. we then studied the impact on gender bias of non-informative context in basque to spanish, i.e., where context lacks gender disambiguating information but nonetheless provides information that may impact the translation. in this case, our results showed significant increases in accuracy for masculine translation options, but notable losses for feminine ones. the use of context was thus detrimental in this case, exacerbating gender biases present in sentence-level models. although context can contribute positively to more accurate gender translation, our results highlight the complexity of gender bias in contextaware nmt systems and the need for more finegrained approaches to mitigate this type of bias. ## conclusions this study investigated the impact of context-aware models on mitigating gender bias in nmt, focusing on the translation of professions from english into german and french, as well as translation with uninformative context in basque to spanish. our results show that, although contextual models can significantly improve the overall transla-tion accuracy on gender-specific terms, this was achieved mainly over stereotypically feminine professions. the use of context actually increased the disparity between stereotypical genders in this case. non-informative context was also shown to increase gender-related bias in a domain with strong latent bias, when using context from a different domain had no such effects. these results underscore the need for more comprehensive approaches to bias in nmt, including more specific evaluations over balanced datasets. novel mitigation techniques and a deeper understanding of the impact of context will also be needed to further address translation bias.