How sensitive are translation systems to extra contexts? Mitigating gender bias in Neural Machine Translation models through relevant contexts. (2022)

Abstract: Neural Machine Translation systems built on top of Transformer-based architectures are routinely improving the state-of-the-art in translation quality according to word-overlap metrics. However, a growing number of studies also highlight the inherent gender bias that these models incorporate during training, which reflects poorly in their translations. In this work, we investigate whether these models can be instructed to fix their bias during inference using targeted, guided instructions as contexts. By translating relevant contextual sentences during inference along with the input, we observe large improvements in reducing the gender bias in translations, across three popular test suites (WinoMT, BUG, SimpleGen). We further propose a novel metric to assess several large pre-trained models (OPUS-MT, M2M-100) on their sensitivity towards using contexts during translation to correct their biases. Our approach requires no fine-tuning, and thus can be used easily in production systems to de-bias translations from stereotypical gender-occupation bias. We hope our method, along with our metric, can be used to build better, bias-free translation systems.

## introduction despite the ongoing success of large pre-trained transformer @xcite based models in neural machine translation (nmt), these systems are immensely prone to various forms of gender biases in their learned representations. recent work @xcite has found out that a specific kind of gender bias exists the manager asked the librarian a lot of questions and thanked her in the end. ## approach to mitigate gender-stereotyped bias for occupations in nmt, we consider the approach of adding a context to the input, as a separate sentence either prepended or appended with the original input sentence. this context is generated using handcrafted templates, which contain unambiguous signals about the gender of the profession in the source sentence. in particular, we investigate the capability of nmt models to extract signals from these contexts and mitigate gender bias in the translation during inference. our approach can also be thought of as an in-context learning scheme, as popularized by brown et al. (2020). ## results & analysis in this section, we conduct a series of experiments and analysis to understand the viability of correcting gender bias in translation by using contexts. ## conclusion in this work, we propose a simple and effective approach to correct stereotypical gender associations for occupations in translations during inference. specifically, we add an unambiguous context to the input to state-of-the-art nmt models and observe that it enables the model to fix its own gender-bias towards gender-stereotyped occupations. popular nmt models, such as m2m-100 and opus-mt, can effectively learn "in-context" how to correct their own biased translations provided the relevant context along the input. future work could consist of automatically choosing the correct template to add to the model during inference, or even generating "prompts" dynamically @xcite .