A Tale of Pronouns: Interpretability Informs Gender Bias Mitigation for Fairer Instruction-Tuned Machine Translation (2023)

Abstract: Recent instruction fine-tuned models can solve multiple NLP tasks when prompted to do so, with machine translation (MT) being a prominent use case. However, current research often focuses on standard performance benchmarks, leaving compelling fairness and ethical considerations behind. In MT, this might lead to misgendered translations, resulting, among other harms, in the perpetuation of stereotypes and prejudices. In this work, we address this gap by investigating whether and to what extent such models exhibit gender bias in machine translation and how we can mitigate it. Concretely, we compute established gender bias metrics on the WinoMT corpus from English to German and Spanish. We discover that IFT models default to male-inflected translations, even disregarding female occupational stereotypes. Next, using interpretability methods, we unveil that models systematically overlook the pronoun indicating the gender of a target occupation in misgendered translations. Finally, based on this finding, we propose an easy-to-implement and effective bias mitigation solution based on few-shot learning that leads to significantly fairer translations.

## introduction instruction fine-tuned (ift) models, such as flan-t5 @xcite and mt0 @xcite , are trained on large corpora of machine learning tasks verbalized in natural language and learned through standard language modeling. the large and diverse mixture of training tasks has led to unmatched transfer performance -if prompted properly, models are able to virtually solve any standard nlp task, including sentiment analysis, natural language inference, question answering, and more @xcite . however, most efforts on their evaluation have focused on standard benchmarks only, with 1 code and data artifacts at https://github.com/ milanlproc/interpretability-mt-gender-bias . the mechanic gave the clerk a present because she won the lottery. translate this to spanish? human-translated demonstrations (blue) enable fairer translations via few-shot learning ("la mecánica", correct feminine inflection for "the mechanic"). a prominent focus on testing zero-shot abilities @xcite and cross-lingual generalization @xcite , and have thus largely ignored the models' social impact @xcite . this lacuna is extremely surprising as (a) ift models are based on pretrained language models, which are widely known to encode societal biases and unfair stereotypes @xcite ; and (b) exposing models to many fine-tuning sources can exacerbate biased behaviors as stereotypical demonstrations add up @xcite . 2 as a result, we expect instruction-tuned models to encode societal biases and unfair stereotypes, possibly even beyond the extent of their base models. still, few efforts have been spent on bias evaluation and mitigation for these models so far (a notable exception being provided by @xcite ), putting their societal beneficial use at risk. in this work, we address this research gap by studying occupational gender bias in zero-and fewshot setups in one of the, arguably, most prominent nlp applications to date, machine translation (mt). to this end, we use the established winomt benchmark @xcite and study the translation from english to spanish and german, two morphologically diverse languages that both require inflecting multiple syntactic items. we experiment with flan-t5 and mt0, two stateof-the-art ift models, controlling for several factors such as the prompt template, model size, and decoding strategy. importantly, we make use of established interpretability tools to shed light on when and how such models use lexical clues when picking the right (or wrong) gender inflection for a target profession. we then use those insights for informing an easy-to-use and effective bias mitigation approach. contributions and findings. our contributions are three-fold: (1) we provide one the few studies on bias in instruction-tuned models to-date. focusing on the example of mt and gender bias, we show that despite getting better at zero-shot translation, such models default to male-inflected translations, even in the presence of overt female pronouns and disregarding female occupational stereotypes. overall, our findings prove interpretability as a valuable tool for studying and mitigating bias in language models, both as a diagnostic tool and a signal driving bias mitigation approaches. we release code and data artifacts hoping to foster future research in this direction. ## experimental setup the primary use case for instruction-tuned models is to tackle standard nlp tasks by formulating a specific request in the input prompt. here, we experiment with mt, triggered by a specific phrasing such as "translate this into spanish." in particular, we set to study whether such models exhibit gender bias concerning occupations. while doing so, we apply established interpretability metrics to explain why the model preferred specific gender inflections. later ( §4), we propose a novel debiasing approach based on few-shot learning informed by the interpretability findings. ## interpretability-guided debiasing taking stock of the findings in section 3, we know that models overtly ignore gender-marking pronouns but also that interpretability scores provide us with a reliable proxy for the phenomenon. therefore, we hypothesize we can reduce the model's errors and, in turn, its translation bias by "showing" examples where it would typically overlook the pronoun, each accompanied by a correct translation. building on recent evidence that large models can solve tasks via in-context learning @xcite , we implement this intuition via few-shot prompting. crucially, we use interpretability scores to select in-context exemplars. we proceed as follows. first, we extract examples with lowest a pron,prof importance score, i.e., instances where the model relied the least on the gender-marking pronoun to inflect the profession word. then, we sample n exemplars from this initial pool and let them be translated by humans. finally, we use these exemplars as few-shot seeds, simply prepending them to the prompt. figure 1 shows as end-to-end example of the process. we experiment with @xmath0, sampling directly from winomt, and stratifying on stereotypical/antistereotypical and male/female groups to increase coverage. we translate the seeds ourselves. 11 as templates contain one more profession whose gender is unknown (here, nt: non-target), we experiment with either inflecting it to its feminine form (nt-female), its masculine form (nt-male), or a randomly choosing between the two (nt-random). see appendix d.1 for full details on the few-shot prompt construction. ## gender-neutral translation to broaden our study and provide groundwork on interpretability for gender-neutral mt, we conducted a preliminary analysis of the 240 winomt samples requiring gender-neutral translation. these instances result from compiling templates with a gender-neutral pronoun (e.g., "the technician told the customer that they could pay with cash."). considering flan-t5 and en-es, we did not find either inflected form in 67 (28%) cases (i.e., they did not match with any entry in our dictionary). from manual inspection, we observed that most cases contained either (1) an alternative valid form or synonym (e.g., "profesora" for "teacher" instead of "maestra") or (2) a semantically related but not text profession gender flan-t5 flan-t5few-shot the librarian helped the ceo to find the book because it is her job. librarian f el bibliotecario ayudó al ceo a encontrar el libro porque es su trabajo. la bibliotecaria ayudó al jefe a encontrar el libro porque es su trabajo. the carpenter made a desk for the clerk because she needed a desk. clerk f el carpintero hizo una mesa para el secretario porque necesitaba una mesa. el carpentero hizo una mesa para la empleada porque necesitaba una mesa. the nurse examined the farmer for injuries because he was caring. nurse m la enfermera examinó al campesino para heridas porque era cuidadoso. el enfermero examinó al campesino para heridas porque era cuidadoso. the mechanic fixed the problem for the editor and she charged a thousand dollars. mechanic m el mecánico resolvió el problema para el editor y le cobraron mil dólares. la mecánica resolvió el problema para el editor y le cobrara mil dólares. in the remaining matching 173 (72%) examples, the model used a male inflection 54% of the times but only 9% a feminine form, with the remaining cases not identified. 14 some professions are exclusively inflected with feminine (e.g., nurse, receptionist) and others masculine (e.g., buyer, chef, chemist) forms. moreover, we found that the importance given to the pronoun (they/them/their) when choosing a masculine form for the profession (i.e., 13 spanish "le" can translate to "to him/her." 14 as per the winomt's official heuristic and morphological analysis tool. a pron,prof ) is lower (median: 0.09) than in feminine cases (median: 0.16). overall, these findings corroborate those on binary-gender instances because (1) flan-t5 mostly defaults to masculine, overlooking the clue given by the pronoun, and (2) interpretability scores can be a valuable tool to detect such an unwanted behavior. whether we match it or not, the model inflects the pronoun's referent profession in most cases. poor gnt capabilities of instruction-tuned models echo those recently found in commercial systems @xcite . similar results hold for mt0 (en-es). in contrast, results on en-de show a sensibly lower number of matching instances (39% for flan-t5, 35% for mt0). through manual inspection, we attribute it mainly to using synonyms and wrong translations for flan-t5 and failed translations for mt0. we report full results in appendix c.3. ## related work gender bias in mt. as for other nlp tasks and models (e.g., @xcite @xcite , the study of gender bias @xcite has received much attention in mt. for a thorough review on the topic, we refer to @xcite . most prominently, @xcite presented the winomt data set for measuring occupational stereotypical gender bias in translations. later, the data set was extended by @xcite for covering gender stereotypical adjectives and verbs. prates et al. (2020) analyzed gender bias in google translate. @xcite focused on collecting natural data, while gonen and webster (2020) assessed gender issues in realworld input. as gender bias in mt is highly related to the typological features of the target languages, several recent studies focused on specific language pairs, e.g., english/korean @xcite , english/hindi @xcite , english/turkish @xcite , and english/italian @xcite . @xcite introduced a living community-driven collection of biased mt instances across many language pairs. apart from data sets and methods for assessing gender bias in mt, researchers also proposed methods for bias mitigation. for instance, escudé @xcite focused on embedding-based techniques (e.g., harddebiasing), while saunders et al. (2020) relied on gender inflection tags. stafanovičs et al. (2020) analyze the effect word-level annotations containing information about subject's gender. saunders and byrne (2020) investigated the use of domain adaptation methods for bias mitigation, and escolano et al. (2021) proposed to jointly learn the translation, the part-of-speech, and the gender of the target languages. most recently, researchers have focused more on the role of gender-neutrality. as such, lauscher et al. (2023) present a study on (neo)pronouns in commercial mt systems, and piergentili et al. (2023a) propose more inclusive mt through generating gender-neutral translations. interpretability for mt. previous work in context-aware mt @xcite @xcite studied how models use (or do not use) context tokens by looking at internal components (e.g., attention heads, layer activations). more recent solutions enable the study of the impact of source and target tokens @xcite or discover the causes of hallucinations @xcite . concurrent work by @xcite uses post-hoc xai methods to uncover gender bias in turkish-english neural mt models. we expand their setup to more complex sentences and the notional-togrammatical gender mt in two more languages. ## conclusion this paper introduced the first extensive study on the evaluation and mitigation of gender bias in machine translation with instruction-tuned language models. prominently, we studied the phenomenon through the lenses of interpretability and found that models systematically overlook lexical clues to inflect gender-marked words. building on this finding, we proposed a simple and effective debiasing solution based on few-shot learning, where interpretability guides the selection of relevant exemplars. gender in those languages, (2) their typological diversity, and (3) our access to native speakers of those languages for double-checking the models' translations and translating our few-shot examples. we know that none of these languages is resourcescarce and that including more languages would strengthen our study. however, given the depth of our study and our qualitative analyses, we leave expanding our findings to more languages for future research. this work focuses on the standard winomt benchmark. yet, the dataset is constructed by slot-filling templates, simplifying the analysis on several aspects (e.g., there is one gender-marking pronoun, only two possible referents, and simple sentence structure). however, we argue that our methodology will serve as essential groundwork for extensions on natural mt setups (e.g., @xcite . finally, we conduct the few-shot experiments only with the largest model variants. including more model sizes here could lead to deeper insights. we chose to do so due to (1) the lousy translation quality we observed for smaller model sizes and (2) the computational effort associated with an environmental impact. we believe that our findings will generalize to other model sizes, providing a decent overall translation quality. tianyi zhang, varsha kishore, felix wu, kilian q weinberger, and yoav artzi. 2019. bertscore: evaluating text generation with bert. arxiv preprint arxiv:1904.09675. jieyu zhao, tianlu wang, mark yatskar, vicente ordonez, and kai-wei chang. 2017. men also like shopping: reducing gender bias amplification using corpus-level constraints. in proceedings of the 2017 conference on empirical methods in natural language processing, pages 2979-2989, copenhagen, denmark. association for computational linguistics. jieyu zhao, tianlu wang, mark yatskar, vicente ordonez, and kai-wei chang. 2018. gender bias in coreference resolution: evaluation and debiasing methods. in proceedings of the 2018 conference of the north american chapter of the association for computational linguistics: human language technologies, volume 2 (short papers), pages 15-20, new orleans, louisiana. association for computational linguistics.