A Tale of Pronouns: Interpretability Informs Gender Bias Mitigation for Fairer Instruction-Tuned Machine Translation
Authors: Giuseppe Attanasio
Flor Miriam Plaza del Arco
Debora Nozza
Anne Lauscher
Venue: EMNLP 2023 Main (2023)
Keywords: machine translation
gender bias
ethics
interpretability
instruction fine-tuned language models

Abstract: Recent instruction fine-tuned models can solve multiple NLP tasks when prompted to do so, with machine translation (MT) being a prominent use case. However, current research often focuses on standard performance benchmarks, leaving compelling fairness and ethical considerations behind. In MT, this might lead to misgendered translations, resulting, among other harms, in the perpetuation of stereotypes and prejudices. 
In this work, we address this gap by investigating whether and to what extent such models exhibit gender bias in machine translation and how we can mitigate it.
Concretely, we compute established gender bias metrics on the WinoMT corpus from English to German and Spanish. We discover that IFT models default to male-inflected translations, even disregarding female occupational stereotypes. 
Next, using interpretability methods, we unveil that models systematically overlook the pronoun indicating the gender of a target occupation in misgendered translations.
Finally, based on this finding, we propose an easy-to-implement and effective bias mitigation solution based on few-shot learning that leads to significantly fairer translations.

A Tale of Pronouns: Interpretability Informs Gender Bias Mitigation for Fairer Instruction-Tunen Machine Translation
Giuseppe Attanasio ?
Flor Miriam Plaza-del-Areo Debora Nozza 9 Anne Lauscher
Bocconi University, Milan, Italy University of Hamburg, Hamburg, Germany giuseppe at attanssioenninbocoom .tt
Abstract
The rochanic, gave the clark present because
Recent instruction fine-tuned models can solve multiple NLP tasks when prompted to do so, with machine translation (MT) being a promi- nent use case. However, current research often focuses on standard performance benchmarks, leaving compelling fairness and ethical consid- erations behind, In MT, this might lead to mis- gendered translations, resulting, among other harms, in the perpetuation of stereotypes and prejudices. this work, we address this gap by investigating whether and what extent such models exhibit gender bias in machine transla- tion and how we can mitigate Concretely, compute established gender bias metries the WinoMT corpus from English to German and Spanish. We discover that IFT models de- fault to male-inflected translations, even disre- garding female occupational stereotypes. Next, using interpretability methods, unveil that models systematically overlook the pronoun indicating the gender of a target occupation in misgendered translations. Finally, based on this finding, we propose an casy-to-implement and effective bias mitigation solution based on few- shot learning that leads to significantly fairer translations.
Kia Aon inttery.
Trame late this Spanish??
El necánioo dic regalo wap leado parque gand
The mechanis, gave the clark present bicause she mon The lottery. Translate this to Spanish?
Le escénice 1e die un regalo al emploado porcue gand 1a loterio.
Figure Model translation suffering from occupational gender bias (top): coreferents underlined. Our inter- pretability analysis on pronouns and professions (pur- ple boxes) informs the selection of debiasing examples. Human-translated demonstrations (blue) enable fairer translations via few-shot learning ("La mecánica", cor- rect feminine inflection for "the mechanic").
a prominent focus on testing zero-shot abili- ties (Chung et al., 2022) and cross-lingual general- ization (Muennighoff et al., 2023b), and have thus largely ignored the models' social impact (Hovy and Spruit, 2016). This lacuna is extremely surpris- ing as (a) IFT models are based on pretrained lan- guage models, which are widely known to encode societal biases and unfair stereotypes (Nadeem et al., 2021; Nozza et al., 2021, inter alia); and (b) exposing models to many fine-tuning sources can exacerbate biased behaviors as stereotypical demonstrations add up (Srivastava al., 2022).² 2 As result, expect instruction-tuned models encode socictal biases and unfair stereotypes, pos- sibly even beyond the extent of their base models. Still, few efforts have been spent on bias evaluation and mitigation for these models so far (a notable
1 Introduction
Instruction fine-tuned (IFT) models, such as Flan- T5 (Chung et al., 2022) and mTO (Muennighoff et al., 2023a), are trained large corpora of ma- chine learning tasks verbalized in natural language and learned through standard language modeling. The large and diverse mixture of training tasks has led to unmatched transfer performance prompted properly, models are able to virtually solve any standard NLP task, including sentiment analysis, natural language inference, question an- swering, and more (Sanh et al., 2022).
However, most efforts their evaluation have focused standard benchmarks only, with
'Code and data artifacts at httpp://githh/ com/ Diiaw..Proc/ineerreeaallll metelite--tt-eeeee--biiss biider-bder..
2 For a reference, FLAN models are trained on a mixture of 1.8K tasks (Chung et al., 2022).

---

exception being provided by Akyürek al. (2022)), putting their socictal beneficial use risk.
with few as four human-translated exemplars.
Overall, our findings prove interpretability as valuable tool for studying and mitigating bias in language models, both as a diagnostic tool and a signal driving bias mitigation approaches. We release code and data artifacts hoping to foster future research in this direction.
In this work, we address this research gap by studying occupational gender bias in zero- and few- shot setups in one of the, arguably, most promi- nent NLP applications to date, machine transla- tion (MT). To this end, we use the established WinoMT benchmark (Stanovsky et al., 2019) and study the translation from English to Spanish and German, two morphologically diverse languages that both require inflecting multiple syntactic items. We experiment with Flan-T5 and mTO, two state- of-the-art IFT models, controlling for several fac- tors such the prompt template, model size, and decoding strategy- Importantly, make use of established interpretability tools to shed light on when and how such models use lexical clues when picking the right (or wrong) gender inflection for a target profession. We then use those insights for informing an casy-to-use and effective bias mitiga- tion approach.
Experimental Setup
The primary use case for instruction-tuned models is to tackle standard NLP tasks by formulating specific request in the imput prompt. Here, we ex- periment with MT, triggered by specific phrasing such as "Translate this into Spanish."
In particular, we set to study whether such mod- els exhibit gender bias concerning occupations. While doing so, we upply established interpretabil- ity metrics to explain why the model preferred spe- cific gender inflections. Later ($4), we propose novel debiasing approach based on few-shot learn- ing informed by the interpretability findings.
Contributions and Findings. Our contributions are three-fold: (1) we provide one the few studies on bias in instruction-tuned models to-dute. Fo- cusing on the example of MT and gender bias, we show that despite getting better at zero-shot transla- tion, such models default to male-inflected transla- tions, even in the presence of overt female pronouns and disregarding female occupational stereotypes. (2) To our knowledge, we are among the first to acknowledge the potential of interpretability meth- ods to study IFT language models and why they produce biased predictions. Based on attribution interpretability, we find that models systematically ignore the pronoun (and thus, the conveyed gender information) when producing misgendered trans- lations. In contrast, correctly translated profes- sions relate to higher contributions of the pronoun in the choices taken. (3) Based on our insights, we propose a novel and easy-to-us. bias mitiga- tion method informed by interpretability scores! The differences in the attribution scores lead us to hypothesize that models that are used in a few- shot setup would benefit from provided translations mostly, if exactly in those examples they would normally overlook the pronoun. We hence propose few-shot learning-based debiasing approach, in which we use interpretability scores select the in- context exemplars. Figure shows an example of the resulting approach. The solution is simple-yet- effective, leading to significantly fairer translations
2.1 Gender Bias in Machine Translation
Bias Statement. We expect to inflect gender in occupation words according to overt contextual and lexical clues. Instead, a biased model is one, which relies on stereotypical gender-role associa- tions. Both open source and commercial MT sys- tems have been shown rely on these associations, with a marked tendency to associate women with less prestigious roles (e.g., Stanovsky al.. 2019; Saunders and Byrne, 2020; Chung et al., 2022, inter alia). Echoing Blodgett et al. (2020), such systems risk representational harms, as they portray women in less favorable light than men.
Dataset. We base our experiments on WinoMT (Stanovsky et al., 2019), a well-known benchmark for evaluating gender bias in MT.
The collection is based on templates. Each in- stance mentions two professions and a pronoun coreferent to one of them (see Figure for an ex- ample). When translating from English, notional gender language. to Spanish German, two gram- matical gender languages, the pronoun dictates the coreferent inflection because of syntactic agree- ment. For example, the sentence Figure with "she" as the leading pronoun should translate to "La

---

mecánica"" (eng: the female mechanic). 3 The task is challenging as many of the occupations found WinoMT have stereotypical gender-role associ- ations in society (e.g., nurse to women, developer to men). Indeed, the WinoMT corpus distinguishes between stereotypical and atti-sterellypical tem- plates, which permits us to derive more insights.
Interpretability Signals for Gender Bias. Word attribution scores provide a clear, measurable quan- tity to inspect and debug machine translation mod- els. We, therefore, study such scores and relate recurring patterns to misgendered translations.
We extracted several word attribution scores. First, observe the "alignment" importance be- tween translations Oprof.prof- the importance of the English profession word for the target profession (mechanic and mecánico in Figure 1). Then, we also report a control attribution score Octrl,prof as the importance of the first source token ("The") toward the target profession.
Evaluation Metrics. We evaluate gender bias using the measures proposed along with WinoMT Stanovsky et al. (2019). The authors conceptualize bias as differences in group performance indicated by Ag and As. Ag corresponds to the difference in performance (Fi macro) between translations with female and male referent. As measures the difference in performance between stereotypical and anti-stereotypical examples, as per US Labour statistics in Zhao et al. (2018). Additionally, we also report the overall accuracy.
The most promising aspect we study is the con- tribution score apron.proft i.c. the importance the source pronoun has in choosing the final form of the target profession (she and mecánico in Fig- ure 1). Intuitively, the models need to use this overt lexical clue to choose the correct inflection. Note that extracting Oprof.pron requires aligning the source and target sentences because WinoMT does not release the profession position within the sentence, while the pronoun position is given. We tested both Fast Align (Dyer et al., 2013) and cus- tom dictionary matching approach and proceeded with the latter because of better quality. In partic- ular, we prompted GPT-3.5 (Ouyang et al., 2022, accessed in early June 2023) to translate all the target professions in WinoMT into the masculine and feminine inflected forms in Span- ish and German.* + After checking and fixing any grammatical errors, we perform hard string match- ing of the MT output against the word translations.
2.2 Interpretability for Gender Bias in MT
Interpretability for Translations. For every translated instance, we compute and collect word attribution interpretability scores from target to source tokens. Word attribution scores (i.e., saliency scores), measure each input token's 's con- tribution to the choice of any translation token.
We compute word attributions follows: first, we extract raw attribution scores using Integrated Gradients (IG; Sundararajan et al., 2017), com- monly used feature attribution algorithm. Let Ar = R. xT,xh be the output of IG, a matrix of attribution scores, where S, and S₁ are the number of source and target tokens, respectively, and h is the hidden dimension of the input embeddings.
2.3 Instruction Line-TTund Models
Next, we aggregate scores applying two consec- utive functions, i.e., = g(((A,)), where f xh. + SxTxx and g SxTxxx + R aggregate raw scores over each word's sub-tokens and the hidden dimension, respectively. S and T are the number of source and target words as split by whitespaces, respectively. We set f to take the highest absolute value in the span, preserving the sign, and g to the Euclidean norm. We provide more details in Appendix B.I. As a result, each item ai.j E A will reflect the contribution that source word had in choosing the target jord
Base Models. We study variants of two recently introduced IFT models.5
Flan-75 (Chung al., 2022) is sequence-to- sequence language model based on the T5 archi- tecture (Raffel et al., 2020). The model has been pre-trained with standard language modeling ob- jectives and subsequently fine-tuned on the FLAN collection (Longpre et al., 2023), counting more than 1,800 NLP tasks in over 60 languages. We test the 80M (Small), 250M (Base), 780M (Large),
"This dictionary contains single entry pair per profession. Hence, we do not match professions against multiple correct translations (e.g., we match the Spanish "maastro/maestra" but not "profesor/profesora"" for "teacher").
'WinoMT includes a small set of examples using neutral pronouns to test gender-neutral translation (GNT). As GNT is a developing field, we report preliminary insights on GNT cases ($5). Otherwise, we restrict the rest of the work to binary gender.
'We follow the nomenclature from Chung et al. (2022) and refer to these models as instruction fine-tuned inseructionfunettmmee models, in favor of alternatives such as multi-task prompted models. See Appendix A for more details.

---

Model
C-22
C-20
BERTScore
template "[src_text) Translate this {tgt_lang)?". Most importantly, we found that model size is key to enabling zero-shot translation (see Table 9). This crucial finding suggests that smaller models (i.e., <11B) do not yield trans- lations of sufficient quality and their usage in a zero-shot setup can be problematic. We will focus the largest models (XXL variants) for the rest of the paper and simply refer them Flan-T5 and mTO for conciseness.
Marian NMT
0.85
0.42
0.89
Flan-T5
0.78
0.18
0.86
mTO
0.80
0.27
0.82
Marian NMT
0.80
0.43
0.85
Flan-T5 0.75
0.31
0.82
mTO
0.75
0.42
0.81
Table I: COMET-22, COMET-20, and BERTScore per- formance on Europarl in En-Es (top) and En-De (bot- tom). Best results per language in hold.
Table reports the zero-shot performance of Flan-T5 and mTO compared to supervised base- line Marian NMT models (Junczys-Dowmunt et al., 2018)². Flan-T5 and mTO slightly underperform supervised baselines. However, they show com- petitive zero-shot performance measured by COMET-22 and BERTScore. COMET-20 qual- ity estimation metric show less encouraging results, especially for Flan-TS (see Table 9 for a full break- down). Overall, these results suggest that zero-shot translation with instruction-tuned models is almost as valid as specialized supervised models, further motivating their adoption in real use cases.
3B (XL), and IIB (XXL) model sizes. mTO (Muennighoft et al., 2023b) a mTS model (Xue et al., 2020) fine-tuned on xP3, covering 13 tasks across 46 languages with English prompts. We test the 300M (Small), 580M (Base), 1.2B (Large), 3.7B (XL), and 13B (XXL) model sizes. Both model types have been fine-tuned verbaliz- ing NLP tasks into a text-to-text format and using standard encoder-decoder language modeling loss. Morcover, FLAN and xP3 training mixtures both contain machine translation tasks. For all open models, we use the Hugging Face Transformers library (Wolf et al., 2020).
3.2 Gender Bias in Instruction- Tuned Models
Table 2 reports the results WinoMT gender bias metrics. We report several interesting find- ings. Generally, Flan-T5 competitive. For both languages, significantly outperforms mTTO in terms of accuracy and bias evaluation. More- over, considering commercial systems reported in Stanovsky et al. (2019), GPT-3.5 (Ouyang et al., 2022, gpt-3.5-turbo accessed in early June 2023), and our supervised baseline, Flan-T5 achieves the best accuracy and Ag in En-Es, and the best As in En-De. However, it falls severely short on As En-Es, where the supervised Marian NMT model tops the board. We addressed this weakness using few-shot learning (54).
Model Configuration and Tuning. We consider two standard prompt templates and five decoding strategies to account for possible variations with instruction-tuned models. See Appendix C.2 for details. In order to assess the translation quality and select the best instruction-tuned model config- uration, we perform extensive evaluation within benchmark evaluation framework.
We use the state-of-the-art Europarl corpus (Koehn, 2005) to evaluate zero-shot translation quality. We use the benchmark evaluation metrics COMET (reference-baste
-22 (Rei et al., 2022) and reference-free -20 (Rei et al., 2020)) and BERTScore (Zhang al., 2019). We also include BLEU (-2 and -4) (Papineni et al., 2002) for com- parison with prior work.
As for negative findings, we see that mTO re- tains high occupation biases. It is the second worst system for accuracy, Ag and As in En-Es, with similar results in En-De. Notably, zero-shot translations from GPT-3.5 are biased and worse than supervised baselines and instruction-tuned Flan-T5. Despite the interesting finding, we do not explore GPT-3.5 further since the attribution
3 Results
3.1
General Translation Quality
The results on EuroParl (Appendix C.2) show that the best overall quality is obtained with beam search decoding (n=4, no sampling) and the prompt
EEn-Es:
http://ggggn.afee..////enen/naaa httt::://huggingfaee co/
opus-nt-en-es. En-De: Hel sinkiiNNoLL//pus -ttt------eppssttteeeeeeeeeeeee
"We use the WMT 06 test splits at https://www statmt. arg/wmtt6/shal thared-task
"This negative finding is even more significant considering that systems from Stanovsky et al. (2019) date back to 2019.

---

Spanish
German
Model
Acc
AG
Ag
Acc
AG
As
Google Translate
53.1
23,4
21.3
59.4
12.5
12.5
Microsoft Translator*
47.3
36.8
23.2
74.1
0.0
30.2
Amazon Translate"
59.4
15.4
22.3
62.4
12.0
16.7
Marian NMT
56.8
16.9
19.7
62.0
9.9
15.2
GPT-3.5
55.2
23.1
48.5
48.3
25.2
24.6
Flan-T5-XXL
65.1
7.2
35.1
66.9
2.7
-0.2
mTO-XXL
52.5
27.8
42.7
56.3
26.1
25.6
Table 2: Gender bias evaluation WinoMT. "Results reported from Stanovsky et al. (2019).
techniques require access to the model weights. Overall, these findings suggest that instruction- tuned models can reasonably solve the task in zero-shot setup, with Flan models being superior mTO.
accuracy and Agraf-praf-
More insightful findings can be derived by the word attribution score dpron.prof- i..., the source pronoun importance for translating the gendered profession. Intuittielly, source pronoun should be the model's primary source of information for se- lecting the correct gender inflection. If we ob- serve low values for this score, we can assume the model has ignored the pronoun for translat- ing. This pattern is especially true for stereotypical male cases: despite their high accuracy, apron.prof scores are low. We observed an opposite trend for stereotypical female cases, where apron.prof scores are the highest, but accuracy is low. Interestingly, Opron.prof is highly asymmetrical between female and male cases. In six out of eight (model, lan- guage, and stereotype) groups, Opron.proff is higher for females than males. Regarding stereotypical vs. anti-stereotypical occupations, Opron.prof is higher for the latter on three out of four model- language pairs. This statistic supports the intuition that anti-stereotypical cases are where the model is most challenged, particularly for female profes- sions, which consistently have the lowest accuracy. These findings, taken together, reveal a concerning bias the way professions are portrayed in the models. Even after making an extra effort to con- sider pronouns, professions are frequently trans- lated into their male inflection, even when they would be stereotypically associated with the fe- male gender.
3.3 Inspecting Word Attribution Scores
Word attribution scores give us additional insights into the model's biased behavior. Table 3 shows the average word attribution scores introduced in Sec- tion 2.2 grouped by model, language, gender, and stereotypical and anti-stereotypical cases. The ta- ble also provides disaggregated accuracy for better understanding. Using our fictionary-basee string matching. we found the target profession (i.e., in- flected in either of the two forms) in 64% (En-Es) and 39% (En-De) for Flan-T5 and 70% and 49% for mT0."
Male cases are always associated with the high- est accuracy, with a difference of 21 1% and 62% be- tween male and female cases for Flan-T5 and mTO, respectively. Moreover, stereotypical male cases hold the highest performance across all groups This finding highlights (1) strong tendency to default to masculine forms and (2) that male stereo- average. These results confirm those obtained by observing Ag and As in the previous paragraph.
typical cases easier translate
In three out of four cases, stereotypical male cases also hold the highest Oprof-prof value. How- ever, while the most accurate cases are those in which the source-target profession attribution is the strongest, the opposite is not true (m'O0, En-Es, stereo, M has not the highest aprof.pre prof). Therefore, we conclude there no clear correlation between
Finally, we studied how aprof and poronn.roo relate to translation emror. Specifically, we computed the average Oprof.prof and apron.prof across all correctly and non-correctly translated ex- amples and measured their relative difference. Ta-
'Manual inspection revealed that fewer matches En-De are due to the model's frequent use of synonyms, English profession words, or wrong translation.
1'ehe word attribution score created as a control variable, Octel,prof: does not show any clear trend as expected.

---

Model Lang
Stereotypical
Gender
Aetri.prof
dyron.prof
Acc
F
0.192
0.163
0.174
39.49
Anti
M
0.142
0.166
0.150
79.97
Es
F
0.155
0.157
0.161
76.39
Stereo
M
0.187
0.167
0.162
94.95
Flan-T5
F M
0.244
0,145
0.254
68.10
Anti
0,209
0.133
0.221
77.08
De
F
0.231
0.129
0,266
64,65
Sterco
M
0.218
0.164
0.149
80.18
0.240
0.132
0.201
11 11.52
Anti
M
0,201
0.149
0.177
86.02
Es
F
0,203
0.143
0.203
36.62
Sterco
M
0,236
0,131
0.189
94.32
mTO
0.230
0.137
0.199
21.65
Anti
M
0.220
0,124
0.210
86.15
De
0.230
0.121
0.244
38.89
Stereo
M
0.219
0.141
0.165
91.92
Table 3: Word attribution scores and accuracy (%) disaggregated by model, language. stereotypical and anti- stereotypical, and expected gender inflection. Highest values each (model, language) group bold.
Stereotypical
Gender
A proffprof
pron.prof
us with a reliable proxy for the phenomenon. Therefore, hypothesize
F
+3.70
-14.23
reduce the model's errors and, in turn, its translation bias by "showing" examples where it would typically overlook the pronoun, each accompanied by cor- rect translation. Building on recent evidence that large models can solve tasks via in-context learning (Brown et al., 2020b), we implement this intuition via few-shot prompting. Crucially, we use inter- pretability scores select in-context exemplars. We proceed as follows. First, extract exam- ples with lowest Opron.roo importance score, i.c., instances where the model relied the least on the gender- pronoun to inflect the profession word. Then, we sample N exemplars from this initial pool and let them be translated by humans. Finally, we use these exemplars as few-shot seeds, simply prepending them to the prompt. Figure shows as end-to-end example of the process.
Anti
M
-3.46
+5.03
F
+2.36
-13.80
Stereo
M
-0.04
-2.58
Average
+1.14
-6.40
Table 4: Relative difference (%) between correct and wrong translations for mean Oprof.rrff and apron.prof- Results averaged over models and languages and dis- aggregated by stereotypical setup and expected gender inflection. Positive values indicate that the score is higher in correct translations.
ble 4 reports the results. Aprof.prof does not show any clear associations with errors, and it is hard to relate it to biased behaviors. @pron.prof- on the other hand, shows again high asymmetry between female and male cases. Interestingly, models at- tend to the source pronoun sensibly less when wrongly translating female referents (-14% in both stersottrrtt and stereotypical cases), but the same is not valid for male cases.
We experiment with N=4, sampling directly from WinoMT, and stratifying on stereotypical/ant- stereotypical and malc/female groups to increase coverage. We translate the seeds ourselves." !! As templates contain one more profession whose gen- der is unknown (here, NT: non-target), we experi- ment with either inflecting it to its feminine form (NT-Female), its masculine form (NT-Male), randomly choosing between the two (NT-Random).
All these results support the use of ad-hoe in- terpretability methods for discovering word attri- bution scores associations with desirable (or un- desirable) behavior, thereby serving proxies for subsequent interventions.
See Appendix D.1 for full details on the few-shot prompt construction.
4 Interpretabillty-Guieedd Debiasing
Taking stock of the findings Section we know that models overtly ignore Londer-marking pro- nouns but also that interpretability scores provide
As a baseline debiasing approach, we sample
"There is one Spanish and German native speaker among the authors.

---

Model
Acc
Ac
As
by Flan-TSrew-Shot. In both stereotypical and non- stereotypical examples, we observe correct shift in articles ("EI" for male, "La" for female) and gen- der inflection corresponding to the profession (e.g., "the librarian" - "el bibliotecario" (male), "alle) "la bib- liotecaria" (female)). Interestingly, while Flan-T5 translates poorly the profession "clerk" with "el sec- retario" (second row), Flan-T3rew-shoo chooses the right word and gender inflection ("la empleada"), We attribute this improvement translation to the presence of the profession "clerk" in the few-shot examples, which likely allows the model to learn the correct profession translation.
Flan-T5
65.1
7.2
35.1
68.9
4.1
22.2
Flan-T'5 Flan-TSSrasshet
72.2*
2.1
19.6
Flan-T5
67.5
2.3
-1.2
Flan-T55ee-stee
67.4
5.5
24.4
Flan- -Son-TSron-shot
69.8
2.3
-10.4
Table 5: Comparison of zero-shot and interpretabilliy- guided few-shot debiasing with Flan-TS in En-Es (top) and En-De (bottom). Best results per language in bold. p I .011 : .05
and translate random examples from WinoMT (Random). For a fair comparison with the interpretability-guui sampling, we stratify ran- dom sampling too and build the prompts likewise." 12
We also observe the behavior of the Flan-TS and Flan-T5rew-Shin models across stereotypical and anti-stereotypical examples. Using the few- shot debiasing Spanish, the model demonstrates a higher success rate in correcting the profession translations associated with anti-stereotypical ex- amples (235) compared to stereotypical examples (80) out of a total of 348 identified examples.
4.1 Results
Table 5 reports the comparison between Flan-T5 in zero-shot and using our debiasing approach. We report the best NT variants, which is NT-Female for Spanish and NT-Male for German. See Ap- pendix D for full results.
Finally, we report results aggregated by profes- sion in Table 7 for En-Es. By looking the pro- fessions whose Ag changed the most, we see that appraiser, therapist, sheriff, designer, and edi tor achieved the most significant improvement in correct gender inflection with Flan-T5 On the other hand, veterinarian. teenager, resident, farmer, and advisor worsened. L.e.. despite showing overall fairer translations, improve- ments with Flan-T5Few-Shoo are uneven across pro- fessions. This finding calls for more fine Ane-grained inspections and overall datasct-level assessments.
In En-Es, choosing in-context examples guided by interpretability leads to a strong improvement in accuracy (+7.1), Ag (-5.1), and As (-15.5). Im- provements in En-De are less marked, and As gets worse (+9.2 in absolute value). However, lower As might be artificially given by lower accuracy (Saun- ders and Byrne, 2020), as it happens for Flan-T5 zero-shot compared Flan- T5 Moreover, our approach leads to significant improvement over random sampling (see Appendix D.2 for details on significance).
Gender-Neutral Translation
Overall, these findings prove that interpretabil- ity scores, here ppron.prof= can serve reliable signal make fairer translations. We highlight how such improvements are enabled by a simple solution that requires no fine-tuning and only four human-wrinn written examples.
To broaden our study and provide groundwork on interpretability for gender-neutral MT, we conducted preliminary analysis of the 240 WinoMT samples requiring gender-neutral transla- tion. These instances result from compiling tem- plates with gender-neutral pronoun (e.g., "The technician told the customer that they could pay with cash."). Table 10 provides detailed overview of the results for Flan-T5 and mTO.
4.2 Qualitative Analysis
This section provides qualitative analysis of the results obtained by Flan-T5rew-Shet in the En-Es setup, compared with the zero-shot Flan-T5.
Considering Flan-T5 and En-Es, we did not find either inflected form in 67 (28%) cases (i.c., they did not match with any entry in our dictionary). From manual inspection, we observed that most cases contained either (1) an alternative valid form or synonym (c.g., "profesora" for "teacher" instead of "maestra") or (2) semantically related but not
Table 6 illustrates instances of wrong gender inflection in zero-shot translation (Flan-T5), con- trasting them with the accurate inflection achieved
12 Among the three possible configurations, i.c., Random- NT-Male, Handom--T-Feemle and Random-NTT Random, -anadomm-T- we report here the latter.

---

Text
Profession
Gender
Flan-T5
Flan-T3ro-sto:
The librarian helped the CEO find the book be- cause it her job.
librarian
F
bibliotecario ayudó al CEO encontrar el libro porque trabajo.
La bibliotecaria ayudó al jefe encontrar el libro porque trabajo.
The carpenter made a desk for the clerk be- cause she needed desk.
clerk
F
El carpintero hizo una mesa para el secretario porque necesitaba una mesa.
El carpentero hizo una mesa para la empleada porque necesitaba una mesa.
The nurse examined the farmer for injuries be- cause he was caring.
nurse
La enfermera cxaminó al campesino para heridas porque era cuidadoso.
enfermero examinó al campesino para heridas porque era cuidadoso.
The mechanic fixed the problem for the editor she charged a thousand dollars.
mechanic
mecánico resolvió el problema para editor y cobraron mil dólares.
La meeánica resolvió el problema para editor y cobrara mil dólares.
Table 6: Gender misalignment examples in Flan-T5 En-Es translation vs. correct alignment in Flan-T5 TSseu-shot- Two stereotypical (top) and two aati-stereotypicll (bottom) examples are shown. Target profession bold.
Profession
Flan-Ts
Flan-Te Man-T5rrw-Shn Trw--shot
A (J)
Opron.prof) is lower (median: 0.09) than in femi- nine cases (median: 0.16). Overall, these findings corroborate those on binary-gender instances be-
appraiser
0.41
0.17
-0.25
therapist
0.56
0.41
-0.14
sheriff'
0.52
0.45
-0.06
cause (1) Flan-T5 mostly defaults to masculine,
designer
0.64
0.59
-0.05
editor
0.79
0.77
-0.03
overlooking the clue given by the pronoun, and
(2) interpretability scores can be a valuable tool
veterinarian
0.17
0.56
0.39
teenager
0.17
0.56
0.39
detect such an unwanted behavior.
resident
0.17
0.56
0.39
Whether we match it or not, the model inflects the pronoun's referent profession in most cases. Poor GNT capabilities of instruction-tun els echo those recently found in commercial sys- tems (Piergentili et al., 2023b). Similar results hold for mTO (En-lis). In contrast, results on En- show a sensibly lower number of matching in- stances (39% for Flan-T5, 35% for mTO). Through manual inspection, we attribute mainly to using synonyms and wrong translations for Flan-T5 and failed translations for mTO. We report full results in Appendix C.3.
advisor
0.17
0.56
0.39
farmer
0.38
0.70
0.31
mod-
Table 7: Flan-T5 and Flan-T5 Ag performance across professions in En-Es and their difference (A).
strictly correct translation - e.g., "pasajero" (eng:
passenger) for "pedestrian" instead of "peatón"). Interestingly, in some rarer cases, the model re- places the target profession with an indirect object pronoun (c.g., "The supervisor gave [the employec] feedback on their stellar performance." -> "El su- pervisión [le] dio información sobre su trabajo ex- traordinario."" 133. While these cases are form of GNT, they come the expense of information loss. In the remaining matching 173 (72%) examples, the model used male inflection 54% of the times but only 9% feminine form, with the remain- ing cases not identified." 14 Some professions are exclusively inflected with feminine (e.g., nurse, re- ceptionist) and others masculine (e.g., buyer, chef, chemist) forms. Moreover, we found that the impor- tance given to the pronoun (they/them/their) when choosing masculine form for the profession (i.e.,
Related Work
Gender Bias in MT. As for other NLP tasks and models (e.g., Bolukbasi al., 2016; Zhao et al., 2017: Rudinger et al., 2018, inter alia), the study of gender bias (Sun et al., 2019) has received much attention in MT. For a thorough review on the topic, we refer to Savoldi et al. (2021).
Most prominently, Stanovsky et al. (2019) pre- sented the WinoMT data set for measuring occu- pational stereotypical gender bias in translations. Later, the data was extended by Troles and Schmid (2021) for covering gender stereotypical adjectives and verbs. Prates et al. (2020) analyzed gender bias Google Translate. Levy et al. (2021)
13 "Spanish "Ie" can translate to him/her. 14 As per the Win'M''s official heuristic and morphnlogical analysis tool.

---

focused on collecting natural data, while Gonen and Webster (2020) assessed gender issues in real- world input. As gender bias MT is highly re- lated to the typological features of the target lan- guages, several recent studies focused on specific language pairs, c.g., English/Korean (Cho et al., 2019), English/Hindi (Ramesh et al., 2021), En- glish/Turkish (Ciora et al., 2021), and English/Ital- ian (Vanmassenhove and Monti, 2021). Daems and Hackenbuchner (2022) introduced living community-driven collection of biased MT in- stances across many language pairs. Apart from data sets and methods for assessing gender bias in MT, researchers also proposed methods for bias mitigation. For instance, Escudé Font and Costa- jussà (2019) focused on embedding-based tech- niques (e.g., HardDebiasing), while Saunders et al. (2020) relied on gender inflection tags. Stafanovičs al. (2020) analyze the effect word-level annota- tions containing information about subject's gender. Saunders and Byme (2020) investigated the use of domain adaptation methods for bias mitigation, and Escolano et al. (2021) proposed to jointly learn the translation, the part-of-speech, and the gender of the target languages. Most recently, rescarchers have focused more on the role of gender-neutrality. As such, Lauscher et al. (2023) present study on (neo)pronouns commercial MT systems, and Picrgentili et al. (2023a) propose more inclusive MT through generating gender-neutral translations.
posed to tune models for following instructions in multi-task learning regimes achieving surpris- ingly good task generalizability (e.g., Sanh et al., 2022; Chung et al., 2022, inter alia). However, also prompt-hased models are prone to encode and amplify social biases. In this context, Lucy and Bamman (2021) showed that GPT-3 exhibits stereo- typical gender bias in story generation. Schick et al. (2021) proposed self-diagnosis and self-debiasing for language models which they test on T5 and GPT-2. (Akyürek et al., 2022) investigated whether the form of a prompt, independent of the content, influences the measurable bias. In contrast, Prab- humoye et al. (2021) use instruction- to detect social biases in given texts.
models
In this work, we are the first explore the of interpretability scores for informing bias mitiga- tion in instruction tuned models, bridging the gap between fairness and transparency in MT.
Conclusion
This paper introduced the first extensive study on the evaluation and mitigation of gender bias in ma- chine translation with instruction-tuned language models. Prominently, we studied the phenomenon through the lenses of interpretability and found that models systematically overlook lexical clues to inflect mender-marked words. Building this finding, we proposed simple and effective debi- asing solution based few-shot learning, where interpretability guides the selection of relevant ex- emplars.
Interpretability for MT. Previous work in context-aware MT (Voita et al., 2018; Kim et al.. 2019; Yin et al., 2021: Sarti et al., 2023a) stud- ied how models use (or do not use) context tokens by looking at internal components (e.g., attention heads, layer activations). More recent solutions enable the study of the impact of source and tar- get tokens (Ferrando et al., 2022) or discover the causes of ballucinations (Dale et al., 2023). Con- current work by Sarti et al. (2023b) uses post-hoc XAI methods to uncover gender bias in Turkish- English neural MT models. We expand their setup to more complex sentences and the notional-to- grammatical gender MT in two more languages.
Acknowledgments
This project has in part received funding from Fon- dazione Cariplo (grant No. 2020-4288, MONICA) and the European Research Council (ERC) under the European Union's Horizon 2020 research and innovation program (No. 949944, INTEGRATOR). GA, FP, and DN are member of the MilaNLP group and the Data and Marketing Insights Unit of the Bocconi Institute for Data Science and Analysis.
AL's work is funded under the Excellence Strategy of the German Federal Government and the Länder.
Mitigating Bias in Prompt-based Models. Given that sufficiently large LMs exhibit increas- ingly good few-shot and zero-shot abilities (Brown et al., 2020a), researchers investigated variants of prompting as a particular promising technique. Rel- evant to us, several works (Sanh et al., 2022) pro-
Limitations
Our work comes with a number of limitations.
We chose German and Spanish as the transla- tion target languages for our analysis. Our choice was motivated by (1) the presence of grammatical

---

gender in those languages, (2) their typological di- versity, and (3) our access to native speakers of those languages for double-checking the models' translations and translating our few-shot examples. We know that none of these languages is resource- scarce and that including more languages would strengthen our study. However, given the depth of our study and our qualitative analyses, we leave ex- panding our findings to more languages for future research.
Venkatesh Saligrama, and Adam T Kalai. 2016. Man is to computer programmer as woman is to home- maker? debiasing word embeddings. Advances in neural information processing systems, 29.
Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neclakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert- Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel Ziegler, Jeffrey Wu, Clemens Winter, Chris Hesse, Mark Chen, Eric Sigler, Ma- teusz. Litwin, Scott Gray, Benjamin Chess, Jack Clark. Christopher Berner, Sam McCandlish, Alec Radford, 1lya Sutskever. and Dario Amodei. 2020a. Language models are few-shot learners. In Ad- vances in Neural Information Processing Systems, volume 33, pages 1877-1901. Curran Associates, Inc.
This work focuses on the standard WinoMT benchmark. Yet. the dataset is constructed by slot-filling templates, simplifying the analysis on several aspects (e.g., there is one gender-marking pronoun, only two possible referents, and simple sentence structure). However, methodology will serve essential groundwork for extensions on natural MT setups (e.g., Bentivogli et al., 2020; Currey et al., 2022).
argue that our
Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neclakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020b. Language models are few-shot learners. Advances in neural information processing systems, 33: 1377-1111
Finally, we conduct the few-shot experiments only with the largest model variants. Including more model sizes here could lead to deeper insights. We chose to do so due to (1) the lousy translation quality we observed for smaller model sizes and (2) the computational effort associated with an en- vironmental impact. We believe that our findings will generalize other model sizes, providing a decent overall translation quality.
Wei-Lin Chiang, Zhuohan Li, Zi Lin, Ying Sheng, Zhanghao Wu, Hao 7hang, Lianmin Zheng, Siyuan Zhuang, Yonghao Zhuang, Joseph E. Gonzalez, Ion Stoica, and Eric P. Xing. 2023. Vicuna: An open- source chatbot impressing gpt-4 with 90%* chatgpt quality.
Won Ik Cho, Ji Won Kim, Seok Min Kim, and Nam Soo Kim. 2019. On measuring gender hias in translation of gender-neutral pronouns. In Proceedings of the First Workshop on Gender Bias in Natural Language Processing, pages 173-181, Florence, Italy. Associa- tion for Computational Linguistics.
References
Afra Feyza Akyürek. Sejin Paik, Muhammed Kocyigit, Seda Akbiyik, Serife Leman Runyun, and Derry Wi- jaya. 2022. On measuring social biases in prompt- based multi-task learning. In Findings of the Associ- ation for Computational Linguistics: NAACL 2022, pages 55 151466, Seattle, United States. Association for Computational I Linguistics.
Hyung Won Chung, Le Hou, Shayne Longpre, Barret Zoph, Yi Tay, William Fedus, Yunxuan Li, Xuezhi Wang, Mostafa Dehghani, Siddhartha Brahma, Al- bert Webson, Shixiang Shane Gu, Zhuyun Dai, Mirac Suzgun, Xinyun Chen, Aakanksha Chowdh- ery. Alex Castro- Castro-Ros, Marie Pellat, Kevin Robinson, Dasha Valter, Sharan Narang, Gaurav Mishra, Adams Yu, Vincent Zhao, Yanping Huang, Andrew Dai, Hongkun Yu, Slav Petrov, Ed H. Chi, Jeff Dean, Ja- cob Devlin, Adam Roberts, Denny Zhou, Quoe V. Le, and Jason Wei. 2022. Scaling instruction-finetuneed language models. arXiv preprint arXiv:2210.11441
Luisa Bentivogli. Beatrice Savoldi, Matteo Negri. Mat- tia A. Di Gangi, Roldano Cattoni, and Marco Turchi. 2020. Gender in danger? evaluating speech transla- tion technology on the MuST- corpus. In Pro- ceedings of the 58th Annual Meeting of the Asso- ciation for Computational Linguistics, pages 6923 6933, Online. Association for Computational Lin- guistics.
Chloe Ciora, Nur Iren, and Malihe Alikhani. 2021. Ex- amining covert gender bias: case study in Turkish and English machine translation models. In Proceed- ings of the 14th International Conference Natural Language Generation, pages 55-63, Aberdeen, Scot- land, UK. Association for Computational Linguistics.
Su Lin Blodgett, Solon Barocas, Hal Daumé III, and Hanna Wallach. 2020. Language (technology) is power: A critical survey of "bias" in NLP. In Pro- ceedings of the 58th Annual Meeting of the Asso-
ciation for Computational Linguistics, pages 5454- 5476, Online. Association for Computational Lin- guistics.
Anna Currey, Maria Nadejde, Raghavendra Reddy Pap- pagari, Mia Mayer, Stanislas Lauly, Xing Niu, Ben- jamin Hsu, and Georgiana Dinu. 2022. MT-GenEval: A counterfactual and contextual dataset for evaluating
Tolga Bolukbasi, Kai-Wei Chang, James Y Zou,

---

gender accuracy in machine translation. In Proceed- ings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 4287-4299, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics.
Hila Gonen and Kellie Webster. 2020. Automatically identifying gender issues in machine translation us- ing perturbations. In Findings of the Association for Computational Linguistics: EMNLP 2020, pages 1991-1995, Online. Association for Computational Linguisties.
Joke Daems and Janiça Hackenbuchner. 2022. DeBias- Raising awareness and creating a database of MT bias. In Proceedings of the 23rd Annual Confer- ence of the European Association for Machine Trans- lation, pages 289-290, Ghent, Belgium. European Association for Machine Translation.
ByUs:
Xiaochuang Han, Byron C. Wallace, and Yulia Tsvetkov. 2020. Explaining black box predictions and unveil- ing data artifacts through influence functions. In Proceedings of the 58th Annual Meeting of the Asso- ciation for Computational Linguistics, pages 5553- 5563, Online. Association for Computational Lin- guistics.
David Dale, Elena Voita, Loic Barrault, and Marta R. Costa-jussà, 2023. Detecting and mitigating halluci- nations in machine translation: Model internal work- ings alone do well, sentence similarity Even better. In Proceedings of the 6l Annual Meeting of the Association for Compulational Linguistics (Volume Long Papers), pages 36-50, Toronto, Canada. As- sociation for Computational Linguistics.
Dirk Hovy and Shannon Spruit. 2016. The social impact of natural language processing. In Proceed- ings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Pa- pers), pages 591-598, Berlin, Germany. Association for Computational Linguistics.
Marcin Junczys-Dowman Dowmunt, Roman Grundkiewicz. Tomasz Dwojak, Hieu Hoang, Kenneth Heafield, Tom Neckermann, Frank Seide, UIrich Germann. Alham Fikri Aji, Nikolay Bogoychev. André F. T. Martins, and Alexandra Birch. 2018. Marian: Fast neural machine translation in C++. In Proceedings of ACL 2018, System Demonstrations, pages 116 116-121, Melbourne, Australia. Association for Computational Linguistics.
Tim Dettmers, Mike Lewis, Younes Belkada, and Luke Zettlemoyer. 2022. Gpt3. int8 (): 8-bit matrix mul- tiplication for transformers at scale. Advances in Neural Information Processing Systems, 35:303 18- 30332.
Tim Dettmers, Artidoro Pagnoni, Ari Holtzman, and Luke Zettlmmyer. 2023. Qlora: Efficient finetuning of quantized Ilms. arXiv preprint arXiv: 14314.
Yunsu Kim, Duc Thanh Tran, and Hermann Ney. 2019. When and why document-eevel context useful in neural machine translation? In Proceedings of the Fourth Workshop on Discourse in Machine Trans- lation (DiscoMT 2019). pages 24-34, Hong Kong. China. Association for Computational Linguistics.
Chris Dyer, Victor Chahuneau, and Noah A. Smith. 2013. A simple, fast, and effective reparameteriza- tion of IBM model 2. In Proceedings of the 2013 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. pages 644-648, Atlanta, Georgia. Association for Computational Linguistics.
Philipp Koehn. 2005. Europarl: A parallel corpus for statistical machine translation. In Proceedings of Machine Translation Summit X: Papers, pages 79-86, Phuket, Thailand.
Carlos Escolano, Graciela Ojeda, Christine Basta, and Marta R. Costa-jussa. 2021. Multi-task learning for improving gender accuracy in neural machine translation. In Proceedings of the 18th Interna- tional Conference on Natural Language Processing (ICON), pages 12-17, National Institute of Technol- ogy Silchar, Silchar, India. NLP Association of India (NLPAD).
Alexandre Lacoste, Alexandra Luccioni, Victor Schmidt, and Thomas Dandres. 2019. Quantifying the carbon emissions of machine learning. arXiv preprint arXiv: 1xXx::9100099000
Anne Lauscher, Debora Nozza, Ehm Miltersen, Archie Crowley, and Dirk Hovy. 2023. What about "em"? how commercial machine translation fails to handle (neo-)pronouns. In Proceedings of the 61 Annual Meeting of the Association for Computational Lin- guistics (Volume 1: Long Papers). pages 377-392. Toronto, Canada. Association for Computational Lin- guistics.
Joel Escudé Font and Marta R. Costa-jussà. 2019. Equalizing gender bias in neural machine translation with word embeddings techniques. In Proceedings of the First Workshop on Gender Bias in Natural Lan- guage Processing, pages 147-154, Florence, Italy. Association for Computational Linguistics.
Javier Ferrando, Gerard I. Gállego. Belen Alastruey, Carlos Escolano, and Marta R. Costa-jussà. 2022. Towards opening the black box of neural machine translation: Source and target interpretations of the transformer. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Process- ing. pages 8756-8766 Abu Dhabi, United Arab Emi- rates. Association for Computational Linguistics.
Shahar Levy, Koren Lazar, and Gabriel Stanovsky. 2021. Collecting large-scale gender bias dataset for coref- erence resolution and machine translation. In Find- ings of the Association for Computational Linguis- tics: EMNLP 2021, pages 2470-2480, Punta Cana, Dominican Republic, Association for Computational Linguistics.

---

Shayne Longpre, Le Hou, Tu Vu, Albert Webson, Hyung Won Chung. Yi Tay, Denny Zhou, Quoe Le, Barret 7oph, Jason Wei, et al. 2023. The flan collection: Designing data and methods for effective instruction tuning. arXiv preprint urXiv2230113663
Andrea Piergentili, Beatrice Savoldi, Dennis Fucci, Mat- teo Negri, and Luisa Bentivogli. 2023b. Hi guys or hi folks? benchmarking gender-neutral machine translation with the gente corpus. arXiv preprint 1rX0v:2310.05294.
Li Lucy and David Bamman. 2021. Gender and rep- resentation bias in GPT-3 generated stories. In Pro- ceedings of the Third Workshop on Narrative Un- derstanding, pages 48-55, Virtual. Association for Computational Linguistics.
Shrimai Prabhumoye. Rafal Kocielnik, Mohammad Shoeybi, Anima Anandkumar, and Bryan Catan- zaro. 2021. Few-shot instruction prompts for pre- trained language models to detect social biases. arXiv preprint arXiv:21
Niklas Muennighoff, Nouamane Tazi, Loic Magne, and Nils Reimers. 2023a. MTEB: Massive text embed- ding benchmark. In Proceedings of the 17th Con- ference of the European Chapter the Association for Computational Linguistics, pages 2014-2037. Dubrovnik, Croatia. Association for Computational Linguistics.
Marcelo OR Prates, Pedro Avelar, and Luís C Lamb. 2020. Assessing gender bias in machine translation: case study with google translate. Neural Computing and Applications, 32:6363-6381.
Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang. Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu. 2020. Exploring the limits of transfer learning with a unified text-to-text trans- former. The Journal of Machine Learning Research, 21(1):5485-555 5551.
Niklas Muennighoff, Thomas Wang, Lintang Sutawika, Adam Roberts, Stella Biderman, Teven Le Scao, M Saiful Bari, Sheng Shen, Zheng-Xin Yong. Hai- ley Schoelkopf, Xiangru Tang, Dragomir Radev, Al- ham Fikri Aji, Khalid Almubarak, Samuel Albanie. Zaid Alyafeai, Albert Webson, Edward Raff, and Colin Raffel 2023b, Crosslingual generalization through multitask finetuning.
Krithika Ramesh, Gauri Gupta, and Sanjay Singh. 2021. Evaluating gender bias in Hindi Hindi-English machine translation. In Proceedings of the 3rd Workshop on Gender Bias in Natural Language Processing, pages 16-23, Online. Association for Computational Lin- guistics.
Moin Nadeem, Anna Bethke, and Siva Reddy. 2021. StereoSet: Measuring stereotypical bias pretrained language models. In Proceedings of the 59th Annual Meeting of the Association for Computational Lin- guistics and the 1 11th International Joint Conference on Natural Language Processing (Volume I: Long Pupers), pages 5356-5371, Online. Association for Computational Linguistics.
Ricardo Rei, José G. C. de Souza, Duarte Alves, Chrysoula Zerva, Ana C Farinha, Taisiya Glushkova, Alon Lavie, Luisa Coheur, and André F. T. Martins. 2022. COMET-22: Unbabel-IST 2022 submission for the metrics shared task. In Proceedings of the Seventh Conference on Machine Translation (WMT), pages 578-585, Abu Dhabi. United Arab Emirates (Hybrid). Association for Computational Linguistics.
Debora Nozza, Federico Bianchi, and Dirk Hovy. 2021. HONEST: Measuring hurtful sentence completion in language models. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 2398-2406, Online. Association for Computational Linguistics.
Ricardo Rei, Craig Stewart, Ana C Farinha, and Alon Lavie. 2020. Unbabel's participation in the WMT20 metrics shared task. In Proceedings of the Fifth Con- ference on Machine Translation, pages 91 1-920, On- line. Association for Computational Linguistics.
Rachel Rudinger, Jason Naradowsky, Brian Leonard. and Benjamin Van Durme. 2018. Gender bias in coreference resolution. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Short Papers), pages 8 14, New Orleans, Louisiana. Association for Computational Linguistics.
Long Ouyang, Jeffrey WU, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. 2022. Training language models to follow instruc- tions with human feedback. Advances in Neural Information Processing Systems, 35:27730-27747 27744..
Kishore Papineni. Salim Roukos, Todd Ward, and Wei- Jing Zhu. 2002. Bleu: method for automatic evalu- ation of machine translation. In Proceedings of the 40th Annual Meeting of the Association for Compu- tational Linguistics, pages 311-318, Philadelphia, Pennsylvania, USA. Association for Computational Linguistics.
Victor Sanh, Albert Webson, Colin Raffel, Stephen Bach, Lintang Sutawika, 7aid Alyafeai, Antoine Chaffin, Arnadd Stiegler, Arun Raja, Manan Dey. M Saiful Bari, Canwen Xu, Urmish Thakker, Shanya Sharma Sharma, Eliza Szezechla, Taewoon Kim. Gunjan Chhablani, Nihal Nayak, Debajyoti Datta, Jonathan Chang, Mike Tian-Jian Jiang, Han Wang, Matteo Manica, Sheng Shen, Zheng Xin Yong, Harshit Pandey, Rachel Bawden, Thomas Wang, Tr- ishala Neeraj, Jos Rozen, Abheesht Sharma, An- drea Santilli, Thibault Fevry, Jason Alan Fries, Ryan
Andrea Piergentili, Dennis Fucci, Beatrice Savoldi, Luisa Bentivogli, and Matteo Negri. 2023a. From inclusive language to gender-neutral machine transla- tion. arXiv preprint 2301 10075.

---

Techan, Teven Le Scao. Stella Biderman, Leo Gao, Thomas Wolf, and Alexander M Rush. 2022. Multi- task prompted training enables zero-shot task gener- alization. In International Conference on Learning Representations.
Gabriel Stanovsky, Noah Smith, and Luke Zettle- moyer. 2019. Evaluating gender bias in machine translation. In Proceedings of the 57th Annual Meet- ing of the Asvociation for Computational Linguistics, pages 1679-1684, Florence, Italy. Association for Computational Linguistics.
Gabriele Sarti, Grzegorz Chrupata, Malvina Nissim, and Arianna Bisazza. 2023a. Quantifying the plausibility of context reliance in neural machine translation.
Tony Sun, Andrew Gaut, Shirlyn Tang. Yuxin Huang, Mai EISherief, Jieyu Zhao, Diba Mirza, Elizabeth Belding, Kai-Wei Chang, and William Yang Wang. 2019. Mitigating gender bias in natural language processing: Literature review. Proceedings of the 57th Annual Meering of the Association for Computa- tional Linguistics, pages 1630-1640, Florence, Italy. Association for Computational Linguistics.
Gabriele Sarti, Nils Feldhus, Ludwig Sickert, and Os- kar van der Wal. 2023b. Inseq: An interpretability toolkit for sequence generation models. In Proceed- ings of the sl Anmual Meeting of the Association for Computational Linguisties (Volume 3: System Demonstrations), pages 421 -435, Toronto, Canada. Association for Computational Linguistics.
Mukund Sundararajan, Ankur Taly, and Qiqi Yan. 2017.
A Axiomatic attribution for deep networks. In Interna- rional conference on machine learning, pages 3319- 3328. PMLR.
Danielle Saunders and Bill Byrne 2020. Reducing gen- der bias in neural machine translation as domain adaptation problem. In Proceedings of the 58th An- nual Meeting of the Association for Computational Linguistics, pages 7724-7736, Online. Association for Computational Linguistics.
Jonas- Maris-Dario Troles and Ute Schmid. 2021. Extending challenge sets to uncover gender bias in machine translation: Impact of stereotypical verbs and adjec- tives. In Proceedings of the Sixth Conference on Machine Translation, pages 531-541, Online. Asso- ciation for Computational Linguistics.
Danielle Saunders, Rosie Sallis, and Bill Byrne. 2020. Neural machine translation doesn't 't translate gender coreference right unless you make it. In Proceedings of the Second Workshop on Gender Bias in Natural Language Processing, pages 35-43, Barcelona, Spain (Online). Association for Computational Linguistics.
Eva Vanmassenhove and Johanna Monti. 2021. gENder- IT: An annotated English-Italin Italian parallel challenge set for cross-lime pros-linguistic natural gender phenomena. In Proceedings of the 3rd Workshop on Gender Bias in Natural Language Processing, pages 1-7, Online, Association for Computational Linguisties.
Beatrice Savoldi, Marco Gaido, Luisa Bentivogli, Mat- teo Negri, and Marco Turchi. 2021. Gender bias in machine translation. Transactions of the Association for Computational Linguistics, 9:845-872 874.
Elena Voita, Pavel Serdyukov, Rico Semnrich, and Ivan Titov. 2018 Context-aware neural machine trans- lation learns anaphora resolution. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). pages 1264-1274, Melbourne, Australia. Association for Computational Linguistics.
Timo Schick. Sahana Udupa, and Hinrich Schütze. 2021. Self-diagnosis and self-debiasing: A proposal for re- ducing corpus-based bias in NLP. Transactions of the Association for Computational Linguistics, 9:1408- 1424.
Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue. Anthony Moi, Pier- ric Cistac, Tim Rault, Remi Louf, Morgan Funtow- ic7, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite. Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mariama Drame, Quentin Lhoest, and Alexander Rush. 2020. Trans- formers: State-of State-of-the-ant the-hhart natural language processing. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pages 38-45, Online. Association for Computational Linguistics.
Anders Søgaard, Anders Johannsen, Barbara Plank, Dirk Hovy, and Hector Martinez Alonso. 2014. What's in a p-value in NLP? In Proceedings of the Computational Natural Language Learning, pages 1-10, Ann Arbor, Michi- gan, Association for Computational Linguistics.
Eighteenth Conference
Aarohi Srivastava, Abhinav Rastogi, Abhishek Rao, Abu Awal Md Shoeb, Abubakar Abid, Adam Fisch, Adam R Brown, Adam Santoro, Aditya Gupta, Adríà Garriga-Alonso, et al. 2022. Beyond the imitation game: Quantifying and extrapolating the capabilities of language models. arXiv preprint urXiv:2206.066155
Linting Xue. Noah Constant, Adam Roberts, Mihir Kale, Rami Al-Rfou, Aditya Siddhant, Aditya Barua, and Colin Raffel. 2020. mt5: A massively multilingual pre-trained text-to-text transformer. arXiv preprint MxXiv:201..1994.
Artürs Stafanovičs, Toms Bergmanis, and Marcis Pinnis. 2020. Mitigating gender bias in machine translation with target gender annotations. In Proceedings of the Fifth Conference Machine Translation, pages 629-638, Online. Association for Computational Lin- guistics.
Kayo Yin, Patrick Fernandes, Danish Pruthi, Aditi Chaudhary, André T. Martins, and Graham Neu- big. 2021. Do context-aware translation models pay

---

the right attention? In Proceedings of the 59th An- nual Meeting of the Association for Computational Linguistics and the 11 1th International Joint Confer- ence on Natural Language Processing (Volume I: Long Pupers), pages 788-801, Online. Association for Computational Linguistics.
by imput embeddings (Han al., 2020) and in- tegrate over 16 steps from the baseline. We run model inference pass using 8-bit quantization for efficiency (Dettmers et al., 2022). We set aggrega- tion functions f to take the highest absolute value score across the aggregated span, preserving the sign (e.g., f(0001, - 0.3, 0.1]) = -0.3), and to the Euclidean norm. We aggregate first over f and then 9 because expect token-level per-unit scores to represent token attribution more expres- sively, and we do not want to lose such information with an initial pooling along the hidden size. We use Inseq (Sarti et al., 2023b) to compute and ag- gregate the scores.
Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q Weinberger, and Yoav Artzi. 2019. Bertscore: Eval- uating text generation with bert. arXiv preprint
arXiv:1904.06775.
Jieyu Zhao, Tianlu Wang. Mark Yatskar, Vicente Or- donez, and Kai-Wei Chang. 2017. Men also like shopping: Reducing gender bias amplification using corpus-level constraints. In Proceedings of the 2017 Conference on Empirical Methods Natural Lan- guage Processing, pages 2979-2989, Copenhagen, Denmark. Association for Computational Linguis- tics.
B.2 The WinoMT Corpus
WinoMT (Stanovsky et al., 2019) was created by combining the Winogender (Rudinger et al., 2018) and WinoBias (Zhao et al., 2018) coreference test sets. WinoMT consists of 3,888 instances, equally balanced between male and female genders and be- tween stereotypical and non-stereotypical gender- occupation assignments.
Jieyu Zhao, Tianlu Wang, Mark Yatskar, Vicente Or- donez, and Kai-Wei Chang. 2018. Gender bias in coreference resolution: Evaluation and debiasing methods. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Tech- nologies, Volume 2 (Short Papers), pages 15-20, New Orleans, Louisiana. Association for Computational Linguistics.
Instances in WinoMT are constructed from tem- plates where two professions interact in an arbi- trary activity (e.g., "The developer argued with the designer"), and a pronoun, either personal or possessive, coreferent of one of the two pro- fessions ("because she did not like the design"). Two hundred forty corpus instances re- sult from templates compiled using the pronouns they/their/them. They are intended to test gender- neutral translation capabilities.
A A Note on Model Nomenclature
The terms multitask prompted tuning and instruc- tion finetuning-ol or simply instruction tuning-refer to the same strategy of recasting existing NLP datasets using natural language and particular prompt-answer templates. Raffel et al. (2020) orig- inally introduced instruction tuning as comple- ment of pre-training of text-to-text models on variety of NLP tasks. More recently, instruction tuning has also been used to indicate the training process on human instructions, that lead to the rise modern assistant LMs (c.g., Ouyang et al., 2022; Chiang et al., 2023; Dettmers et al., 2023, inter alia).
refer
B.3 Translation with Baselines
We compare IFT models to two competitive baseline translation systems. We test Mar- ian NMT models fine-tuned on the OPUS cor- pus, matching the decoding strategy used for instruction models and no prompts. Moreover, we test GPT-3.5 using top p sampling (p=0.9, temperature=0.2, max_ max_tokens=256) and prompt "Translate the following sentence into {tgt_lang): (src_text)".
Although the two procedures have conflated, we reasonably see recasted dataset (e.g., FLAN (Chung et al., 2022) or xP3) as instruction follow- ing data. Therefore, in this paper, we adopt the instruction finetuning nomenclature as a reasonble umbrella defimition.
C Additional Results
B Details on the Experimental Setup
C.1 Gender Bias Evaluation
B.1 Interpretability
Table reports full results on the gender bias eval- uation of IFT models and several baselines. For Flan T5, generally accuracy and fairness improve
We use Integrated Gradient (IG) token-level attribu- tion scores (Sundararajan et al., 2017). Following recent work on IG for NLP, we multiply raw scores
hhttps://opus /https://opuu.nipl.e nlw...unpl.u./

---

Translate (sre_text_few_shee { tgt_lang (Bt__tett.ew_shott/a
XXL and m'T0XXLL. We observe several interest- ing findings. (1) Both models tend to inflect into masculine or feminine forms more frequently when translating into Spanish (avg: 75%) than German (avg: 37%). (2) Frequent gender inflections de- spite neutral pronoun constructions underscore models' poor capabilities in this task, as they rarely resort to gender-neutral translations. (3) When in- flecting the gender, both models use masculine sub- stantially more often than feminine, showing a per- sistent gender bias even in GNT. (4) apron.prof for masculine translations lower than that for femi- nine ones across both languages and models. This finding supports the intuition that models "over- look" overt contextual clues to inflect gender and default to male forms.
Figure 2: Few-shot debiasing prompt, individ- ual exemplar template. src_text_few few_ss____w_hht and @gt_text_few_shot are human-translated parallel WinoMT instances.
with size. Flan-T5-Small is an exception, being the best model for As in En-Es. However, low accu- racy might induce a very low As (Saunders and Byrne, 2020). Indeed, the same model is not able to produce meaningful results in En-De. Smaller mTO models share this trait of low accuracy and unstable translations in As.
Since En-De has sensibly lower number of matching instance, we manually inspected them looking for the causes. Table reports full statis- tics. Flan-T5 omitted the target phrase once; mTO did the same twice and used a gender neutral plu- ral once. Notably, while Flan-T5 mainly uses synonyms or translates profession words wrongly, mTO fails many translations by verbatim copying the input sentence the output.
C.2 Impact of Decoding and Prompt Template
We conducted an extensive hyperparameter search on five decoding strategies and two prompt templates. We conducted all tests the Europarl WMT' 06 sets. We experimented with greedy search, beam search (n=4, no sampling), top sampling (k = [5, 10, 20, 50, 80]), top sampling (p= [0.4, 0.6, 0.8, 0.9, 0.95, 1.0] and temperature = =0.4, 0.7, 1J), and contrastive decoding (k = (2, 5, 101, and penalty alpha 0.6). As for prompt tem- plates, we used two from the FLAN collection, i.e.,
D Details the Debiasing
"(src_text) Translate this to (tgt_lang)?" and "Translate from (src_lang) to (tgt_lang): hnw.rsssr....eeetnnnnntteeeeeeee
Table 12 reports the full list of results with dif- ferent strategies for few-shot prompting. In three cases out of four (En-Es Flan, En-De Flan, and En-Es mTT),, the proposed solution for few-shot examples selection leads to better accuracy with at least one variant of the non-target profession. In En-Es, for both Flan-T5 and mTO, the best ver- sion is Few-Shot NT-Female (i.e., providing exam- ples where the non-target profession is feminine- inflected).
We ran an exhaustive grid search on En-Es, identify the best setup, and translate En-De with the best setup found.
In line with prior literature, we found that beam search (n=4, sampling) is the best decoding strat- egy across all languages, models, and model sizes, Moreover, Figure 3 and show clear increasing trend in performance as models get bigger, with mTO being slightly better than Flan-T5 on aver- age. Notably, only the largest models achieve pos- itive quality estimation scores (COMET20), even though non comparable to those of supervised mod- els. Table 9 reports all results on both En-Lis and En-De for all tested models.
We do not report the results for mTO En-De as the model is not able to handle any few-shot tem- plate in this setup. The resulting generation are almost always in language different than the tar- get they are empty.
C,3 Gender-Neutral Translation
D.1 Few-Shot Prompt
Table 10 reports complete overview of the results on the 240 gender-neutral instances for Flan-T5-
We reuse FLAN few-shot template for our few- shot debiasing approach. Specifically, we concate- nate N=4 times the exemplar template reported Figure 2.
htattp:://githuu.cmmm// htt.uuu..cooggoooglooo wtw./ggooleeessrrrr/////////bbb main//Flann/22/temllaes...

---

Spanish
German
Model
Acc
ÅG
As
Acc
AG
A
Google Translate*
53.1
23.4
21.3
59,4
12.5
12.5
Microsoft Translator'
47.3
36.8
23.2
74.1
0.0
30.2
Amazon Translate"
59.4
15.4
22.3
62.4
12.0
16.7
Marian NMT
56.8
16.9
19.7
62.0
9.9
15.2
GPT-3.5
55.2
23.1
48.5
48.3
25.2
24.6
44.4
38.5
8.7
Flan-T5-Base
47.7
31.5
22.8
39.5
-17.8
-18.7
L--age
51.7
17.1
44.7
52.3
-4.9
-5.9
F5-n-T5-XI.
58.5
16.0
40.3
68.0
2.39
14.3
Flan-T5-XXL
65.6
6.7
34.3
67.5
2.3
112
mTO-Small
40.6
17.9
13.4
46.9
59.0
66
m'T-Base
54.2
12.2
27.4
47.0
59.0
m'TO-Large
61.5
5.8
26.7
46.9
59.0
m'TO-LL
51.0
29.6
51.6
47.6
54.3
8.2
mTO-XXL
51.9
29.7
33.8
56.0
24.2
21.1
Table Gender bias evaluation WinoMT all tested models. Instruction-tur -uned models with beam search (n=4, no sampling) decoding. Best models bold. 'Results reported from Stanovsky et al. (2019). "*Results not reported due unstable translations (i.e., mostly empty or the target language).
Maria NMT
0.7
Flan-TS-Small
Flan-Ts Base
Flen-TS-Large
COMET22
0.5
Flan-Ts XL
Flan-TS-XOL.
0.4
Small
mTO-8ase
mTO Large
0.2
MTD XXL
Beam Search
Greedy Search
Contrastive
Figure 3: Flan-T5 T5 and mTO zero-shot Translation performance (reference-based COMET-22) for different decoding and model size on the Europarl WMT" 06 En-Ei Es test set. Marian NMT supervised model for comparison.
0.2
COMET20
0.0
Beam Search
-0.4
Greedy Search
Top
-0.6
-0.8
Contrastive
Marian NMT
Flan-TS-Small
Flan-T5-Bess
Fan-Ts-Large
FlanTS-X
Flam-TS-oe
mTD ITTO-Small Smalll
mT0-Bese
iTO-Large
MT00XL
MITO XOC.
Figure 4: Flan-T5 and mTO zero-shot Translation performance (reference-free COMET-20) for different decoding and model sizes on the Europarl WMT 06 En-Es test set. Marian NMT supervised model for comparison.

---

Model
COMET-22
COMET-20
BERTScore
BLEU-2
BLEU-4
Marian NMT
0.85
0.42
0.89
0.55
0.39
Flan-TS-Small
0.58
-0.48
0,81
0.31
0.16
Han-T5-BBas
0.70
-0.10
0.84
0.39
0.23
Flan-T5-Large
0.76
0.10
0.85
0.44
0.27
-lan-T5-XB
0.78
0.16
0.86
0.45
0.29
FI-X-T
0.79
0.18
0.86
0.45
0.29
mTO-Small
0.51
-0.53
0.73
0.14
0.05
nTT--Base
0.70
-0.02
0.81
0.29
0.15
m'To-Large
0.75
0.15
0.82
0.32
0.18
mTO-XL
0.78
0.18
0.81
0.34
0.21
mTO-XXL
0.80
0.27
0.82
0.42
0.27
Marian NMT
0.80
0.43
0.85
0.42
0.26
-ln--T--Small
0.52
-0.51
0.76
0.18
0.07
Tlan-T5-Base
0.63
-0.12
0.80
0.26
0.12
LS-Laee
0,70
0.15
0.82
0.32
0.17
F5an-T5-XL.
0.73
0.26
0.82
0.32
0.17
Flan-T5-XXI.
0.75
0.31
0.82
0.82
0.18
mTO-Small
0.44
0.02
0.70
0.01
0.00
mT0-Base
0.54
0.39
0.75
0.04
0.01
mTO-Large
0.53
0.33
0.74
0.03
0.01
mTO-XL
0.60
0.32
0.77
0.10
0.04
mTO-XXL
0.75
0.42
0.81
0.28
0.15
Table 9: Zero-shot performance for the best configuration of decoding and prompt template on Europarl WMT 06 En-Es (top) and En-De (bottom) test sets. Bold and italic indicate best instruction-tunee and overall models per target language, respectively. Supervised Marian NMT baselines (top rows) for comparison.
Spanish
German
Translated Gender
Matching (%)
apron.prof
Matching (%)
apron.prof
Female
0.1588
11
0.1765
Male
39
0.0892
27
0.
Neutral/Unkkown
26
0.1510
0.2134
Non-Matching
28
61
Female
0.3565
0.2996
Male
50
0.2008
30
0.2021
Neutral/Unknown
23
0.1863
0.3722
Non-Matching
22
65
Table 10: Statistics on gender-neutral cases WinoMT for Flan-T5 TS-XXL (top) and mTO-XXL (bottom). Number of matching occurrences of female- male- inflected pronoun referents, and median apron.
D.2 Statistical Significance
et al., 2014, n=1000, sample size-30%). Inter- pretability informed sampling is better than random sampling in terms of macro Fl score (p < .01, in En-Es and En-De), and accuracy (p S .01 in En-Es, .05 in En-De).
We compute statistical significance of the differ- ence performance between few-shot with ran- dom sampling of examples and choosing using Oprofprof- We use bootstrap sampling (Søgaard

---

Reason
Flan-T5
mTO
Synonym (malc)
34.3
23.4
Wrong Translation
21
0
Grammatical inflection (male)
18.2
16.2
Synonym (female)
10.5
2.6
Grammatical inflection (neutral)
8.4
16.2
Synonym (neutral)
4.9
3.2
English Translation
2.1
36.4
Phrase omitted
0.7
1.3
Gender Neutral Plural
0.6
Table 11: Frequency (%) each cause in non-maten non-matching instances in gender-neutral translation for En-De.
E
Carbon Footprint
Experiments were conducted using private in- frastructure, which has carbon efficiency of 0.29 kgCO2eq/kWh. A cumulative of 184 hours (53 for translation, 14 for evaluation, 117 for generat- ing feature attribution scores) of computation was performed on hardware of type A100 PCIe 80GB (TDP of 250W). Total emissions are estimated to be 13.34 kgCOzeq, none of which were directly offset. Estimations were conducted using the Machine- Learning Impact calculator presented in (Lacoste et al., 2019).
F Release of Data Artifacts
We release code to reproduce our exper- iments at https: wtwp:tthub..mmmmMiaaPPPP... interpretability-ntt gender-bias. -tntterr-yy over, plan to release all data artifacts produced in study hoping foster future research in the field, including (1) integrated gradient scores, (2) human-refined GPT-3.5 translation of WinoMT professions, and (3) human-translane translated seed demon- strations for few-shot learning. We will release any additional content not listed here in the paper repository.
More-

---

Spanish AG
German
Model
Demonstration Sampling
Acc
As
Acc
AG
As
Flan-T5-XXL
65.6
6.7
34.3
67.5
2.3
-1.2
Flan-T5-2 T--X-555-X---X--
Random
68.9
4.10
22.2
67.4
5.5
24.4
Flan-T5 T5-----XX--. TT5-XXT-Few-She
Shot
NT-Female
72.2
2.1
19.6
65.1
-6.1
-8.2
Flan-T's T5-XXLFew-Sher
NT-Male
70.3
3.4
24.1
69.8
2.3
-10.4
Flan-T5 T5-XXI-Few-Shot
NT-Random
68.7
4.4
19.9
64.5
-6.8
-7.4
mTO-XXL
51.9
29.7
33.8
56.0
24.2
21.1
TTO-XXLFew-Shee
Random
46.6
25.1
11.7
MTT-XXXLee-SShor
NT-Female
60.8
9.9
21.1
m TO-XXL XXL=Few-hot
NT-Male
58.2
16.0
31.0
6.
*+
mT0-XXI FeexXLFee-SS0 SeotShot
NT-Random
60.6
11.9
26.5
**
*+
Table Gender bias evaluation on WinoMT different strategies of human- wwrtttn demonstration sampling. Zhot variants in top rows. Best results in bold. ""Results not reported due unstable translations (i.e., mostly empty or not in the target language).


Reviews:
[{"review_id": "Ud9tBMun5l", "reviewer": "Reviewer_6GEX", "summary": "", "questions": "A. Results from Table 10 in Appendix indicate that the inflective form of NT (profession whose gender is unknown) affects performance considerably. While all the NT variants for interpretability-guided debiasing is reported and the best variant is included in the main paper, it is unclear whether all NT variants are considered for the baseline method of random sampling. Which version is used?\n\nB. In L458, the baseline model having lower $\\Delta_S$ of -1.2 compared to proposed method's -10.4 is explained that it might be artificially lower due to its lower accuracy. However, I'm wondering whether this is possible since  $\\Delta_S$ represents the accuracy difference between stereotypical and anti-stereotypical samples, and while the accuracy difference between baseline and proposed is 69.8-67.5=2.3, $\\Delta_S$ difference is -10.4-(-1.2)=-9.2, and thus much bigger. ", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your helpful feedback. Let us address the points you have raised as Weaknesses (W) and reply to your Questions (Q).\n\n**(W) Generalization to Different Styles.** Our paper lays the foundation for linking interpretability and occupational gender bias on recent instruction fine-tuned models, proposing a mitigation strategy. We used WinoMT because it is an established benchmark for studying this issue, and it also provides several meaningful markers (e.g., the source pronoun and target profession) that helped us perform the analysis. Generalization to different domains and styles would require labeling these markers on new datasets or finding different ones, which is beyond the scope of this paper. We agree that exploring more complex setups, e.g., natural sentences like MT-GenEval (Currey et al., 2022), is an exciting research we will explore.\n\n**(W) Bias Reduction vs. Translation Performance.** Table 5 reports improved performance on correct gender inflection (Accuracy column), which we can loosely relate to an on-par or better overall translation quality after debiasing. We cannot measure translation quality automatically as WinoMT does not provide gold translations. Moreover, our solution uses in-context learning and cannot quickly adapt to out-of-domain data, as in Saunders and Bryne, 2020.\n\n**(Q-A)** For the random baseline, we randomly sample a female- or male-inflected Non-Target target occupation (i.e., equivalent to NT-Random in Table 10), keeping the stratification over target gender and stereotype type. Thank you for catching this missing detail; we will include it in the main body. \n\n**(Q-B)** As reported by Saunders and Byrne, 2020, Delta_S can be heavily skewed by low accuracy: imagine a system using the male form for each sample. Since WinoMT has the same number of stereotypical and anti-stereotypical instances, Delta_S will be 0 (best value) as accuracy over the two sets will be identical. We hence believe Flan-T5 is outscoring the debiased variant on Delta_S because of that.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "Strengths:\n- this paper investigates the well-defined MT gender bias problem for the instruction-tuned models and few-shot prompting, which is being rapidly adopted and used in recent NLP works. As the authors state, the gender bias problem of these few-shot setup is relatively not well known. \n- Their proposed method of using integrated gradients based attribution scores to  further analyze gender bias problems provides interesting insights. ", "reasons_to_reject": "Weaknesses:\n- The proposed bias mitigation method makes use of sampling from WinoMT dataset to use as few-shot examples, and then proceeds to use the same set for gender bias evaluation (Table 5). This raises a question of whether the proposed method is especially effective for only the WinoMT dataset, or is equally effective and generalizable to other sentences of different styles. \n- Previous works on gender debiasing note a trade-off between translation quality and bias reduction [1]. While the gender debiasing effect of the proposed method has been measured, its effect on translation performance is unclear. \n\n[1]  Saunders and Bryne, Reducing Gender Bias in Neural Machine Translation as a Domain Adaptation Problem, ACL 2020.", "questions_to_authors": "A. Results from Table 10 in Appendix indicate that the inflective form of NT (profession whose gender is unknown) affects performance considerably. While all the NT variants for interpretability-guided debiasing is reported and the best variant is included in the main paper, it is unclear whether all NT variants are considered for the baseline method of random sampling. Which version is used?\n\nB. In L458, the baseline model having lower $\\Delta_S$ of -1.2 compared to proposed method's -10.4 is explained that it might be artificially lower due to its lower accuracy. However, I'm wondering whether this is possible since  $\\Delta_S$ represents the accuracy difference between stereotypical and anti-stereotypical samples, and while the accuracy difference between baseline and proposed is 69.8-67.5=2.3, $\\Delta_S$ difference is -10.4-(-1.2)=-9.2, and thus much bigger. ", "paper_topic_and_main_contributions": "This paper investigates the gender bias issue of instruction-finetuned models in machine translation task. They use a standard WinoMT benchmark to evaluate gender bias of two instruction-finetuned models, Flan-T5 and mT0, in zero/few-shot settings. They also provide further analysis and bias mitigation methods based on existing interpretability methods. The main contributions are as follows:\n- while many work cover gender bias on supervised MT models, this work specifically focuses on instruction-tuned models and its susceptibility to biases. \n- it uses integrated gradients (IG) interpretability-based method to analyze how model-internal effects lead to gender bias.\n- it proposes a bias mitigation method based on few-shot prompting using the interpretability scores.", "excitement": 4, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "l7leLuxHW1", "reviewer": "Reviewer_eP6E", "summary": "", "questions": "The paper states early in section 1 that gender bias in pretrained LMs puts \"their societal beneficial use at risk\" - this is not directly related to the paper's approach, but it was an unusual phrase and I wondered what was meant by it?", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your insightful comments and for recognizing the novelty of the approach and the potential insights it can provide. We appreciate you found the paper well-written, and we will further revise the writing in the final draft following your suggestions. Let us address the points you have raised as Weaknesses (W) and reply to your Questions (Q).\n\n**(W) Inquiries in Gender-Neutral Translations.** We believe observing the models' behavior in gender-neutral cases is indeed of extreme interest, and it would require a quite careful and extensive analysis from the interpretability angle (e.g., different forms of gender-neutral translations might exist when translating to a grammatical gender language, and each of them might require different observations and considerations). While this may require a specific research effort, we will provide in the paper some preliminary considerations based on the interpretability scores on the 240 gender-neutral examples in WinoMT.\n \nConsidering Flan-T5-XXL and En-Es, we found the target profession in 173 (72%) examples. Among these, the model used a male inflection 54% of the time but only 9% a feminine form, with the remaining cases not identified. Some professions are exclusively inflected with feminine (nurse, receptionist, victim) and others masculine (buyer, chef, chemist). Moreover, we found that the importance given to the pronoun (they/them/their) when choosing a masculine form for the profession (i.e., a_{prof,pron}) is lower (median: 0.9) than in feminine cases (median: 0.16).\nThese findings corroborate those in our paper because 1) Flan-T5 uses the masculine form by default, overlooking the clue given by the pronoun, and 2) interpretability can be a valuable tool to detect such an unwanted model behavior.\n\n**(W) Overlap between Professions in the Prompt and Examples.** Our proposed debiasing uses the same examples (i.e., in-context “shots”) for translating every sample. By doing so, the average overlap across the 3,888 WinoMT set is close to none (very few examples out of the total will contain one of the professions mentioned in the four examples we sample to build the debiasing prompt).\nMoreover, we thought about using different shots for every input; however, that would entail manually translating more examples (in the worst case, one-fourth of the dataset), whereas we show that we can reduce bias and improve performance with as little as translating 4.\n\n**(Q) Societal Beneficial Use at Risk.** Instruction Fine-tuned models are promising tools for many language tasks; hence, they can have a highly beneficial impact on society. However, once a model learns societal biases and unfair stereotypes, it risks perpetrating them at scale. Failing to evaluate and mitigate such biases puts their use at risk because it will be unsafe to deploy and let people use them.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "- Taking an explainability approach towards the gender translation problem is novel and interesting, as well as having the potential to guide future work on the problem by making it clearer what the problem is.\n\n- Section 6 describes an interesting experiment. It seems from the discussion that there was some overlap between the professions in the few-shot-example-list, and the professions to translate in WinoMT. If so, it would be interesting to know how much of an impact that overlap had, perhaps by choosing a non-profession-overlapping list for each input. But regardless the experiment is interesting to read.\n\n- The paper is for the most part clearly written (5.2 a little hard to follow), and the results clearly laid out and understandable. Great to see both numerical and qualitative analysis also.\n\n- There was plenty of detail given for reproducibility purposes, and especially good to see scripts attached to reproduce the work", "reasons_to_reject": "- The paper claims that there are insufficient resources and data for inquiries on non-binary and gender-neutral MT. It makes use of WinoMT (2019) - an add-on to WinoMT, including an adaptation dataset, to extend to non-binary translation from English into Spanish and German was made available in 2020 (Saunders, Sallis & Byrne, which is already cited by this paper). The paper would be stronger if it made use of the resources, or offered an explanation for why the existing resources are unsuitable. An alternative would be to simply try translating such sentences with gender-neutral pronouns in the source, and qualitatively examine even a few of the results, similar to Table 6. \n\nEdit to add: The authors have responded with some interesting preliminary notes on the neutral-pronoun sentences in WinoMT. Some analysis along these lines would be a great addition to the paper if accepted.", "questions_to_authors": "The paper states early in section 1 that gender bias in pretrained LMs puts \"their societal beneficial use at risk\" - this is not directly related to the paper's approach, but it was an unusual phrase and I wondered what was meant by it?", "paper_topic_and_main_contributions": "The paper addresses gender bias in pretrained language models from the standpoint of interpretability. It explores the source of gender-based mistakes, and uses the findings to motivate a prompt-based mitigation method.", "excitement": 4, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {"Typos_Grammar_Style_and_Presentation_Improvements": "- The experiments in 5.2, while interesting, were a little difficult for me to follow. Perhaps it would be clearer if it were broken up a little, say into self-contained sections for the meaning and findings of each of a_prof,prof, a_pron,prof, etc."}}, {"review_id": "6pmwoEowdV", "reviewer": "Reviewer_HKEt", "summary": "", "questions": "A. The example of Figure 1 showcases an ambiguous coreference that humans can resolve employing semantic grounded information (i.e. it is usually the person that wins money that buys gifts for others, and not vice-versa). While in the example the feminine gender is missing regardless of the choice of coreferent, do you think that even a debiased MT model would be able to perform the coreference correctly, in the absence of further information that we usually may give for granted? This could be potentially related to the large accuracy gap between NT-Female and NT-Random results in Table 10.\n\nB. Why is aggregation of raw attribution scores performed as $A = g(f(A_r))$ instead of $A = f(g(A_r))$ i.e. why is subword aggregation performed before aggregating per-unit scores at a token level? I suspect the reason to be the expressivity of the resulting vectors, but if this is the case it would need to be clearly mentioned at least in a footnote/appendix.\n\nC. If the Euclidean norm is used to obtain per-token attribution scores, these are strictly positive. In light of this, are scores normalized across input words, such that $\\sum_{s \\in S} a_s = 1,  \\forall a_s \\in A_s = \\mathbb{R}^S$? If this is the case, $a$ values from Table 3 can be interpreted as the relative % of attention to $ctrl, prof$ and $pron$ respectively when generating the target occupation term, which would be preferable to simplify understanding of the results.\n\nD. Were more modern alignment approaches, such as AWESOME (https://github.com/neulab/awesome-align), tested alongside FastAlign? The improved quality of those might have saved quite some effort with lexical translation and matching.\n\nE. You mention that the instruction-tuned models were provided with MT tuning examples in xP3 and FLAN. It is not clear to me whether the same template adopted by these datasets was used, given that it would be the most natural choice to ensure in-domain prompts.If this is not the case, could you justify your choice of using another MT prompt, possibly harming model performance?", "limitations": "", "rating": 4, "confidence": 4, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your detailed feedback and for recognizing our work's novelty. Let us address your comments and Questions.\n\n**Real-World Adoption of Instruction Fine-tuned LMs.** We agree that supervised (smaller) LMs still outperform larger Instruction Fine-tuned (IFT) LMs. However, studying biases and applications of the latter is highly timely for two main reasons. First, pressing evidence shows that IFT LMs are narrowing the gap against task-specific supervised models (e.g., Flan-T5, Chung et al., 2022; GPT3.5, and GPT4, Ouyang et al., 2022; PaLM, Chowdhery et al., 2022) in many NLP tasks, mainly due to larger scale or training sources (for a reference, see Table 9 and Figure 2, where translation quality improves as models get larger). Hence, we believe IFT models will match or surpass task-supervised MT LMs. Second, IFT models will become widespread (if not already) as they are more accessible with easy-to-use UI (e.g., ChatGPT).\nWe will include and deepen this discussion in the main body upon acceptance.\n\n**(Q-A)** We agree with your point, which is of extreme interest. Indeed, the WinoMT suite exposes models to this ambiguity with most samples. Take, for instance, “The mover outran the hairdresser because she was in shape.” Here, common sense would help us relate the action (outrunning) to the explanation (being in good shape) to identify what is the antecedent of “she” (all of this even before considering any stereotypical association on roles). As the WinoMT examples do not let us disentangle coreference capabilities from stereotypical resolution, it is hard to tell what debiasing models look at (e.g., are NT-Female few shots helpful because the model has at least one feminine form in the sentence?). We believe this is an exciting research direction we will explore in future work.           \n  \n**(Q-B)** That is correct; we expect token-level per-unit scores to represent token attribution more expressively, and we do not want to lose that information with an initial hidden pooling. We will discuss this further in the paper.  \n  \n**(Q-C)** We do not normalize scores across input words. We thought about that and decided not to pursue this way because non-normalized scores do not depend on the input length and can be compared across samples. \n\n**(Q-D)** We did not test other alignment methods, but thank you for pointing this out. We will consider AWESOME for future work.\n\n**(Q-E)** We used unmodified templates from the FLAN collection. This detail is reported in Appendix D.1.\n\n**Missing References.** Thank you for pointing out additional related work. We will further discuss them in the main body upon acceptance.\n\n**Comments on Writing and Clarifications.** Thank you for the detailed feedback. We will clarify each point in the revised paper.\n- 35-36, 182-185, 196-197, Figure 1, Appendix A, Table 9: We agree with the suggested changes and will implement them.\n- Appendix B: We will discuss that given the chosen f and g, we have token attribution scores that are strictly positive.\n- 231: We manually checked for grammatical errors and fixed them if present. We will clarify it in the paper.\n- 288-289: Yes, they correspond to the XXL variants; we will specify it in Table 1.\n- 294-298: We will fix the typo and clarify that the COMET-20 automatic metric is a reference-free neural quality estimation model whose output reflects human-generated quality scores used for training. \n- 351-357: In 3 out of 4 cases (datasets+models), the most accurate class is associated with the highest a_prof,prof values. However, as you noticed, there is no clear correlation with all the accuracy and a_prof,prof values. We will re-write the sentence to clarify it better.\n- 396-403: The findings were indeed obtained by looking at the percentage difference and checked with a correlation test; we will provide values to support the sentence.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "The paper is well-written and showcases an interesting and timely application of post-hoc interpretability methods to support the analysis and reduction of gender bias in established generative language models. The authors provide an extensive empirical evaluation of several large-scale language models and generation settings while also analyzing more granular failure cases of these models from a qualitative standpoint. To my knowledge, this paper is one of the first to show a direct application of interpretability techniques that could mitigate representational biases in deployed MT systems. Moreover, the proposed method to reduce gender bias is simple but effective and suggests the potential of leveraging such techniques for debiasing other natural language generation tasks.", "reasons_to_reject": "While results are generally promising, some parts of the paper contain strong statements/conclusions but sometimes lack further clarifications or explicit supporting evidence. I refer to these more explicitly in the \"Typos Grammar Style And Presentation Improvements\" section. Importantly, these are limited to some sections and in my opinion do not compromise the overall validity and quality of the work.\n\nMore generally, I am not quite convinced by the authors' assumption that results of Section 4 \"motivate their [instruction-tuned LMs] adoption in real use cases\", given the best instruction-tuned model with 11B parameters, (requiring significant resources for inference) cannot perform on par with a small (60M?) Marian NMT model trained for the task. In scenarios where the usage of these models is constrained to translation, the choice of the latter would be preferable. In light of this, the benefits of less-performant multi-purpose systems like the ones employed in this study should be discussed more explicitly to convince the reader of their relevance.", "questions_to_authors": "A. The example of Figure 1 showcases an ambiguous coreference that humans can resolve employing semantic grounded information (i.e. it is usually the person that wins money that buys gifts for others, and not vice-versa). While in the example the feminine gender is missing regardless of the choice of coreferent, do you think that even a debiased MT model would be able to perform the coreference correctly, in the absence of further information that we usually may give for granted? This could be potentially related to the large accuracy gap between NT-Female and NT-Random results in Table 10.\n\nB. Why is aggregation of raw attribution scores performed as $A = g(f(A_r))$ instead of $A = f(g(A_r))$ i.e. why is subword aggregation performed before aggregating per-unit scores at a token level? I suspect the reason to be the expressivity of the resulting vectors, but if this is the case it would need to be clearly mentioned at least in a footnote/appendix.\n\nC. If the Euclidean norm is used to obtain per-token attribution scores, these are strictly positive. In light of this, are scores normalized across input words, such that $\\sum_{s \\in S} a_s = 1,  \\forall a_s \\in A_s = \\mathbb{R}^S$? If this is the case, $a$ values from Table 3 can be interpreted as the relative % of attention to $ctrl, prof$ and $pron$ respectively when generating the target occupation term, which would be preferable to simplify understanding of the results.\n\nD. Were more modern alignment approaches, such as AWESOME (https://github.com/neulab/awesome-align), tested alongside FastAlign? The improved quality of those might have saved quite some effort with lexical translation and matching.\n\nE. You mention that the instruction-tuned models were provided with MT tuning examples in xP3 and FLAN. It is not clear to me whether the same template adopted by these datasets was used, given that it would be the most natural choice to ensure in-domain prompts.If this is not the case, could you justify your choice of using another MT prompt, possibly harming model performance?", "paper_topic_and_main_contributions": "The paper aims to leverage established post-hoc interpretability techniques to study and debias gender bias in instruction-tuned models for the machine translation task. Through several experiments on WinoMT En -> De and En -> Es, authors showcase the gender bias in translations produced by recent instruction-tuned multilingual models. By leveraging importance scores obtained during the generation of gendered occupation terms, authors highlight the propensity of these models to overlook the gendered pronouns in the source. Finally, a sampling procedure to select in-context examples from the importance given to pronouns is introduced, showing significant improvements in debiasing translations compared to baseline results.", "excitement": 4, "reproducibility": 5, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {"Missing_References": "- Gender Bias in MT + Interpretability: The demo paper associated to the Inseq library (https://aclanthology.org/2023.acl-demo.40/) you employed contains a case study on gender bias in machine translation that leverages the same techniques you used, and which should be cited as previous/concurrent work showing how feature attribution methods can be used to identify gender bias in machine translation settings.\n- Gender Bias in MT + Interpretability: Previous work in context-aware MT (https://aclanthology.org/2021.acl-long.65.pdf#cite.voita-etal-2018-context, https://aclanthology.org/2021.acl-long.65/, https://aclanthology.org/D19-6503/) focused on the translation of texts containing inter-sentential gendered pronoun coreference, finding that the cue gendered word in the context is not attended adequately during translation. This finding seems very related to the overlooking of gendered pronouns shown in this work.\n- Other MT Bias + Interpretability: Recent work use similar feature attribution techniques on MT models to identify toxic generations (https://arxiv.org/abs/2210.03070) and hallucinations (https://aclanthology.org/2023.acl-long.3/). While not strictly related to gender bias, these seem relevant to mention given the match in area of application and methods used.\n- Mitigating Bias in Prompt-based Models: PALMS (https://proceedings.neurips.cc/paper/2021/hash/2e855f9489df0712b4bd8ea9e2848c5a-Abstract.html) and more recently LIMA (https://arxiv.org/abs/2305.11206) are two seminal works addressing the debiasing/alignment of prompted language models thanks to a small representative set of value-aligned examples.\n- Mitigating Bias in Prompt-based Models: Related to your Interpretability-guided Debiasing for prompted LMs, RAMP (https://aclanthology.org/2023.acl-short.126/) is a sampling procedure for in-context examples selection to improve the style accuracy of LMs prompted for gender-controlled MT.", "Typos_Grammar_Style_and_Presentation_Improvements": "35-36: It is advised to refrain from using ill-defined expressions like \"solving NLP tasks\", especially when models do not obtain perfect scores during evaluation. A formulation like \"models can be proficiently applied to most standard NLP tasks\" might better reflect the actual contribution of instruction tuning, without overblowing its effects.\n\nFigure 1: I believe the choice of using \"...\" for in-context examples was due to space reasons. However, I would suggest to replace these placeholders (at least for one of the examples) with an actual gold demonstration that might showcase similar attributes (e.g. feminine anti-stereotypical occupation highlighted in bold), to facilitate readers' comprehension.\n\n182-185: In line 183 you refer to extracted values \" raw token attributions\", but you actually have h scores for every (source, target) token pair, making the \"token\" term misleading here. Referring to those on the lines of \"granular per-unit attribution scores\" would clarify the nature of these scores. Moreover, when you mention that Integrated gradients is \"state-of-the-art\" you are likely implicitly referring to faithfulness and plausibility metrics (it would be best to make this explicit). However, Integrated Gradients could hardly be defined state-of-the-art for NLP tasks (see e.g. https://aclanthology.org/2023.eacl-main.245/ for a recent comprehensive comparison of several methods in which IG fares quite poorly, https://aclanthology.org/2022.emnlp-main.64/ for experimental evidence in a controlled setup showing simpler gradient-based methods obtaining better results). In light of this, referring to IG as \"commonly used\" would be more appropriate, without making an explicit statement on its effectiveness.\n\n196-197: I found a clear mention as to what the selected aggregation methods for $f$ and $g$ are in Appendix B, but I could not find any reference or mention of this in the main body of the paper.\n\nAppendix A: It would be fair to mention that the instruction tuning procedure was first introduced by the T5 authors (Raffel et al. 2020) as a complement of pre-training to further boost performance of text-to-text models on a variety of NLP tasks. Moreover, in the paper you mention several time the \"zero-shot performance\" of instruction-tuned models, implicitly referring to the setting where they perform the task without additional in-context examples. However, zero-shot performance is routinely used to describe generalization on unseen tasks, which is not the case for these systems given that MT was part of the instruction-tuning data. This point could also be clarified in Appendix A.\n\nAppendix B: If the euclidean norm is used for $g$, then it should be clarified that the resulting $A = \\mathbb{R}_{>0}^{S \\times T}$ s.t. $\\mathbb{R}_{>0} = \\{x \\in \\mathbb{R} | x > 0\\}$, given that in the generic definition provided in the main body of the paper the resulting scores are not strictly positive.\n\nTable 9: Bold is used to mark best instruction-tuned scores in the top part of the table, while it marks overall best (including MarianNMT) in the bottom part. The standard for this settings is to use one of (bold, underline) for marking best in category, and the other for overall best.\n\n231: Moderate manual polishing is too vague of an expression, please provide specifics of what adjustments were performed.\n\n288-289: Do \"best-performing models\" here correspond to the XXL variants, as mentioned in the previous paragraph?\n\n294-298: Punctuation typo, and I find the link with human raters dubious in this context, since automatic metrics are used as reference.\n\n351-357: I am not sure how the results of Table 3 support these affirmations, given that I don't see any particular relation between the distribution of $a_{prof, prof}$ scores and the Accuracy results in the last column. This should be further clarified.\n\n396-403: How are the terms \"showing a trend\" and \"does not show associations\" defined here? Looking at Table 4, both columns contain very low scores that can be hardly considered relevant, and calling this proof of their \"undeniable relevance\" seems quite out of place. In order to quantify whether this is truly the case, one could 1) perform statistical testing, reporting significance in difference between correct/wrong translations and/or 2) convert absolute difference scores into relative percentages, using the difference in $a_{ctrl, prof}$ in the two cases as baseline."}}]

Author rebuttal:

Meta-review: The paper "A Tale of Pronouns: Interpretability Informs Gender Bias Mitigation for Fairer Instruction-Tuned Machine Translation" presents work on examining how gender is translated from languages not having grammatical gender to languages having grammatical gender. To this end, the paper presents work on computing the gender bias in translated data. The paper also proposes a method for reducing the bias in translations. 

The main criticism points mentioned by the reviewers refers to previous work, but also to the generalization of the results based on the used data set to general MT.
In support of the paper the reviewers mention the experimental setup, the presented details and the insights gained from the experiments.
Decision: Accept-Main