SlowGuess/ACL_2020_OCR huggingface dataset:
For en-de and en-es we learn joint 32K BPE vocabularies on the training data (Sennrich et al., 2016). For en-he we use separate source and target vocabularies. The Hebrew vocabulary is a 2k- merge BPE vocabulary, following the recommendations of Ding et al. (2019) for smaller vocabularies when translating into lower-resource languages. For the en-he source vocabulary we experimented both with learning a new  $32\mathrm{K}$  vocabulary and with reusing the joint BPE vocabulary trained on the largest set - en-de - which lets us initialize the enhe system with the pre-trained en-de model. The latter resulted in higher BLEU and faster training.
For all models we use a Transformer model (Vaswani et al., 2017) with the 'base' parameter settings given in Tensor2Tensor (Vaswani et al., 2018). We train baselines to validation set BLEU convergence on one GPU, delaying gradient updates by factor 4 to simulate 4 GPUs (Saunders et al., 2018). During fine-tuning training is continued without learning rate resetting. Normal and lattice-constrained decoding is via  $\mathrm{SGNMT}^3$  with beam size 4. BLEU scores are calculated for cased, detokenized output using SacreBLEU (Post, 2018)
3.3 Lattice rescoring with debiased models
For lattice rescoring we require a transducer  $T$  containing gender-inflected forms of words in the target vocabulary. To obtain the vocabulary for German we use all unique words in the full target training dataset. For Spanish and Hebrew, which have smaller and less diverse training sets, we use 2018
<sup>3</sup>https://github.com/ucam-smt/sgnmt
7729
OpenSubtitles word lists<sup>4</sup>. We then use DEMorphy (Altinok, 2018) for German, spaCy (Honnibal and Montani, 2017) for Spanish and the small set of gendered suffixes for Hebrew (Schwarzwald, 1982) to approximately lemmatize each vocabulary word and generate its alternately-gendered forms. While there are almost certainly paths in  $T$  containing non-words, we expect these to have low likelihood under the debiasing models. For lattice compositions we use the efficient OpenFST implementations (Allauzen et al., 2007).
3.4.1 Baseline analysis
In Table 2 we compare our three baselines to commercial systems on WinoMT, using results quoted directly from Stanovsky et al. (2019). Our baselines achieve comparable accuracy, masculine/feminine bias score  $\Delta G$  and pro/anti stereotypical bias score  $\Delta S$  to four commercial translation systems, outscoring at least one system for each metric on each language pair.
The  $\Delta S$  for our en-es baseline is surprisingly small. Investigation shows this model predicts male and female entities in a ratio of over 6:1. Since almost all entities are translated as male, pro- and anti-stereotypical class accuracy are both about  $50\%$ , making  $\Delta S$  very small. This highlights the importance of considering  $\Delta S$  in the context of  $\Delta G$  and M:F prediction ratio.
3.4.2 Counterfactual adaptation
Table 3 compares our baseline model with the results of unregularised fine-tuning on the counterfactual sets described in Section 2.2.2.
Fine-tuning for one epoch on original, a subset of the original data with gendered English stopwords, gives slight improvement in WinoMT accuracy and  $\Delta G$  for all language pairs, while  $\Delta S$  worsens. We suggest this set consolidates examples present in the full dataset, improving performance on gendered entities generally but emphasizing stereotypical roles.
On the  $FTrans$  original set  $\Delta G$  increases sharply relative to the original set, while  $\Delta S$  decreases. We suspect this set suffers from bias amplification (Zhao et al., 2017) introduced by the baseline system during forward-translation. The model therefore over-predicts male entities even more heavily
than we would expect given the gender makeup of the adaptation data's source side. Over-predicting male entities lowers  $\Delta S$  artificially.
Adapting to  $FTrans$  swapped increases accuracy and decreases both  $\Delta G$  and  $\Delta S$  relative to the baseline for en-de and en-es. This is the desired result, but not a particularly strong one, and it is not replicated for en-he. The balanced set has a very similar effect to the  $FTrans$  swapped set, with a smaller test BLEU difference from the baseline.
We do find that the largest improvement in WinoMT accuracy consistently corresponds to the model predicting male and female entities in the closest ratio (see Appendix A). However, the best ratios for models adapted to these datasets are 2:1 or higher, and the accuracy improvement is small.
The purpose of EWC regularization is to avoid catastrophic forgetting of general translation ability. This does not occur in the counterfactual experiments, so we do not apply EWC. Moreover, WinoMT accuracy gains are small with standard fine-tuning, which allows maximum adaptation: we suspect EWC would prevent any improvements.
Overall, improvements from fine-tuning on counterfactual datasets (FTrans swapped and balanced) are present. However, they are not very different from the improvements when fine-tuning on equivalent non-counterfactual sets (original and FTrans original). Improvements are also inconsistent across language pairs.
3.4.3 Handcrafted profession set adaptation
Results for fine-tuning on the handcrafted set are given in lines 3-6 of Table 4. These experiments take place in minutes on a single GPU, compared to several hours when fine-tuning on the counterfactual sets and far longer if training from scratch.
Fine-tuning on the handcrafted sets gives a much faster BLEU drop than fine-tuning on counterfactual sets. This is unsurprising since the handcrafted sets are domains of new sentences with consistent sentence length and structure. By contrast the counterfactual sets are less repetitive and close to subsets of the original training data, slowing forgetting. We believe the degradation here is limited only by the ease of fitting the small handcrafted sets.
Line 4 of Table 4 adapts to the handcrafted set, stopping when validation BLEU degrades by  $5\%$  on each language pair. This gives a WinoMT accuracy up to 19 points above the baseline, far more improvement than the best counterfactual result. Difference in gender score  $\Delta G$  improves by at least
4Accessed Oct 2019 from https://github.com/hermitdave/FrequencyWords/
7730
en-de	en-es	en-he
Acc	ΔG	ΔS	Acc	ΔG	ΔS	Acc	ΔG	ΔS
Microsoft	74.1	0.0	30.2	47.3	36.8	23.2	48.1	14.9	32.9
Google	59.4	12.5	12.5	53.1	23.4	21.3	53.7	7.9	37.8
Amazon	62.4	12.9	16.7	59.4	15.4	22.3	50.5	10.3	47.3
SYSTRAN	48.6	34.5	10.3	45.6	46.3	15.0	46.6	20.5	24.5
Baseline	60.1	18.6	13.4	49.6	36.7	2.0	51.3	15.1	26.4
Table 2: WinoMT accuracy, masculine/feminine bias score  $\Delta G$  and pro/anti stereotypical bias score  $\Delta S$  for our baselines compared to commercial systems, whose scores are quoted directly from Stanovsky et al. (2019).
en-de	en-es	en-he
BLEU	Acc	ΔG	ΔS	BLEU	Acc	ΔG	ΔS	BLEU	Acc	ΔG	ΔS
Baseline	42.7	60.1	18.6	13.4	27.8	49.6	36.7	2.0	23.8	51.3	15.1	26.4
Original	41.8	60.7	15.9	15.6	28.3	53.0	24.3	10.8	23.5	53.6	12.2	31.7
FTrans original	43.3	60.0	20.0	13.9	27.4	51.6	31.6	-4.8	23.4	48.7	23.0	20.9
FTrans swapped	43.4	63.0	15.4	12.7	27.4	53.7	24.5	-3.8	23.7	48.1	20.7	22.7
Balanced	42.5	64.0	12.6	12.4	27.7	52.8	26.2	1.9	23.8	48.3	20.8	24.0
Table 3: General test set BLEU and WinoMT scores after unregularised fine-tuning the baseline on four gender-based adaptation datasets. Improvements are inconsistent across language pairs.
en-de	en-es	en-he
BLEU	Acc	ΔG	ΔS	BLEU	Acc	ΔG	ΔS	BLEU	Acc	ΔG	ΔS
1	Baseline	42.7	60.1	18.6	13.4	27.8	49.6	36.7	2.0	23.8	51.3	15.1	26.4
2	Balanced	42.5	64.0	12.6	12.4	27.7	52.8	26.2	1.9	23.8	48.3	20.8	24.0
3	Handcrafted (no overlap)	40.6	71.2	3.9	10.6	26.5	64.1	9.5	-10.3	23.1	56.5	-6.2	28.9
4	Handcrafted	40.8	78.3	-0.7	6.5	26.7	68.6	5.2	-8.7	22.9	65.7	-3.3	20.2
5	Handcrafted (converged)	36.5	85.3	-3.2	6.3	25.3	72.4	0.8	-3.9	22.5	72.6	-4.2	21.0
6	Handcrafted EWC	42.2	74.2	2.2	8.4	27.2	67.8	5.8	-8.2	23.3	65.2	-0.4	25.3
7	Rescore 1 with 3	42.7	68.3	7.6	11.8	27.8	62.4	11.1	-9.7	23.9	56.2	2.8	23.0
8	Rescore 1 with 4	42.7	74.5	2.1	6.5	27.8	64.2	9.7	-10.8	23.9	58.4	2.7	18.6
9	Rescore 1 with 5	42.5	81.7	-2.4	1.5	27.7	68.4	5.6	-8.0	23.6	63.8	0.7	12.9
Table 4: General test set BLEU and WinoMT scores after fine-tuning on the handcrafted profession set, compared to fine-tuning on the most consistent counterfactual set. Lines 1-2 duplicated from Table 3. Lines 3-4 vary adaptation data. Lines 5-6 vary adaptation training procedure. Lines 7-9 apply lattice rescoring to baseline hypotheses.
a factor of 4. Stereotyping score  $\Delta S$  also improves far more than for counterfactual fine-tuning. Unlike the Table 3 results, the improvement is consistent across all WinoMT metrics and all language pairs.
The model adapted to no-overlap handcrafted data (line 3) gives a similar drop in BLEU to the model in line 4. This model also gives stronger and more consistent WinoMT improvements over the baseline compared to the balanced counterfactual set, despite the implausibly strict scenario of no English profession vocabulary in common with the challenge set. This demonstrates that the adapted model does not simply memorise vocabulary.
The drop in BLEU and improvement on WinoMT can be explored by varying the training procedure. The model of line 5 simply adapts to handcrafted data for more iterations with no regularisation, to approximate loss convergence on the handcrafted set. This leads to a severe drop in BLEU, but even higher WinoMT scores.
In line 6 we regularise adaptation with EWC. There is a trade-off between general translation performance and WinoMT accuracy. With EWC regularization tuned to balance validation BLEU and WinoMT accuracy, the decrease is limited to about 0.5 BLEU on each language pair. Adapting to convergence, as in line 5, would lead to further WinoMT gains at the expense of BLEU.
3.4.4 Lattice rescoring with debiased models
In lines 7-9 of Table 4 we consider lattice-rescoring the baseline output, using three models debiased on the handcrafted data.
Line 7 rescores the general test set hypotheses (line 1) with a model adapted to handcrafted data that has no source language profession vocabulary overlap with the test set (line 3). This scheme shows no BLEU degradation from the baseline on any language and in fact a slight improvement on en-he. Accuracy improvements on WinoMT
7731
en-de	en-es	en-he
Acc	ΔG	ΔS	Acc	ΔG	ΔS	Acc	ΔG	ΔS
1	82.0 (74.1)	-3.0 (0.0)	4.0 (30.2)	65.8 (47.3)	3.8 (36.8)	1.9 (23.2)	63.9 (48.1)	-2.6 (14.9)	23.8 (32.9)
2	80.0 (59.4)	-3.0 (12.5)	2.7 (12.5)	68.9 (53.1)	0.6 (23.4)	4.6 (21.3)	64.6 (53.7)	-1.8 (7.9)	21.5 (37.8)
3	81.8 (62.4)	-2.6 (12.9)	4.3 (16.7)	71.1 (59.4)	0.7 (15.4)	6.7 (22.3)	62.8 (50.5)	-1.1 (10.3)	26.9 (47.3)
4	78.4 (48.6)	-4.0 (34.5)	5.3 (10.3)	66.0 (45.6)	4.2 (46.3)	-2.1 (15.0)	62.5 (46.6)	-2.0 (20.5)	10.2 (24.5)
Table 5: We generate gender-inflected lattices from commercial system translations, collected by Stanovsky et al. (2019) (1: Microsoft, 2: Google, 3: Amazon, 4: SYSTRAN). We then rescore with the debiased model from line 5 of Table 4. Scores are for the rescored hypotheses, with bracketed baseline scores duplicated from Table 2.
are only slightly lower than for decoding with the rescoring model directly, as in line 3.
In line 8, lattice rescoring with the nonconverged model adapted to handcrafted data (line 4) likewise leaves general BLEU unchanged or slightly improved. When lattice rescoring the WinoMT challenge set,  $79\%$ ,  $76\%$  and  $49\%$  of the accuracy improvement is maintained on en-de, en-es and en-he respectively. This corresponds to accuracy gains of up to  $30\%$  relative to the baselines with no general translation performance loss.
In line 9, lattice-rescoring with the converged model of line 5 limits BLEU degradation to 0.2 BLEU on all languages, while maintaining  $85\%$ ,  $82\%$  and  $58\%$  of the WinoMT accuracy improvement from the converged model for the three language pairs. Lattice rescoring with this model gives accuracy improvements over the baseline of  $36\%$ ,  $38\%$  and  $24\%$  for en-de, en-es and en-he.
Rescoring en-he maintains a much smaller proportion of WinoMT accuracy improvement than en-de and en-es. We believe this is because the en-he baseline is particularly weak, due to a small and non-diverse training set. The baseline must produce some inflection of the correct entity before lattice rescoring can have an effect on gender bias.
3.4.5 Reducing gender bias in 'black box' commercial systems
Finally, in Table 5, we apply the gender inflection transducer to the commercial system translations<sup>5</sup> listed in Table 2. We find rescoring these lattices with our strongest debiasing model (line 5 of Table 4) substantially improves WinoMT accuracy for all systems and language pairs.
One interesting observation is that WinoMT accuracy after rescoring tends to fall in a fairly narrow range for each language relative to the performance range of the baseline systems. For example, a  $25.5\%$  range in baseline en-de accuracy
becomes a  $3.6\%$  range after rescoring. This suggests that our rescoring approach is not limited as much by the bias level of the baseline system as by the gender-inflection transducer and the models used in rescoring. Indeed, we emphasise that the large improvements reported in Table 5 do not require any knowledge of the commercial systems or the data they were trained on; we use only the translation hypotheses they produce and our own rescoring model and transducer.
We treat the presence of gender bias in NMT systems as a domain adaptation problem. We demonstrate strong improvements under the WinoMT challenge set by adapting to tiny, handcrafted gender-balanced datasets for three language pairs.
While naive domain adaptation leads to catastrophic forgetting, we further demonstrate two approaches to limit this: EWC and a lattice rescoring approach. Both allow debiasing while maintaining general translation performance. Lattice rescoring, although a two-step procedure, allows far more debiasing and potentially no degradation, without requiring access to the original model.
We suggest small-domain adaptation as a more effective and efficient approach to debiasing machine translation than counterfactual data augmentation. We do not claim to fix the bias problem in NMT, but demonstrate that bias can be reduced without degradation in overall translation quality.
This work was supported by EPSRC grants EP/M508007/1 and EP/N509620/1 and has been performed using resources provided by the Cambridge Tier-2 system operated by the University of Cambridge Research Computing Service funded by EPSRC Tier-2 capital grant EP/P020259/1.
5The raw commercial system translations are provided by the authors of Stanovsky et al. (2019) at https://github.com/gabrielStanovsky/mt_gender
http://www.hpc.cam.ac.uk
7732
Lauren Ackerman. 2019. Syntactic and cognitive issues in investigating gendered coreference. *Glossa: a journal of general linguistics*, 4(1).
Cyril Allauzen, Michael Riley, Johan Schalkwyk, Wojciech Skut, and Mehryar Mohri. 2007. OpenFst: A general and efficient weighted finite-state transducer library. In International Conference on Implementation and Application of Automata, pages 11-23. Springer.
Duygu Altinok. 2018. DEMorphy, German language morphological analyzer. arXiv preprint arXiv:1803.00902.
Antonio Valerio Miceli Barone, Barry Haddow, Ulrich Germann, and Rico Sennrich. 2017. Regularization techniques for fine-tuning in neural machine translation. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 1489-1494, Copenhagen, Denmark. Association for Computational Linguistics.
Loic Barrault, Ondrej Bojar, Marta R. Costa-jussa, Christian Federmann, Mark Fishel, Yvette Graham, Barry Haddow, Matthias Huck, Philipp Koehn, Shervin Malmasi, Christof Monz, Mathias Müller, Santanu Pal, Matt Post, and Marcos Zampieri. 2019. Findings of the 2019 conference on machine translation (WMT19). In Proceedings of the Fourth Conference on Machine Translation (Volume 2: Shared Task Papers, Day 1), pages 1-61, Florence, Italy. Association for Computational Linguistics.
Tolga Bolukbasi, Kai-Wei Chang, James Y Zou, Venkatesh Saligrama, and Adam T Kalai. 2016. Man is to computer programmer as woman is to homemaker? Debiasing word embeddings. In Advances in neural information processing systems, pages 4349-4357.
Mauro Cettolo, Jan Niehues, Sebastian Stüker, Luisa Bentivogli, and Marcello Federico. 2014. Report on the 11th IWSLT evaluation campaign, IWSLT 2014. In Proceedings of the International Workshop on Spoken Language Translation, Hanoi, Vietnam, page 57.
Shuoyang Ding, Adithya Renduchintala, and Kevin Duh. 2019. A call for prudent choice of subword merge operations in neural machine translation. In Proceedings of Machine Translation Summit XVII Volume 1: Research Track, pages 204-213, Dublin, Ireland. European Association for Machine Translation.
Joel Escudé Font and Marta R. Costa-jussà. 2019. Equalizing gender bias in neural machine translation with word embeddings techniques. In Proceedings of the First Workshop on Gender Bias in Natural Language Processing, pages 147-154, Florence, Italy. Association for Computational Linguistics.
M. Amin Farajian, Marco Turchi, Matteo Negri, and Marcello Federico. 2017. Multi-domain neural machine translation through unsupervised adaptation. In Proceedings of the Second Conference on Machine Translation, pages 127-137, Copenhagen, Denmark. Association for Computational Linguistics.
Markus Freitag and Yaser Al-Onaizan. 2016. Fast domain adaptation for Neural Machine Translation. CoRR, abs/1612.06897.
Robert M French. 1999. Catastrophic forgetting in connectionist networks. Trends in cognitive sciences, 3(4):128-135.
Hila Gonen and Yoav Goldberg. 2019. Lipstick on a pig: Debiasing methods cover up systematic gender biases in word embeddings but do not remove them. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 609-614.
AI HLEG. 2019. Ethics guidelines for trustworthy AI. High-Level Expert Group on Artificial Intelligence.
Matthew Honnibal and Ines Montani. 2017. spaCy 2: Natural language understanding with bloom embeddings. Convolutional Neural Networks and Incremental Parsing.
Dirk Hovy and Shannon L. Spruit. 2016. The social impact of natural language processing. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 591-598, Berlin, Germany. Association for Computational Linguistics.
James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei A Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Grabska-Barwinska, et al. 2017. Overcoming catastrophic forgetting in neural networks. Proceedings of the National Academy of Sciences of the United States of America, 114(13):3521-3526.
Kaiji Lu, Piotr Mardziel, Fangjing Wu, Preetam Amancharla, and Anupam Datta. 2018. Gender bias in neural natural language processing. arXiv preprint arXiv:1807.11714.
Paul Michel and Graham Neubig. 2018. Extreme adaptation for personalized neural machine translation. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 312-318, Melbourne, Australia. Association for Computational Linguistics.
Julia Misersky, Asifa Majid, and Tineke M Snijders. 2019. Grammatical gender in German influences how role-nouns are interpreted: Evidence from erps. Discourse Processes, 56(8):643-654.
7733
Amit Moryossef, Roee Aharoni, and Yoav Goldberg. 2019. Filling gender & number gaps in neural machine translation with black-box context injection. In Proceedings of the First Workshop on Gender Bias in Natural Language Processing, pages 49-54, Florence, Italy. Association for Computational Linguistics.
Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. BLEU: a method for automatic evaluation of machine translation. In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, pages 311-318, Philadelphia, Pennsylvania, USA. Association for Computational Linguistics.
Ji Ho Park, Jamin Shin, and Pascale Fung. 2018. Reducing gender bias in abusive language detection. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 2799-2804, Brussels, Belgium. Association for Computational Linguistics.
Matt Post. 2018. A call for clarity in reporting BLEU scores. In Proceedings of the Third Conference on Machine Translation: Research Papers, pages 186-191, Belgium, Brussels. Association for Computational Linguistics.
Marcelo OR Prates, Pedro H Avelar, and Luís C Lamb. 2019. Assessing gender bias in machine translation: a case study with google translate. *Neural Computing and Applications*, pages 1-19.
Rachel Rudinger, Jason Naradowsky, Brian Leonard, and Benjamin Van Durme. 2018. Gender bias in coreference resolution. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Short Papers), pages 8-14, New Orleans, Louisiana. Association for Computational Linguistics.
Danielle Saunders, Felix Stahlberg, Adrià de Gispert, and Bill Byrne. 2018. Multi-representation ensembles and delayed SGD updates improve syntax-based NMT. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 319-325, Melbourne, Australia. Association for Computational Linguistics.
Danielle Saunders, Felix Stahlberg, Adrià de Gispert, and Bill Byrne. 2019. Domain adaptive inference for neural machine translation. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 222-228, Florence, Italy. Association for Computational Linguistics.
Ora Schwarzwald. 1982. Feminine formation in modern Hebrew. Hebrew Annual Review, 6:153-178.
Rico Sennrich, Barry Haddow, and Alexandra Birch. 2016. Neural machine translation of rare words with subword units. In Proceedings of the 54th Annual Meeting of the Association for Computational