Reducing Gender Bias in Neural Machine Translation as a Domain Adaptation Problem	Training data for NLP tasks often exhibits gender bias in that fewer sentences refer to women than to men. In Neural Machine Translation (NMT) gender bias has been shown to reduce translation quality, particularly when the target language has grammatical gender. The recent WinoMT challenge set allows us to measure this effect directly (Stanovsky et al, 2019). Ideally we would reduce system bias by simply debiasing all data prior to training, but achieving this effectively is itself a challenge. Rather than attempt to create a `balanced' dataset, we use transfer learning on a small set of trusted, gender-balanced examples. This approach gives strong and consistent improvements in gender debiasing with much less computational cost than training from scratch. A known pitfall of transfer learning on new domains is `catastrophic forgetting', which we address both in adaptation and in inference. During adaptation we show that Elastic Weight Consolidation allows a performance trade-off between general translation quality and bias reduction.

Experiments ::: Results ::: Baseline analysis	In Table TABREF36 we compare our three baselines to commercial systems on WinoMT, using results quoted directly from . Our baselines achieve comparable accuracy, masculine/feminine bias score $\Delta G$ and pro/anti stereotypical bias score $\Delta S$ to four commercial translation systems, outscoring at least one system for each metric on each language pair. The $\Delta S$ for our en-es baseline is surprisingly small. Investigation shows this model predicts male and female entities in a ratio of over 6:1. Since almost all entities are translated as male, pro- and anti-stereotypical class accuracy are both about 50%, making $\Delta S$ very small. This highlights the importance of considering $\Delta S$ in the context of $\Delta G$ and M:F prediction ratio.

Gender bias in machine translation ::: WinoMT challenge set and metrics	WinoMT  is a recently proposed challenge set for gender bias in NMT. Moreover it is the only significant challenge set we are aware of to evaluate translation gender bias comparably across several language pairs. It permits automatic bias evaluation for translation from English to eight target languages with grammatical gender. The source side of WinoMT is 3888 concatenated sentences from Winogender  and WinoBias . These are coreference resolution datasets in which each sentence contains a primary entity which is co-referent with a pronoun – the doctor in the first example above and the developer in the second – and a secondary entity – the nurse and the cleaner respectively. WinoMT evaluation extracts the grammatical gender of the primary entity from each translation hypothesis by automatic word alignment followed by morphological analysis. WinoMT then compares the translated primary entity with the gold gender, with the objective being a correctly gendered translation. The authors emphasise the following metrics over the challenge set: Accuracy – percentage of hypotheses with the correctly gendered primary entity.

A model that generates male forms for almost all test sentences, stereotypical roles or not, will have an extremely low $\Delta S$, since its pro- and anti-stereotypical class accuracy will both be about 50%. Consequently we also report: M:F – ratio of hypotheses with male predictions to those with female predictions. Ideally this should be close to 1.0, since the WinoMT challenge set is gender-balanced. While M:F correlates strongly with $\Delta G$, we consider M:F easier to interpret, particularly since very high or low M:F reduce the relevance of $\Delta S$. Finally, we wish to reduce gender bias without reducing translation performance. We report BLEU  on separate, general test sets for each language pair. WinoMT is designed to work without target language references, and so it is not possible to measure translation performance on this set by measures such as BLEU.

To the best of our knowledge this work is the first to attempt NMT bias reduction by fine-tuning, rather than retraining. We consider three aspects of this adaptation problem: creating less biased adaptation data, parameter adaptation using this data, and inference with the debiased models produced by adaptation. Regarding data, we suggest that a small, trusted gender-balanced set could allow more efficient and effective gender debiasing than a larger, noisier set. To explore this we create a tiny, handcrafted profession-based dataset for transfer learning. For contrast, we also consider fine-tuning on a counterfactual subset of the full dataset and propose a straightforward scheme for artificially gender-balancing parallel text for NMT. We find that during domain adaptation improvement on the gender-debiased domain comes at the expense of translation quality due to catastrophic forgetting . We can balance improvement and forgetting with a regularised training procedure, Elastic Weight Consolidation (EWC), or in inference by a two-step lattice rescoring procedure. We experiment with three language pairs, assessing the impact of debiasing on general domain BLEU and on the WinoMT challenge set .

We find that continued training on the handcrafted set gives far stronger and more consistent improvements in gender-debiasing with orders of magnitude less training time, although as expected general translation performance as measured by BLEU decreases. We further show that regularised adaptation with EWC can reduce bias while limiting degradation in general translation quality. We also present a lattice rescoring procedure in which initial hypotheses produced by the biased baseline system are transduced to create gender-inflected search spaces which can be rescored by the adapted model. We believe this approach, rescoring with models targeted to remove bias, is novel in NMT. The rescoring procedure improves WinoMT accuracy by up to 30% with no decrease in BLEU on the general test set. Recent recommendations for ethics in Artificial Intelligence have suggested that social biases or imbalances in a dataset be addressed prior to model training . This recommendation presupposes that the source of bias in a dataset is both obvious and easily adjusted. We show that debiasing a full NMT dataset is difficult, and suggest alternative efficient and effective approaches for debiasing a model after it is trained.

Stereotyping score $\Delta S$ also improves far more than for counterfactual fine-tuning. Unlike the Table TABREF37 results, the improvement is consistent across all WinoMT metrics and all language pairs. The model adapted to no-overlap handcrafted data (line 3) gives a similar drop in BLEU to the model in line 4. This model also gives stronger and more consistent WinoMT improvements over the baseline compared to the balanced counterfactual set, despite the implausibly strict scenario of no English profession vocabulary in common with the challenge set. This demonstrates that the adapted model does not simply memorise vocabulary. The drop in BLEU and improvement on WinoMT can be explored by varying the training procedure. The model of line 5 simply adapts to handcrafted data for more iterations with no regularisation, to approximate loss convergence on the handcrafted set. This leads to a severe drop in BLEU, but even higher WinoMT scores. In line 6 we regularise adaptation with EWC. There is a trade-off between general translation performance and WinoMT accuracy.

The acceptor ${\mathcal {P}}(\mathbf {y_B}) = \text{proj}_\text{output} (Y_B \circ T )$ defines a language consisting of all the gender-inflected versions of the biased first-pass translation $\mathbf {y_B}$ that are allowed by $T$ (Figure FIGREF21). We can now decode with lattice rescoring ($LR$) by constraining inference to ${\mathcal {P}}({\mathbf {y_B}})$: In practice we use beam search to decode the various hypotheses, and construct $T$ using heuristics on large vocabulary lists for each target language.

Experiments ::: Results ::: Reducing gender bias in `black box' commercial systems	Finally, in Table TABREF41, we apply the gender inflection transducer to the commercial system translations listed in Table TABREF36. We find rescoring these lattices with our strongest debiasing model (line 5 of Table TABREF40) substantially improves WinoMT accuracy for all systems and language pairs. One interesting observation is that WinoMT accuracy after rescoring tends to fall in a fairly narrow range for each language relative to the performance range of the baseline systems. For example, a 25.5% range in baseline en-de accuracy becomes a 3.6% range after rescoring. This suggests that our rescoring approach is not limited as much by the bias level of the baseline system as by the gender-inflection transducer and the models used in rescoring. Indeed, we emphasise that the large improvements reported in Table TABREF41 do not require any knowledge of the commercial systems or the data they were trained on; we use only the translation hypotheses they produce and our own rescoring model and transducer.

However, their system focuses on words in isolation, and is difficult to apply to co-reference and conjunction situations with more than one term to swap, reducing its practicality for large MT datasets. Recent work recognizes that NMT can be adapted to domains with desired attributes using small datasets , . Our choice of a small, trusted dataset for adaptation specifically to a debiased domain connects to recent work in data selection by , in which fine-tuning on less noisy data improves translation performance. Similarly we propose fine-tuning on less biased data to reduce gender bias in translations. This is loosely the inverse of the approach described by  for monolingual abusive language detection, which pre-trains on a larger, less biased set.

Experiments ::: Lattice rescoring with debiased models	For lattice rescoring we require a transducer $T$ containing gender-inflected forms of words in the target vocabulary. To obtain the vocabulary for German we use all unique words in the full target training dataset. For Spanish and Hebrew, which have smaller and less diverse training sets, we use 2018 OpenSubtitles word lists. We then use DEMorphy  for German, spaCy  for Spanish and the small set of gendered suffixes for Hebrew  to approximately lemmatize each vocabulary word and generate its alternately-gendered forms. While there are almost certainly paths in $T$ containing non-words, we expect these to have low likelihood under the debiasing models. For lattice compositions we use the efficient OpenFST implementations .

Introduction ::: Related work	 treat gender as a domain for machine translation, training from scratch by augmenting Europarl data with a tag indicating the speaker's gender. This does not inherently remove gender bias from the system but allows control over the translation hypothesis gender.  similarly prepend a short phrase at inference time which acts as a gender domain label for the entire sentence. These approaches are not directly applicable to text which may have more than one gendered entity per sentence, as in coreference resolution tasks.  train NMT models from scratch with debiased word embeddings. They demonstrate improved performance on an English-Spanish occupations task with a single profession and pronoun per sentence. We assess our fine-tuning approaches on the WinoMT coreference set, with two entities to resolve per sentence. For monolingual NLP tasks a typical approach is gender debiasing using counterfactual data augmentation where for each gendered sentence in the data a gender-swapped equivalent is added.  show improvement in coreference resolution for English using counterfactual data.  demonstrate a more complicated scheme for gender-inflected languages.

Experiments ::: Training and inference	For all models we use a Transformer model  with the `base' parameter settings given in Tensor2Tensor . We train baselines to validation set BLEU convergence on one GPU, delaying gradient updates by factor 4 to simulate 4 GPUs . During fine-tuning training is continued without learning rate resetting. Normal and lattice-constrained decoding is via SGNMT with beam size 4. BLEU scores are calculated for cased, detokenized output using SacreBLEU

$\mathbf {\Delta G}$ – difference in $F_1$ score between the set of sentences with masculine entities and the set with feminine entities. $\mathbf {\Delta S}$ – difference in accuracy between the set of sentences with pro-stereotypical (`pro') entities and those with anti-stereotypical (`anti') entities, as determined by  using US labour statistics. For example, the `pro' set contains male doctors and female nurses, while `anti' contains female doctors and male nurses. Our main objective is increasing accuracy. We also report on $\Delta G$ and $\Delta S$ for ease of comparison to previous work. Ideally the absolute values of $\Delta G$ and $\Delta S$ should be close to 0. A high positive $\Delta G$ indicates that a model translates male entities better, while a high positive $\Delta S$ indicates that a model stereotypes male and female entities. Large negative values for $\Delta G$ and $\Delta S$, indicating a bias towards female or anti-stereotypical translation, are as undesirable as large positive values. We note that $\Delta S$ can be significantly skewed by very biased systems.

Experiments ::: Languages and data	WinoMT provides an evaluation framework for translation from English to eight diverse languages. We select three pairs for experiments: English to German (en-de), English to Spanish (en-es) and English to Hebrew (en-he). Our selection covers three language groups with varying linguistic properties: Germanic, Romance and Semitic. Training data available for each language pair also varies in quantity and quality. We filter training data based on parallel sentence lengths and length ratios. For en-de, we use 17.6M sentence pairs from WMT19 news task datasets . We validate on newstest17 and test on newstest18. For en-es we use 10M sentence pairs from the United Nations Parallel Corpus . While still a large set, the UNCorpus exhibits far less diversity than the en-de training data. We validate on newstest12 and test on newstest13. For en-he we use 185K sentence pairs from the multilingual TED talks corpus . This is both a specialized domain and a much smaller training set. We validate on the IWSLT 2012 test set and test on IWSLT 2014.

Table TABREF29 summarises the sizes of datasets used, including their proportion of gendered sentences and ratio of sentences in the English source data containing male and female stopwords. A gendered sentence contains at least one English gendered stopword as used by . Interestingly all three datasets have about the same proportion of gendered sentences: 11-12% of the overall set. While en-es appears to have a much more balanced gender ratio than the other pairs, examining the data shows this stems largely from sections of the UNCorpus containing phrases like `empower women' and `violence against women', rather than gender-balanced professional entities. For en-de and en-es we learn joint 32K BPE vocabularies on the training data . For en-he we use separate source and target vocabularies. The Hebrew vocabulary is a 2k-merge BPE vocabulary, following the recommendations of  for smaller vocabularies when translating into lower-resource languages. For the en-he source vocabulary we experimented both with learning a new 32K vocabulary and with reusing the joint BPE vocabulary trained on the largest set – en-de – which lets us initialize the en-he system with the pre-trained en-de model.

Acknowledgments	This work was supported by EPSRC grants EP/M508007/1 and EP/N509620/1 and has been performed using resources provided by the Cambridge Tier-2 system operated by the University of Cambridge Research Computing Service funded by EPSRC Tier-2 capital grant EP/P020259/1.

Introduction	As language processing tools become more prevalent concern has grown over their susceptibility to social biases and their potential to propagate bias , . Natural language training data inevitably reflects biases present in our society. For example, gender bias manifests itself in training data which features more examples of men than of women. Tools trained on such data will then exhibit or even amplify the biases . Gender bias is a particularly important problem for Neural Machine Translation (NMT) into gender-inflected languages. An over-prevalence of some gendered forms in the training data leads to translations with identifiable errors . Translations are better for sentences involving men and for sentences containing stereotypical gender roles. For example, mentions of male doctors are more reliably translated than those of male nurses , . Recent approaches to the bias problem in NLP have involved training from scratch on artificially gender-balanced versions of the original dataset ,  or with de-biased embeddings , . While these approaches may be effective, training from scratch is inefficient and gender-balancing embeddings or large parallel datasets are challenging problems . Instead we propose treating gender debiasing as a domain adaptation problem, since NMT models can very quickly adapt to a new domain .

Experiments ::: Results ::: Lattice rescoring with debiased models	In lines 7-9 of Table TABREF40 we consider lattice-rescoring the baseline output, using three models debiased on the handcrafted data. Line 7 rescores the general test set hypotheses (line 1) with a model adapted to handcrafted data that has no source language profession vocabulary overlap with the test set (line 3). This scheme shows no BLEU degradation from the baseline on any language and in fact a slight improvement on en-he. Accuracy improvements on WinoMT are only slightly lower than for decoding with the rescoring model directly, as in line 3. In line 8, lattice rescoring with the non-converged model adapted to handcrafted data (line 4) likewise leaves general BLEU unchanged or slightly improved. When lattice rescoring the WinoMT challenge set, 79%, 76% and 49% of the accuracy improvement is maintained on en-de, en-es and en-he respectively. This corresponds to accuracy gains of up to 30% relative to the baselines with no general translation performance loss.

Gender bias in machine translation	We focus on translating coreference sentences containing professions as a representative subset of the gender bias problem. This follows much recent work on NLP gender bias , ,  including the release of WinoMT, a relevant challenge set for NMT . A sentence that highlights gender bias is: The doctor told the nurse that she had been busy. A human translator carrying out coreference resolution would infer that `she' refers to the doctor, and correctly translate the entity to German as Die Ärztin. An NMT model trained on a biased dataset in which most doctors are male might incorrectly default to the masculine form, Der Arzt. Data bias does not just affect translations of the stereotyped roles. Since NMT inference is usually left-to-right, a mistranslation can lead to further, more obvious mistakes later in the translation. For example, our baseline en-de system translates the English sentence The cleaner hates the developer because she always leaves the room dirty. to the German Der Reiniger haßt den Entwickler, weil er den Raum immer schmutzig lässt.

We use the approach described in  which swaps a fixed list of gendered stopwords (e.g. man / woman, he / she).. We then greedily forward-translate the gender-swapped English sentences with a baseline NMT model trained on the the full source and target text, producing gender-swapped target language sentences. This lets us compare four related sets for gender debiasing adaptation, as illustrated in Figure FIGREF11: Original: a subset of parallel sentences from the original training data where the source sentence contains gendered stopwords. Forward-translated (FTrans) original: the source side of the original set with forward-translated target sentences. Forward-translated (FTrans) swapped: the original source sentences are gender-swapped, then forward-translated to produce gender-swapped target sentences. Balanced: the concatenation of the original and FTrans swapped parallel datasets. This is twice the size of the other counterfactual sets. Comparing performance in adaptation of FTrans swapped and FTrans original lets us distinguish between the effects of gender-swapping and of obtaining target sentences from forward-translation.

Gender bias in machine translation ::: Debiasing while maintaining general translation performance ::: Regularized training	Regularized training is a well-established approach for minimizing catastrophic forgetting during domain adaptation of machine translation . One effective form is Elastic Weight Consolidation (EWC)  which in NMT has been shown to maintain or even improve original domain performance , . In EWC a regularization term is added to the original loss function $L$ when training the debiased model (DB): $\theta ^{B}_{j}$ are the converged parameters of the original biased model, and $\theta ^{DB}_j$ are the current debiased model parameters. $F_j=\mathbb {E} \big [ \nabla ^2 L(\theta ^{B}_j)\big ] $, a Fisher information estimate over samples from the biased data under the biased model. We apply EWC when performance on the original validation set drops, selecting hyperparameter $\lambda $ via validation set BLEU.

This avoids the need to identify and remove all possible biases prior to training, and has the added benefit of preserving privacy, since no access to the original data or knowledge of its contents is required. As evidence, in section SECREF43, we show this scheme can be applied to remove gender bias in the output of ‘black box‘ online commercial MT systems.

Gender bias in machine translation ::: Gender debiased datasets ::: Counterfactual datasets	For contrast, we fine-tune on an approximated counterfactual dataset. Counterfactual data augmentation is an intuitive solution to bias from data over-representation . It involves identifying the subset of sentences containing bias – in this case gendered terms – and, for each one, adding an equivalent sentence with the bias reversed – in this case a gender-swapped version. While counterfactual data augmentation is relatively simple for sentences in English, the process for inflected languages is challenging, involving identifying and updating words that are co-referent with all gendered entities in a sentence. Gender-swapping MT training data additionally requires that the same entities are swapped in the corresponding parallel sentence. A robust scheme for gender-swapping multiple entities in inflected language sentences directly, together with corresponding parallel text, is beyond the scope of this paper. Instead we suggest a rough but straightforward approach for counterfactual data augmentation for NMT which to the best of our knowledge is the first application to parallel sentences. We first perform simple gender-swapping on the subset of the English source sentences with gendered terms.

Gender bias in machine translation ::: Debiasing while maintaining general translation performance ::: Gender-inflected search spaces for rescoring with debiased models	An alternative approach for avoiding catastrophic forgetting takes inspiration from lattice rescoring for NMT  and Grammatical Error Correction . We assume we have two NMT models. With one we decode fluent translations which contain gender bias ($B$). For the one-best hypothesis we would translate: The other model has undergone debiasing ($DB$) at a cost to translation performance, producing: We construct a flower transducer $T$ that maps each word in the target language's vocabulary to itself, as well as to other forms of the same word with different gender inflections (Figure FIGREF21). We also construct $Y_B$, a lattice with one path representing the biased but fluent hypothesis $\mathbf {y_B}$ (Figure FIGREF21).

Conclusions	We treat the presence of gender bias in NMT systems as a domain adaptation problem. We demonstrate strong improvements under the WinoMT challenge set by adapting to tiny, handcrafted gender-balanced datasets for three language pairs. While naive domain adaptation leads to catastrophic forgetting, we further demonstrate two approaches to limit this: EWC and a lattice rescoring approach. Both allow debiasing while maintaining general translation performance. Lattice rescoring, although a two-step procedure, allows far more debiasing and potentially no degradation, without requiring access to the original model. We suggest small-domain adaptation as a more effective and efficient approach to debiasing machine translation than counterfactual data augmentation. We do not claim to fix the bias problem in NMT, but demonstrate that bias can be reduced without degradation in overall translation quality.

Here not only is `developer' mistranslated as the masculine den Entwickler instead of the feminine die Entwicklerin, but an unambiguous pronoun translation later in the sentence is incorrect: er (`he') is produced instead of sie (`she'). In practice, not all translations with gender-inflected words can be unambiguously resolved. A simple example is: The doctor had been busy. This would likely be translated with a masculine entity according to the conventions of a language, unless extra-sentential context was available. As well, some languages have adopted gender-neutral singular pronouns and profession terms, both to include non-binary people and to avoid the social biases of gendered language , although most languages lack widely-accepted conventions . This paper addresses gender bias that can be resolved at the sentence level and evaluated with existing test sets, and does not address these broader challenges.

Gender bias in machine translation ::: Gender debiased datasets ::: Handcrafted profession dataset	Our hypothesis is that the absence of gender bias can be treated as a small domain for the purposes of NMT model adaptation. In this case a well-formed small dataset may give better results than attempts at debiasing the entire original dataset. We therefore construct a tiny, trivial set of gender-balanced English sentences which we can easily translate into each target language. The sentences follow the template: The $[$PROFESSION$]$ finished $[$his$|$her$]$ work. We refer to this as the handcrafted set. Each profession is from the list collected by  from US labour statistics. We simplify this list by removing field-specific adjectives. For example, we have a single profession `engineer', as opposed to specifying industrial engineer, locomotive engineer, etc. In total we select 194 professions, giving just 388 sentences in a gender-balanced set. With manually translated masculine and feminine templates, we simply translate the masculine and feminine forms of each listed profession for each target language.

Gender bias in machine translation ::: Debiasing while maintaining general translation performance	Fine-tuning a converged neural network on data from a distinct domain typically leads to catastrophic forgetting of the original domain . We wish to adapt to the gender-balanced domain without losing general translation performance. This is a particular problem when fine-tuning on the very small and distinct handcrafted adaptation sets.

During inference we propose a lattice-rescoring scheme which outperforms all systems evaluated in Stanovsky et al (2019) on WinoMT with no degradation of general test set BLEU, and we show this scheme can be applied to remove gender bias in the output of `black box` online commercial MT systems. We demonstrate our approach translating from English into three languages with varied linguistic properties and data availability.

In line 9, lattice-rescoring with the converged model of line 5 limits BLEU degradation to 0.2 BLEU on all languages, while maintaining 85%, 82% and 58% of the WinoMT accuracy improvement from the converged model for the three language pairs. Lattice rescoring with this model gives accuracy improvements over the baseline of 36%, 38% and 24% for en-de, en-es and en-he. Rescoring en-he maintains a much smaller proportion of WinoMT accuracy improvement than en-de and en-es. We believe this is because the en-he baseline is particularly weak, due to a small and non-diverse training set. The baseline must produce some inflection of the correct entity before lattice rescoring can have an effect on gender bias.

Experiments ::: Results ::: Counterfactual adaptation	Table TABREF37 compares our baseline model with the results of unregularised fine-tuning on the counterfactual sets described in Section SECREF10. Fine-tuning for one epoch on original, a subset of the original data with gendered English stopwords, gives slight improvement in WinoMT accuracy and $\Delta G$ for all language pairs, while $\Delta S$ worsens. We suggest this set consolidates examples present in the full dataset, improving performance on gendered entities generally but emphasizing stereotypical roles. On the FTrans original set $\Delta G$ increases sharply relative to the original set, while $\Delta S$ decreases. We suspect this set suffers from bias amplification  introduced by the baseline system during forward-translation. The model therefore over-predicts male entities even more heavily than we would expect given the gender makeup of the adaptation data's source side. Over-predicting male entities lowers $\Delta S$ artificially. Adapting to FTrans swapped increases accuracy and decreases both $\Delta G$ and $\Delta S$ relative to the baseline for en-de and en-es.

The latter resulted in higher BLEU and faster training.

This is the desired result, but not a particularly strong one, and it is not replicated for en-he. The balanced set has a very similar effect to the FTrans swapped set, with a smaller test BLEU difference from the baseline. One consistent result from Table TABREF37 is the largest improvement in WinoMT accuracy corresponding to the model predicting male and female entities in the closest ratio. However, the best ratios for models adapted to these datasets are 2:1 or higher, and the accuracy improvement is small. The purpose of EWC regularization is to avoid catastrophic forgetting of general translation ability. This does not occur in the counterfactual experiments, so we do not apply EWC. Moreover, WinoMT accuracy gains are small with standard fine-tuning, which allows maximum adaptation: we suspect EWC would prevent any improvements. Overall, improvements from fine-tuning on counterfactual datasets (FTrans swapped and balanced) are present. However, they are not very different from the improvements when fine-tuning on equivalent non-counterfactual sets (original and FTrans original). Improvements are also inconsistent across language pairs.

Experiments ::: Results ::: Handcrafted profession set adaptation	Results for fine-tuning on the handcrafted set are given in lines 3-6 of Table TABREF40. These experiments take place in minutes on a single GPU, compared to several hours when fine-tuning on the counterfactual sets and far longer if training from scratch. Fine-tuning on the handcrafted sets gives a much faster BLEU drop than fine-tuning on counterfactual sets. This is unsurprising since the handcrafted sets are domains of new sentences with consistent sentence length and structure. By contrast the counterfactual sets are less repetitive and close to subsets of the original training data, slowing forgetting. We believe the degradation here is limited only by the ease of fitting the small handcrafted sets. Line 4 of Table TABREF40 adapts to the handcrafted set, stopping when validation BLEU degrades by 5% on each language pair. This gives a WinoMT accuracy up to 19 points above the baseline, far more improvement than the best counterfactual result. Difference in gender score $\Delta G$ improves by at least a factor of 4.

With EWC regularization tuned to balance validation BLEU and WinoMT accuracy, the decrease is limited to about 0.5 BLEU on each language pair. Adapting to convergence, as in line 5, would lead to further WinoMT gains at the expense of BLEU.

In practice this translation is via an MT first-pass for speed, followed by manual checking, but given available lexicons this could be further automated. We note that the handcrafted sets contain no examples of coreference resolution and very little variety in terms of grammatical gender. A set of more complex sentences targeted at the coreference task might further improve WinoMT scores, but would be more difficult to produce for new languages. We wish to distinguish between a model which improves gender translation, and one which improves its WinoMT scores simply by learning the vocabulary for previously unseen or uncommon professions. We therefore create a handcrafted no-overlap set, removing source sentences with professions occurring in WinoMT to leave 216 sentences. We increase this set back to 388 examples with balanced adjective-based sentences in the same pattern, e.g. The tall $[$man$|$woman$]$ finished $[$his$|$her$]$ work.

Question: What metrics are used to measure bias reduction?
Answer: Accuracy, $\mathbf {\Delta G}$, $\mathbf {\Delta S}$, BLEU