SlowGuess/ACL_2020_OCR huggingface dataset:
Department of Engineering, University of Cambridge, UK
{ds636，wjb31}@cam.ac.uk
Training data for NLP tasks often exhibits gender bias in that fewer sentences refer to women than to men. In Neural Machine Translation (NMT) gender bias has been shown to reduce translation quality, particularly when the target language has grammatical gender. The recent WinoMT challenge set allows us to measure this effect directly (Stanovsky et al., 2019).
Ideally we would reduce system bias by simply debiasing all data prior to training, but achieving this effectively is itself a challenge. Rather than attempt to create a 'balanced' dataset, we use transfer learning on a small set of trusted, gender-balanced examples. This approach gives strong and consistent improvements in gender debiasing with much less computational cost than training from scratch.
A known pitfall of transfer learning on new domains is 'catastrophic forgetting', which we address both in adaptation and in inference. During adaptation we show that Elastic Weight Consolidation allows a performance trade-off between general translation quality and bias reduction. During inference we propose a lattice-rescoring scheme which outperforms all systems evaluated in Stanovsky et al. (2019) on WinoMT with no degradation of general test set BLEU, and we show this scheme can be applied to remove gender bias in the output of 'black box' online commercial MT systems. We demonstrate our approach translating from English into three languages with varied linguistic properties and data availability.
As language processing tools become more prevalent concern has grown over their susceptibility to social biases and their potential to propagate bias (Hovy and Spruit, 2016; Sun et al., 2019). Natural language training data inevitably reflects biases present in our society. For example, gender bias
manifests itself in training data which features more examples of men than of women. Tools trained on such data will then exhibit or even amplify the biases (Zhao et al., 2017).
Gender bias is a particularly important problem for Neural Machine Translation (NMT) into gender-inflected languages. An over-prevalence of some gendered forms in the training data leads to translations with identifiable errors (Stanovsky et al., 2019). Translations are better for sentences involving men and for sentences containing stereotypical gender roles. For example, mentions of male doctors are more reliably translated than those of male nurses (Sun et al., 2019; Prates et al., 2019).
Recent approaches to the bias problem in NLP have involved training from scratch on artificially gender-balanced versions of the original dataset (Zhao et al., 2018; Zmigrod et al., 2019) or with debiased embeddings (Escudé Font and Costa-jussà, 2019; Bolukbasi et al., 2016). While these approaches may be effective, training from scratch is inefficient and gender-balancing embeddings or large parallel datasets are challenging problems (Gonen and Goldberg, 2019).
Instead we propose treating gender debiasing as a domain adaptation problem, since NMT models can very quickly adapt to a new domain (Freitag and Al-Onaizan, 2016). To the best of our knowledge this work is the first to attempt NMT bias reduction by fine-tuning, rather than retraining. We consider three aspects of this adaptation problem: creating less biased adaptation data, parameter adaptation using this data, and inference with the debiased models produced by adaptation.
Regarding data, we suggest that a small, trusted gender-balanced set could allow more efficient and effective gender debiasing than a larger, noisier set. To explore this we create a tiny, handcrafted profession-based dataset for transfer learning. For contrast, we also consider fine-tuning on a coun
7724
Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 7724-7736 July 5 - 10, 2020. ©2020 Association for Computational Linguistics
terfactual subset of the full dataset and propose a straightforward scheme for artificially genderbalancing parallel text for NMT.
We find that during domain adaptation improvement on the gender-debiased domain comes at the expense of translation quality due to catastrophic forgetting (French, 1999). We can balance improvement and forgetting with a regularised training procedure, Elastic Weight Consolidation (EWC), or in inference by a two-step lattice rescoring procedure.
We experiment with three language pairs, assessing the impact of debiasing on general domain BLEU and on the WinoMT challenge set (Stanovsky et al., 2019). We find that continued training on the handcrafted set gives far stronger and more consistent improvements in gender-debiasing with orders of magnitude less training time, although as expected general translation performance as measured by BLEU decreases.
We further show that regularised adaptation with EWC can reduce bias while limiting degradation in general translation quality. We also present a lattice rescoring procedure in which initial hypotheses produced by the biased baseline system are transduced to create gender-inflected search spaces which can be rescored by the adapted model. We believe this approach, rescoring with models targeted to remove bias, is novel in NMT. The rescoring procedure improves WinoMT accuracy by up to  $30\%$  with no decrease in BLEU on the general test set.
Recent recommendations for ethics in Artificial Intelligence have suggested that social biases or imbalances in a dataset be addressed prior to model training (HLEG, 2019). This recommendation presupposes that the source of bias in a dataset is both obvious and easily adjusted. We show that debiasing a full NMT dataset is difficult, and suggest alternative efficient and effective approaches for debiasing a model after it is trained. This avoids the need to identify and remove all possible biases prior to training, and has the added benefit of preserving privacy, since no access to the original data or knowledge of its contents is required. As evidence, in section 3.4.5, we show this scheme can be applied to remove gender bias in the output of 'black box' online commercial MT systems.
Vanmassenhove et al. (2018) treat gender as a domain for machine translation, training from scratch by augmenting Europarl data with a tag indicat
ing the speaker's gender. This does not inherently remove gender bias from the system but allows control over the translation hypothesis gender. Moryossef et al. (2019) similarly prepend a short phrase at inference time which acts as a gender domain label for the entire sentence. These approaches are not directly applicable to text which may have more than one gendered entity per sentence, as in coreference resolution tasks.
Escudé Font and Costa-jussà (2019) train NMT models from scratch with debiased word embeddings. They demonstrate improved performance on an English-Spanish occupations task with a single profession and pronoun per sentence. We assess our fine-tuning approaches on the WinoMT coreference set, with two entities to resolve per sentence.
For monolingual NLP tasks a typical approach is gender debiasing using counterfactual data augmentation where for each gendered sentence in the data a gender-swapped equivalent is added. Zhao et al. (2018) show improvement in coreference resolution for English using counterfactual data. Zmigrod et al. (2019) demonstrate a more complicated scheme for gender-inflected languages. However, their system focuses on words in isolation, and is difficult to apply to co-reference and conjunction situations with more than one term to swap, reducing its practicality for large MT datasets.
Recent work recognizes that NMT can be adapted to domains with desired attributes using small datasets (Farajian et al., 2017; Michel and Neubig, 2018). Our choice of a small, trusted dataset for adaptation specifically to a debiased domain connects also to recent work in data selection by Wang et al. (2018), in which fine-tuning on less noisy data reduces translation noise. Similarly we propose fine-tuning on less biased data to reduce gender bias in translations. This is loosely the inverse of the approach described by Park et al. (2018) for monolingual abusive language detection, which pre-trains on a larger, less biased set.
2 Gender bias in machine translation
We focus on translating coreference sentences containing professions as a representative subset of the gender bias problem. This follows much recent work on NLP gender bias (Rudinger et al., 2018; Zhao et al., 2018; Zmigrod et al., 2019) including the release of WinoMT, a relevant challenge set for NMT (Stanovsky et al., 2019).
A sentence that highlights gender bias is:
7725
The doctor told the nurse that she had been busy.
A human translator carrying out coreference resolution would infer that 'she' refers to the doctor, and correctly translate the entity to German as Die Arztin. An NMT model trained on a biased dataset in which most doctors are male might incorrectly default to the masculine form, Der Arzt.
Data bias does not just affect translations of the stereotyped roles. Since NMT inference is usually left-to-right, a mistranslation can lead to further, more obvious mistakes later in the translation. For example, our baseline end-de system translates the English sentence
The cleaner hates the developer because she always leaves the room dirty.
to the German
Der Reiniger haft den Entwickler, weil er den Raum immer schmutzig{lsst.
Here not only is 'developer' mistranslated as the masculine den Entwickler instead of the feminine die Entwicklerin, but an unambiguous pronoun translation later in the sentence is incorrect: er ('he') is produced instead of sie ('she').
In practice, not all translations with gender-inflected words can be unambiguously resolved. A simple example is:
The doctor had been busy.
This would likely be translated with a masculine entity according to the conventions of a language, unless extra-sentential context was available. As well, some languages have adopted gender-neutral singular pronouns and profession terms, both to include non-binary people and to avoid the social biases of gendered language (Misersky et al., 2019). However, the target languages supported by WinoMT lack widely-accepted non-binary inflection conventions (Ackerman, 2019). This paper addresses gender bias that can be resolved at the sentence level and evaluated with existing test sets, and does not address these broader challenges.
2.1 WinoMT challenge set and metrics
WinoMT (Stanovsky et al., 2019) is a recently proposed challenge set for gender bias in NMT. Moreover it is the only significant challenge set we are aware of to evaluate translation gender bias comparably across several language pairs. It permits automatic bias evaluation for translation from English to eight target languages with grammatical gender. The source side of WinoMT is 3888 concatenated sentences from Winogender (Rudinger et al., 2018)
and WinoBias (Zhao et al., 2018). These are coreference resolution datasets in which each sentence contains a primary entity which is co-referent with a pronoun – the doctor in the first example above and the developer in the second – and a secondary entity – the nurse and the cleaner respectively.
WinoMT evaluation extracts the grammatical gender of the primary entity from each translation hypothesis by automatic word alignment followed by morphological analysis. WinoMT then compares the translated primary entity with the gold gender, with the objective being a correctly gendered translation. The authors emphasise the following metrics over the challenge set:
- Accuracy - percentage of hypotheses with the correctly gendered primary entity.
-  $\Delta \mathbf{G}$  - difference in  $F_{1}$  score between the set of sentences with masculine entities and the set with feminine entities.
-  $\Delta \mathbf{S}$  - difference in accuracy between the set of sentences with pro-stereotypical ('pro') entities and those with anti-stereotypical ('anti') entities, as determined by Zhao et al. (2018) using US labour statistics. For example, the 'pro' set contains male doctors and female nurses, while 'anti' contains female doctors and male nurses.
Our main objective is increasing accuracy. We also report on  $\Delta G$  and  $\Delta S$  for ease of comparison to previous work. Ideally the absolute values of  $\Delta G$  and  $\Delta S$  should be close to 0. A high positive  $\Delta G$  indicates that a model translates male entities better, while a high positive  $\Delta S$  indicates that a model stereotypes male and female entities. Large negative values for  $\Delta G$  and  $\Delta S$ , indicating a bias towards female or anti-stereotypical translation, are as undesirable as large positive values.
We note that  $\Delta S$  can be significantly skewed by low-accuracy systems. A model generating male forms for most test sentences, stereotypical roles or not, will have very low  $\Delta S$ , since its pro- and anti-stereotypical class accuracy will both be about  $50\%$ . Consequently in Appendix A we report:
- M:F - ratio of hypotheses with male predictions to those with female predictions.
This should be close to 1.0, since WinoMT balances male- and female-labelled sentences. M:F correlates strongly with  $\Delta G$ , but we consider M:F
7726
easier to interpret, particularly since very high or low M:F reduce the relevance of  $\Delta S$ .
Finally, we wish to reduce gender bias without reducing translation performance. We report BLEU (Papineni et al., 2002) on separate, general test sets for each language pair. WinoMT is designed to work without target language references, and so it is not possible to measure translation performance on this set by measures such as BLEU.
2.2 Gender debiased datasets
2.2.1 Handcrafted profession dataset
Our hypothesis is that the absence of gender bias can be treated as a small domain for the purposes of NMT model adaptation. In this case a well-formed small dataset may give better results than attempts at debiasing the entire original dataset.
We therefore construct a tiny, trivial set of gender-balanced English sentences which we can easily translate into each target language. The sentences follow the template:
The [PROFESSION] finished [his/her] work.
We refer to this as the *handcrafted set*<sup>1</sup>. Each profession is from the list collected by Prates et al. (2019) from US labour statistics. We simplify this list by removing field-specific adjectives. For example, we have a single profession 'engineer', as opposed to specifying industrial engineer, locomotive engineer, etc. In total we select 194 professions, giving just 388 sentences in a gender-balanced set.
With manually translated masculine and feminine templates, we simply translate the masculine and feminine forms of each listed profession for each target language. In practice this translation is via an MT first-pass for speed, followed by manual checking, but given available lexicons this could be further automated. We note that the handcrafted sets contain no examples of coreference resolution and very little variety in terms of grammatical gender. A set of more complex sentences targeted at the coreference task might further improve WinoMT scores, but would be more difficult to produce for new languages.
We wish to distinguish between a model which improves gender translation, and one which improves its WinoMT scores simply by learning the vocabulary for previously unseen or uncommon professions. We therefore create a handcrafted no-overlap set, removing source sentences with profes
sions occurring in WinoMT to leave 216 sentences. We increase this set back to 388 examples with balanced adjective-based sentences in the same pattern, e.g. The tall [man|woman] finished [his|her] work.
2.2.2 Counterfactual datasets
Figure 1: Generating counterfactual datasets for adaptation. The Original set is  $1||2$ , a simple subset of the full dataset. FTrans original is  $1||3$ , FTrans swapped is  $4||5$ , and Balanced is  $1,4||2,5$
For contrast, we fine-tune on an approximated counterfactual dataset. Counterfactual data augmentation is an intuitive solution to bias from data over-representation (Lu et al., 2018). It involves identifying the subset of sentences containing bias – in this case gendered terms – and, for each one, adding an equivalent sentence with the bias reversed – in this case a gender-swapped version.
While counterfactual data augmentation is relatively simple for sentences in English, the process for inflected languages is challenging, involving identifying and updating words that are co-referent with all gendered entities in a sentence. Gender-swapping MT training data additionally requires that the same entities are swapped in the corresponding parallel sentence. A robust scheme for gender-swapping multiple entities in inflected language sentences directly, together with corresponding parallel text, is beyond the scope of this paper. Instead we suggest a rough but straightforward approach for counterfactual data augmentation for NMT which to the best of our knowledge is the first application to parallel sentences.
We first perform simple gender-swapping on the subset of the English source sentences with gendered terms. We use the approach described in Zhao et al. (2018) which swaps a fixed list of gen
Handcrafted sets available at https://github.com/DCSaunders/gender-debias
7727
dered stopwords (e.g. man / woman, he / she).². We then greedily forward-translate the gender-swapped English sentences with a baseline NMT model trained on the full source and target text, producing gender-swapped target language sentences.
This lets us compare four related sets for gender debiasing adaptation, as illustrated in Figure 1:
- Original: a subset of parallel sentences from the original training data where the source sentence contains gendered stopwords.
- Forward-translated (FTrans) original: the source side of the original set with forward-translated target sentences.
- Forward-translated (FTrans) swapped: the original source sentences are gender-swapped, then forward-translated to produce gender-swapped target sentences.
- Balanced: the concatenation of the original and FTrans swapped parallel datasets. This is twice the size of the other counterfactual sets.
Comparing performance in adaptation of FTrans swapped and FTrans original lets us distinguish between the effects of gender-swapping and of obtaining target sentences from forward-translation.
2.3 Debiasing while maintaining general translation performance
Fine-tuning a converged neural network on data from a distinct domain typically leads to catastrophic forgetting of the original domain (French, 1999). We wish to adapt to the gender-balanced domain without losing general translation performance. This is a particular problem when fine-tuning on the very small and distinct handcrafted adaptation sets.
2.3.1 Regularized training
Regularized training is a well-established approach for minimizing catastrophic forgetting during domain adaptation of machine translation (Barone et al., 2017). One effective form is Elastic Weight Consolidation (EWC) (Kirkpatrick et al., 2017) which in NMT has been shown to maintain or even improve original domain performance (Thompson et al., 2019; Saunders et al., 2019). In EWC a
regularization term is added to the original log likelihood loss function  $L$  when training the debiased model (DB):
$$
L ^ {\prime} \left(\theta^ {D B}\right) = L \left(\theta^ {D B}\right) + \lambda \sum_ {j} F _ {j} \left(\theta_ {j} ^ {D B} - \theta_ {j} ^ {B}\right) ^ {2} (1)
$$
$\theta_{j}^{B}$  are the converged parameters of the original biased model, and  $\theta_{j}^{DB}$  are the current debiased model parameters.  $F_{j} = \mathbb{E}\big[\nabla^{2}L(\theta_{j}^{B})\big]$ , a Fisher information estimate over samples from the biased data under the biased model. We apply EWC when performance on the original validation set drops, selecting hyperparameter  $\lambda$  via validation set BLEU.
2.3.2 Gender-inflected search spaces for rescoring with debiased models
Figure 2: Finite State Transducers for lattice rescoring.
An alternative approach for avoiding catastrophic forgetting takes inspiration from lattice rescoring for NMT (Stahlberg et al., 2016) and Grammatical Error Correction (Stahlberg et al., 2019). We assume we have two NMT models. With one we decode fluent translations which contain gender bias  $(B)$ . For the one-best hypothesis we would translate:
$$
\mathbf {y} _ {\mathbf {B}} = \operatorname {a r g m a x} _ {\mathbf {y}} p _ {B} (\mathbf {y} | \mathbf {x}) \tag {2}
$$
The other model has undergone debiasing  $(DB)$  at a cost to translation performance, producing:
$$
\mathbf {y} _ {\mathbf {D B}} = \operatorname {a r g m a x} _ {\mathbf {y}} p _ {D B} (\mathbf {y} | \mathbf {x}) \tag {3}
$$
2The stopwords list and swapping script are provided by the authors of Zhao et al. (2018) at https://github.com/uclanlp/corefBias
7728
We construct a flower transducer  $T$  that maps each word in the target language's vocabulary to itself, as well as to other forms of the same word with different gender inflections (Figure 2a). We also construct  $Y_{B}$ , a lattice with one path representing the biased but fluent hypothesis  $\mathbf{y}_{\mathbf{B}}$  (Figure 2b).
The acceptor  $\mathcal{P}(\mathbf{y}_{\mathbf{B}}) = \mathrm{proj}_{\mathrm{output}}(Y_B\circ T)$  defines a language consisting of all the gender-inflected versions of the biased first-pass translation  $\mathbf{y}_{\mathbf{B}}$  that are allowed by  $T$  (Figure 2c). We can now decode with lattice rescoring  $(LR)$  by constraining inference to  $\mathcal{P}(\mathbf{y}_{\mathbf{B}})$ :
$$
\mathbf {y} _ {\mathbf {L R}} = \operatorname {a r g m a x} _ {\mathbf {y} \in \mathcal {P} (\mathbf {y} _ {\mathbf {B}})} p _ {D B} (\mathbf {y} | \mathbf {x}) \tag {4}
$$
In practice we use beam search to decode the various hypotheses, and construct  $T$  using heuristics on large vocabulary lists for each target language.
3.1 Languages and data
WinoMT provides an evaluation framework for translation from English to eight diverse languages. We select three pairs for experiments: English to German (en-de), English to Spanish (en-es) and English to Hebrew (en-he). Our selection covers three language groups with varying linguistic properties: Germanic, Romance and Semitic. Training data available for each language pair also varies in quantity and quality. We filter training data based on parallel sentence lengths and length ratios.
For en-de, we use 17.6M sentence pairs from WMT19 news task datasets (Barrault et al., 2019). We validate on newstest17 and test on newstest18.
For en-es we use 10M sentence pairs from the United Nations Parallel Corpus (Ziemski et al., 2016). While still a large set, the UNCorpus exhibits far less diversity than the en-de training data. We validate on newstest12 and test on newstest13.
For en-he we use 185K sentence pairs from the multilingual TED talks corpus (Cettolo et al., 2014). This is both a specialized domain and a much smaller training set. We validate on the IWSLT 2012 test set and test on IWSLT 2014.
Table 1 summarises the sizes of datasets used, including their proportion of gendered sentences and ratio of sentences in the English source data containing male and female stopwords. A gendered sentence contains at least one English gendered stopwords as used by Zhao et al. (2018).
Interestingly all three datasets have about the same proportion of gendered sentences:  $11 - 12\%$  of
the overall set. While en-es appears to have a much more balanced gender ratio than the other pairs, examining the data shows this stems largely from sections of the UNCorpus containing phrases like 'empower women' and 'violence against women', rather than gender-balanced professional entities.
Training	Gendered training	M:F	Test
en-de	17.5M	2.1M	2.4	3K
en-es	10M	1.1M	1.1	3K
en-he	185K	21.4K	1.8	1K
Table 1: Parallel sentence counts. A gendered sentence pair has minimum one gendered stopwords on the English side. M:F is ratio of male vs female gendered training sentences.