Pronoun-Targeted Fine-tuning for NMT with Hybrid Losses (2020)

Abstract: Popular Neural Machine Translation model training uses strategies like backtranslation to improve BLEU scores, requiring large amounts of additional data and training. We introduce a class of conditional generative-discriminative hybrid losses that we use to fine-tune a trained machine translation model. Through a combination of targeted fine-tuning objectives and intuitive re-use of the training data the model has failed to adequately learn from, we improve the model performance of both a sentence-level and a contextual model without using any additional data. We target the improvement of pronoun translations through our fine-tuning and evaluate our models on a pronoun benchmark testset. Our sentence-level model shows a 0.5 BLEU improvement on both the WMT14 and the IWSLT13 De-En testsets, while our contextual model achieves the best results, improving from 31.81 to 32 BLEU on WMT14 De-En testset, and from 32.10 to 33.13 on the IWSLT13 De-En testset, with corresponding improvements in pronoun translation. We further show the generalizability of our method by reproducing the improvements on two additional language pairs, Fr-En and Cs-En.

## introduction the advent of neural machine translation (nmt) @xcite brought about significant improvements that left the previously successful statistical machine translation models far behind. however, the availability of large corpora has been no small part of that success, with recent nmt models using millions of sentences for training. a lack of availability of such large parallel corpora across languages has given rise to methods utilizing large amounts of monolingual data, such as for backtranslation @xcite , language modeling (c ¸aglar @xcite , or for large-scale pre-training @xcite . backtranslation @xcite is a commonly used strategy to improve mt models in the absence of adequate parallel data for training. a target-to-source model is first trained using the available parallel data, which is then used to translate a large target-monolingual corpus into the source to create pseudo-parallel data for training a source-to-target mt model. this has been shown to result in improvements in the bleu score, and has become a popular method for improving nmt models, with many recent works proposing strategies to further improve it (hoa; @xcite . however, recent studies have suggested that there is a limit beyond which the addition of synthetic data hurts the performance of the model @xcite . also, recent work @xcite point out that back-translation suffers from the translationese effect, where back-translation only improves the performance when the source sentences are translationese but does not offer any improvement when the sentences are natural text. automatic post-editing (ape) is another common strategy that is used to improve translations. ape models are commonly monolingual, and typically take the output from some mt model as input, which they then modify. in the absence of adequate human post-edited data to train data-hungry neural models, @xcite and @xcite both use round-trip translation data to train their post-editing models. in round-trip translations, target monolingual data is translated using a target-to-source model to the source text, and then back to the target using another source-to-target model. this round-trip translated text is considered an approximation of poor quality mt output, which can be used in combination with the original target reference text to train the post-editing model. @xcite train a model to make corrections in context, using groups of 4 sentences as input, and show improvements in bleu as well as translations of discourse phenomena. nmt models typically fail on rare words that may not be adequately seen during training, such as named entities, or on words whose interpretation depends on the context such as discourse phenomena @xcite . for the latter, nmt models tend to prefer a more typical alternative to a relatively rare but correct one (e.g., french "il" is often wrongly translated to the more common "it" than "he" ). however, these seemingly trivial errors can erode translation to the extent that they can be easily distinguishable from human-translated texts @xcite . there could be several reasons for why nmt models make such mistakes; our hypothesis is that since almost all nmt models are trained with a conditional language model objective, it is clear that this objective alone is proving inadequate to capture all of the information available in the text. we therefore propose a class of conditional generativediscriminative hybrid losses that explicitly teach models what to generate and what not to generate. using these specialized losses, we aim to improve the learning power of the mt model. specifically, in this work, we target the improvement of pronoun translation by focusing our finetuning efforts through our proposed objectives and also through the fine-tuning data. we aim to leverage the training data we already have by extracting a subset of targeted fine-tuning data from the training corpus that the model has failed to learn correctly from. we use the newly proposed training objectives in combination with the targeted data to help the model fully reach its learning potential on the training corpus. we attempt to improve both general translation quality and the pronoun translation without compromising on either, and to do this without any elaborate model architecture. our main contributions are as follows: • a class of conditional generative-discriminative hybrid losses that improve the learning potential of the model ( §2). ## targeted finetuning objectives before introducing our proposed conditional generative-discriminative hybrid losses for finetuning nmt models on a targeted dataset, we first describe the conditional language modeling (clm) objective used to train nmt models. ## experiments we conduct our fine-tuning experiments on the german-english (de-en) translation task. we describe our baseline training and fine-tuning corpus ( §4.1), our experiments and results on fine-tuning using only the targeted subset data ( §4.4), and finetuning using both the targeted subset data and the hybrid training losses ( §4.5). ## conclusions and future work we have proposed a class of conditional generativediscriminative losses to increase the learning potential of nmt models, showing that it is possible to leverage "unlearned" training data to further improve an mt model, by strategically filtering the data and applying additional targeted losses. we demonstrated the effectiveness of our methods on different languages and testsets, also reporting improved pronoun translations. although we focus on pronoun translations, our fine-tuning method is generic and can be used to correct other kinds of errors in machine translations, like named entities or other rare words. in future work, we will explore other such applications of our proposed methods.