State-of-the-art neural machine translation (NMT) systems are data-hungry and perform poorly on domains with little supervised data. As data collection is expensive and infeasible in many cases, unsupervised domain adaptation methods are needed. We apply an Iterative Back Translation (IBT) training scheme on in-domain monolingual data, which repeatedly uses a Transformer-based NMT model to create in-domain pseudo-parallel sentence pairs in one translation direction on the fly and then use them to train the model in the other direction. Evaluated on three domains of German-to-English translation task with no supervised data, this simple technique alone (without any out-of-domain parallel data) can already surpass all previous domain adaptation methods---up to + 9.48 BLEU over the strongest previous method , and up to +27.77 BLEU over the unadapted baseline. Moreover, given available supervised out-of-domain data on German-to-English and Romanian-to-English language pairs, we can further enhance the performance and obtain up to + 19.31 BLEU improvement over the strongest baseline, and +47.69 BLEU increment against the unadapted model .
State-of-the-art neural machine translation (NMT) systems are data-hungry and perform poorly on new domains with no supervised data. As data collection is expensive and infeasible in many cases, domain adaptation methods are needed. In this work, we propose a simple but effect approach to the semi-supervised domain adaptation scenario of NMT, where the aim is to improve the performance of a translation model on the target domain consisting of only non-parallel data with the help of supervised source domain data. This approach iteratively trains a Transformer-based NMT model via three training objectives: language modeling, back-translation, and supervised translation. We evaluate this method on two adaptation settings: adaptation between specific domains and adaptation from a general domain to specific domains, and on two language pairs: German to English and Romanian to English. With substantial performance improvement achieved---up to + 19.31 BLEU over the strongest baseline, and +47.69 BLEU improvement over the unadapted model---we present this method as a simple but tough-to-beat baseline in the field of semi-supervised domain adaptation for NMT .
R
domains with little
new domains with no
D
unsupervised
R
We apply an Iterative Back Translation (IBT) training scheme on in-domain monolingual data, which repeatedly uses a
In this work, we propose a simple but effect approach to the semi-supervised domain adaptation scenario of NMT, where the aim is to improve the performance of a translation model on the target domain consisting of only non-parallel data with the help of supervised source domain data. This approach iteratively trains a
R
NMT model to create in-domain pseudo-parallel sentence pairs in one translation direction on the fly and then use them to train the model in the other direction. Evaluated on three domains of German-to-English translation task with no supervised data, this simple technique alone (without any out-of-domain parallel data) can already surpass all previous domain adaptation methods---up
NMT model via three training objectives: language modeling, back-translation, and supervised translation. We evaluate this method on two adaptation settings: adaptation between specific domains and adaptation from a general domain to specific domains, and on two language pairs: German to English and Romanian to English. With substantial performance improvement achieved---up
D
9.48 BLEU over the strongest previous method , and up to +27.77 BLEU over the unadapted baseline. Moreover, given available supervised out-of-domain data on German-to-English and Romanian-to-English language pairs, we can further enhance the performance and obtain up to +
D
improvement
R
increment against the unadapted model
improvement over the unadapted model---we present this method as a simple but tough-to-beat baseline in the field of semi-supervised domain adaptation for NMT