Iterative Domain-Repaired Back-Translation (2020)

Abstract: In this paper, we focus on the domain-specific translation with low resources, where in-domain parallel corpora are scarce or nonexistent. One common and effective strategy for this case is exploiting in-domain monolingual data with the back-translation method. However, the synthetic parallel data is very noisy because they are generated by imperfect out-of-domain systems, resulting in the poor performance of domain adaptation. To address this issue, we propose a novel iterative domain-repaired back-translation framework, which introduces the Domain-Repair (DR) model to refine translations in synthetic bilingual data. To this end, we construct corresponding data for the DR model training by round-trip translating the monolingual sentences, and then design the unified training framework to optimize paired DR and NMT models jointly. Experiments on adapting NMT models between specific domains and from the general domain to specific domains demonstrate the effectiveness of our proposed approach, achieving 15.79 and 4.47 BLEU improvements on average over unadapted models and back-translation.

## introduction neural machine translation (nmt) has achieved impressive performance when large amounts of parallel sentences are available @xcite @xcite . however, some previous works have shown that nmt models perform poorly in specific domains, especially when they are trained on the corpora from very distinct domains @xcite . the fine-tuning method @xcite is a popular way to mitigate the 1 our code is released in https://github.com/whr94621/ iterative-domain-repaired-back-translation effect of domain drift. however, it is not realistic to collect large amounts of high-quality parallel data in every domain we are interested in. since monolingual in-domain data are usually abundant and easy to obtain, it is essential to explore the unsupervised domain adaptation scenario that utilizes large amounts of out-of-domain bilingual data and in-domain monolingual data. one straightforward and effective solution for unsupervised domain adaptation is to build indomain synthetic parallel data, including copying monolingual target sentences to the source side @xcite or back-translation of in-domain monolingual target sentences @xcite . although the backtranslation approach has proven the superior effectiveness in exploiting monolingual data, directly applying this method in this scenario brings lowquality in-domain synthetic data. in this paper, we extend back-translation by a domain-repair (dr) model to explicitly remedy this issue. specifically, the dr model is designed to re-generate in-domain source sentences given the synthetic data. in this way, the pseudo parallel data's source side can be re-written with the in-domain style, and some wrong translations are fixed. to optimize the dr model, we use the roundtrip translation of monolingual source sentences to construct the corresponding training data. since source monolingual data is involved, it is natural to extend the back-translation method to bidirectional setting @xcite , which src: eine gewichtszunahme wurde nach markteinführung bei patienten berichtet , denen abil-ify verschrieben wurde . ref: weight gain has been reported post-marketing among patients prescribed abilify . mt: a weight gain has been reported :::: after :::::: market :::::::::: introduction in patients who have been prescribed abilify . src: es werden möglicherweise nicht alle packungsgrößen in den verkehr gebracht . ref: not all pack sizes may be marketed. ## iterative domain-repaired back-translation in this section, we first illustrate the overview of iter-drbt framework, then describe the architecture of dr model and the joint training strategy. ## conclusion in this paper, we argue that back-translation, the predominant unsupervised domain adaptation method in neural machine translation, suffers from the domain shift, restricting the performance of unsupervised domain adaptation. we propose to remedy this mismatch by leveraging a domain repair model that corrects the errors in back-translation sentences. then the iterative domain-repaired back-translation framework is designed to make full use of the advantage of the domain repair model. experiments on adapting translation models between specific domains and from general domain to specific domains demonstrate the effectiveness of our method, achieving significant improvements over strong back-translation baselines. in the future, we would like to extend our method to enhance the back-translation method in multidomain settings.