Cross-model Back-translated Distillation for Unsupervised Machine Translation

1 INTRODUCTION . Machine translation ( MT ) is a core task in natural language processing that involves both language understanding and generation . Recent neural approaches ( Vaswani et al. , 2017 ; Wu et al. , 2019 ) have advanced the state of the art with near human-level performance ( Hassan et al. , 2018 ) . However , they continue to rely heavily on large parallel data . As a result , the search for unsupervised alternatives using only monolingual data has been active . While Ravi & Knight ( 2011 ) and Klementiev et al . ( 2012 ) proposed various unsupervised techniques for statistical MT ( SMT ) , Lample et al . ( 2018a ; c ) established a general framework for modern unsupervised MT ( UMT ) that works for both SMT and neural MT ( NMT ) models . The framework has three main principles : model initialization , language modeling and iterative back-translation . Model initialization bootstraps the model with a knowledge prior like word-level transfer ( Lample et al. , 2018b ) . Language modeling , which takes the form of denoising auto-encoding ( DAE ) in NMT ( Lample et al. , 2018c ) , trains the model to generate plausible sentences in a language . Meanwhile , iterative back-translation ( IBT ) facilitates cross-lingual translation training by generating noisy source sentences for original target sentences . The recent approaches differ on how they apply each of these three principles . For instance , Lample et al . ( 2018a ) use an unsupervised word-translation model ( Lample et al. , 2018b ) for model initialization , while Conneau & Lample ( 2019 ) use a pretrained cross-lingual masked language model ( XLM ) . In this paper , we focus on a different aspect of the UMT framework , namely , its data diversification2 process . If we look from this view , the DAE and IBT steps of the UMT framework also perform some form of data diversification to train the model . Specifically , the noise model in the DAE process generates new , but noised , versions of the input data , which are used to train the model with a reconstruction objective . Likewise , the IBT step involves the same UMT model to create synthetic parallel pairs ( with the source being synthetic ) , which are then used to train the model . Since the NMT model is updated with DAE and IBT simultaneously , the model generates fresh translations in each back-translation step . Overall , thanks to DAE and IBT , the model gets better at translating 1Anonymized code : https : //tinyurl.com/y2ru8res . 2By diversification , we mean sentence level variations ( not expanding to other topics or genres ) . by iteratively training on the newly created and diversified data whose quality also improves over time . This argument also applies to statistical UMT , except for the lack of the DAE ( Lample et al. , 2018c ) . However , we conjecture that these diversification methods may have reached their limit as the performance does not improve further the longer we train the UMT models . In this work , we introduce a fourth principle to the standard UMT framework : Cross-model Backtranslated Distillation or CBD ( §3 ) , with the aim to induce another level of diversification that the existing UMT principles lack . CBD initially trains two UMT agents ( models ) using existing approaches . Then , one of the two agents translates the monolingual data from one language s to another t in the first level . In the second level , the generated data are back-translated from t to s by the other agent . In the final step , the synthetic parallel data created by the first and second levels are used to distill a supervised MT model . CBD is applicable to any existing UMT method and is more efficient than ensembling approaches ( Freitag et al. , 2017 ) ( §5.3 ) . In the experiments ( §4 ) , we have evaluated CBD on the WMT ’ 14 English-French , WMT ’ 16 English-German and WMT ’ 16 English-Romanian unsupervised translation tasks . CBD shows consistent improvements of 1.0-2.0 BLEU compared to the baselines in these tasks . It also boosts the performance on IWSLT ’ 14 English-German and IWSLT ’ 13 English-French tasks significantly . In our analysis , we explain with experiments why other similar variants ( §5.1 ) and other alternatives from the literature ( §5.4 ) do not work well and cross-model back-translation is crucial for our method . We further demonstrate that CBD enhances the baselines by achieving greater diversity as measured by back-translation BLEU ( §5.2 ) . 2 BACKGROUND . Ravi & Knight ( 2011 ) were among the first to propose a UMT system by framing the problem as a decipherment task that considers non-English text as a cipher for English . Nonetheless , the method is limited and may not be applicable to the current well-established NMT systems ( Luong et al. , 2015 ; Vaswani et al. , 2017 ; Wu et al. , 2019 ) . Lample et al . ( 2018a ) set the foundation for modern UMT . They propose to maintain two encoder-decoder networks simultaneously for both source and target languages , and train them via denoising auto-encoding , iterative back-translation and adversarial training . In their follow-up work , Lample et al . ( 2018c ) formulate a common UMT framework for both PBSMT and NMT with three basic principles that can be customized . The three principles are : • Initialization : A non-randomized cross- or multi-lingual initialization that represents a knowledge prior to bootstrap the UMT model . For instance , Lample et al . ( 2018a ) and Artetxe et al . ( 2019 ) use an unsupervised word-translation model MUSE ( Lample et al. , 2018b ) as initialization to promote word-to-word cross-lingual transfer . Lample et al . ( 2018c ) use a shared jointly trained sub-word ( Sennrich et al. , 2016b ) dictionary . On the other hand , Conneau & Lample ( 2019 ) use a pretrained cross-lingual masked language model ( XLM ) to initialize the unsupervised NMT model . • Language modeling : Training a language model on monolingual data helps the UMT model to generate fluent texts . The neural UMT approaches ( Lample et al. , 2018a ; c ; Conneau & Lample , 2019 ) use denoising auto-encoder training to achieve language modeling effects in the neural model . Meanwhile , the PBSMT variant proposed by Lample et al . ( 2018c ) uses the KenLM smoothed n-gram language models ( Heafield , 2011 ) . • Iterative back-translation : Back-translation ( Sennrich et al. , 2016a ) brings about the bridge between source and target languages by using a backward model that translates data from target to source . The ( source and target ) monolingual data is translated back and forth iteratively to progress the UMT model in both directions . During training , the initialization step is conducted once , while the denoising and back-translation steps are often executed in an alternating manner.3 It is worth noting that depending on different implementations , the parameters for backward and forward components may be separate ( Lample et al. , 2018a ) or shared ( Lample et al. , 2018c ; Conneau & Lample , 2019 ) . A parameter-shared cross-lingual NMT model has the capability to translate from either source or target , while a UMT system with parameter-separate models has to maintain two models . Either way , we deem a standard UMT system to be bidirectional , i.e . it is capable of translating from either source or target language . 3The KenLM language model in PBSMT ( Lample et al. , 2018c ) was kept fixed during the training process . Our proposed cross-model back-translated distillation ( CBD ) works outside this well-established framework . It employs two UMT agents to create extra diversified data apart from what existing methods already offer , rendering it a useful add-on to the general UMT framework . Furthermore , different implementations of UMT as discussed above can be plugged into the CBD system to achieve a performance boost , even for future methods that may potentially not employ the three principles . 3 CROSS-MODEL BACK-TRANSLATED DISTILLATION . Let Xs and Xt be the sets of monolingual data for languages s and t , respectively . We first train two UMT agents independently with two different parameter sets θ1 and θ2 using existing methods ( Lample et al. , 2018a ; c ; Conneau & Lample , 2019 ) .4 Since a UMT agent with parameter set θ is deemed bidirectional in our setup , we denote yt ∼ P ( ·|xs , θ ) to be the translation sample from language s to t of input sentence xs using parameters θ . In addition , for each language direction s→ t ( and vice versa for t→ s ) , we introduce a supervised MT model with parameters θ̂ that will be trained through our CBD . With these notations , we denote the following training loss L with respect to input variables θα , θβ ∈ { θ1 , θ2 } , the supervised parameters θ̂ and the learning rate η . Figure 1 also illustrates the translation flows of function L. L ( θα , θβ , θ̂ ) = −Ezs∼P ( ·|yt , θβ ) , yt∼P ( ·|xs , θα ) , xs∼Xs [ logP ( yt|xs , θ̂ ) + logP ( yt|zs , θ̂ ) ] − Ezt∼P ( ·|ys , θβ ) , ys∼P ( ·|xt , θα ) , xt∼Xt [ logP ( xt|ys , θ̂ ) + logP ( zt|ys , θ̂ ) ] ( 1 ) In the CBD strategy ( Algorithm 1 , Eq . 1 ) , at each step , each agent θ ∈ { θ1 , θ2 } samples translations from the monolingual data of both languages s and t to acquire the first level of synthetic parallel data ( xs , yt ) and ( xt , ys ) . In the second level , the other agent { θ1 , θ2 } \ θ is used to sample the translation zs of the translation yt of xs ( and similarly for zt from the translation ys of xt ) . This process is basically back-translation , but with the backward model coming from a different regime than that of the forward model . The fact that the first level agent must be different from the second level agent is crucial to achieve the desirable level of diversity in data generation . After this , we update the final model θ̂ using all the synthetic pairs ( x , y ) and ( y , z ) with maximum likelihood estimation ( MLE ) . Algorithm 1 Cross-model Back-translated Distillation : Given monolingual data Xs and Xt of languages s and t , return a s→ t UMT model with parameters θ̂ . 1 : procedure CBD ( s , t ) 2 : Train the 1st UMT agent with parameters θ1 3 : Train the 2nd UMT agent with parameters θ2 4 : Randomly initialize the parameters for MT model , θ̂ 5 : while until convergence do 6 : θ̂ ← θ̂ − η∇θ̂L ( θ1 , θ2 , θ̂ ) 7 : θ̂ ← θ̂ − η∇θ̂L ( θ2 , θ1 , θ̂ ) 8 : return θ̂ In this way , the model θ̂ first gets trained on the translated products ( x − y ) of the UMT teachers , making it as capable as the teachers . Secondly , the model θ̂ is also trained on the second-level data ( y − z ) , which is slightly different from the first-level data . Thus , this mechanism provides extra data diversification to the system θ̂ in addition to what the UMT teachers already offer , resulting in our final model outperforming the UMT baselines ( §4 ) . However , one may argue that since θ1 and θ2 are trained in a similar fashion , z will be the same as x , resulting in a duplicate pair . In our experiments , on the contrary , the back-translated dataset contains only around 14 % duplicates across different language pairs , as shown in our analysis on data diversity in §5.2.In Appendix §8.1 , we provide a more generalized version of CBD with n ( ≥ 2 ) UMT agents , where we also analyze its effectiveness on the IWSLT translation tasks . 4For neural approaches , changing the random seeds would do the trick , while PBSMT methods would need to randomize the initial embeddings and/or subsample the training data .

Summary: The paper proposes an additional stage of training for unsupervised NMT models utilizing synthetic data generated from multiple independently trained models. The generated synthetic data uses two stages of back-translation, with different models, in order to "diversify" the set of training data used for fine-tuning the models. This is similar to the approach in [1], but uses an additional stage of back-translation with a different model. The authors add this additional stage of training to unsupervised NMT models using different pipelines (PB unsupervised MT, Neural Unsupervised MT, XLM) and show that their approach improves all of these approaches by 1.5-2 Bleu on WMT En-Fr, De-En and En-Ro.