Demystifying Learning of Unsupervised Neural Machine Translation

1 INTRODUCTION . Unsupervised Neural Machine Translation or UNMT have grown from its infancy ( Artetxe et al. , 2018 ; Lample et al. , 2018a ) to close-to-supervised performance recently on some translation scenarios ( Lample & Conneau , 2019 ; Song et al. , 2019 ) . Early UNMT works ( Artetxe et al. , 2017 ; Lample et al. , 2018a ; Yang et al. , 2018 ) adopt complex training strategies including model initialization , synthetic parallel data for warming up the model , adversarial loss for making encoder universal , different weight sharing mechanisms etc . Then Lample et al . ( 2018b ) simplifies all these and establishes a two-components framework , involving an initialization strategy followed by iterative training on two tasks , i.e . denoising auto-encoding with the DAE loss and online back-translation with the BT loss . Works afterwards mainly focus on developing better initialization strategies ( Lample & Conneau , 2019 ; Ren et al. , 2019 ; Song et al. , 2019 ; Liu et al. , 2020 ) . Although obtaining impressive performance , it is unclear why this standard training protocol is possible to be successful . Kim et al . ( 2020 ) and Marchisio et al . ( 2020 ) consider the standard training as a black-box and empirically analyze its success or failure regarding different data settings ( i.e . text domains and language pairs ) . Unfortunately , due to the lack of theoretical guidelines , some fundamental questions are still remained unknown : what standard training tries to minimize under the general unsupervised training paradigm ( Ghahramani , 2004 ) and when a certain training protocol can work for training UNMT ? In this paper , we attempt to open the back-box training of UNMT and understand its theoretical essence from two angles : i ) a marginal likelihood maximization view ; and ii ) an information-theoretic view by ablating standard training protocol with other variants . Our contributions are as follows . A . By making an analogy of standard training protocol with marginal likelihood or Evidence Lower BOund ( ELBO ) optimization , we visualize the learning curves of the two terms in ELBO objective , and found that optimizing ELBO is not sufficient for training a successful UNMT model , indicating that specific regularization design i.e . the DAE loss , quite matters . B . By leveraging information theory , we present a formal definition on what does it mean to successfully train an UNMT model , and then readily derive a sufficient condition and a necessary condition for successfully training UNMT in principle . In addition , we validate both sufficient and necessary conditions through empirical experiments , and find that both conditions indeed explain why standard training protocol works while others suffer from degeneration to learning sub-optimal tasks . C. Based on explanations for those failed protocols , we continue experiments to settle the role played by DAE and BT . Firstly , BT is the main task while DAE is a critical auxiliary . Then we clarify that DAE has more important role than just learning word order , accepted as common knowledge in almost all previous works , but also preserving the mutual information between encoder input and encoder output , which is necessary for successful training . Furthermore , DAE also functions as a behavior regularizer for decoding with online BT , and prevents BT from yielding degenerated data . 2 UNDERSTANDING UNMT FROM TWO PERSPECTIVES . In this section , we first introduce background about the standard training protocol proposed in Lample et al . ( 2018b ) , which is adopted by almost all later works . Then we introduce the basic concept of two perspectives on which we rely for analyzing the learning of different training protocol variants . Due to the space limit , please refer to appendix A.1 for a timely literature review of recent advance . 2.1 STANDARD TRAINING PROTOCOL . The standard training protocol involves standard initialization strategy and standard iterative training procedure , and they both are built upon a specific design of encoder-decoder parameterization . Parameterization and initialization UNMT model adopts a shared embedding matrix for a shared vocabulary with joint BPE ( Sennrich et al. , 2016 ) , and the two languages share the same encoder and decoder with only a language embedding for distinguishing the input from different languages . As a result , unconstrained decoding might generate tokens from the same language as the input . Standard initialization means using fastTEXT ( Bojanowski et al. , 2017 ) to initialize the embedding matrix , denoted as JointEmb . XLM ( Lample & Conneau , 2019 ) uses a trained encoder to initialize both encoder and decoder of the UNMT model . We also consider random initialization for completeness . Iterative training strategy The iterative training strategy involves optimization of two critical losses by turns , i.e . the DAE loss and the BT loss as defined in Eq . 1 and Eq . 2 , where s and t denote the two languages . DAE loss is constructed through sampling a monolingual sentence x ( or y ) , construct its noisy version C ( x ) ( C ( y ) ) and minimize the reconstruction error or RecErr : Ldae = − log ps→s ( x|C ( x ) ) + [ − log pt→t ( y|C ( y ) ) ] , ( 1 ) BT loss is constructed through sampling a monolingual sentence x ( or y ) , construct its corresponding translation via the current modelM ( x ) ( M ( y ) ) through back-translation and minimize the RecErr : Lbt = Eŷ∼M ( x ) [ − log pt→s ( x|ŷ ) ] + Ex̂∼M ( y ) [ − log ps→t ( y|x̂ ) ] , ( 2 ) The online BT process involved in the iterative training strategy can be seen as Co-Training ( Blum & Mitchell , 1998 ) , where two models ( with shared weights ) constructed on two views ( source/target sentence ) generate pseudo labels as the other view ( pseudo translation ) for training the corresponding dual model . We summarize the whole standard training protocol in Algorithm 1 in appendix A.2 . Constrained decoding Besides the basics , we further introduce the concept of constrained decoding where the model should be constrained to decode tokens only in the target language regardless of the shared embedding parameterization . This could give us a simple definition of cross-lingual RecErr beyond naive RecErr in Eq . 2 . Details of the algorithm and the definition are shown in appendix A.3 . 2.2 A MARGINAL MAXIMIZATION VIEW . The standard training of UNMT model takes advantage of sole monolingual corpora Ds , Dt , which is similar to the generative modeling setting where only unlabeled data is available ( Ghahramani , 2004 ) . Here we take an analogy of the standard UNMT training as implicitly maximizing marginal of the monolingual data . Due to the duality of translation ( He et al. , 2016 ) , the target sentence not only plays the role of label , but also the input in reverse translation direction . So in essence the standard UNMT training can be seen as maximizing the marginal log likelihood ofDs andDt simultaneously . However , since marginals involve infinite summation over a certain view ( target/source ) , a lower bound is often optimized via Monte Carlo approximation ( Kingma & Welling , 2014 ) . In the following derivation of ELBO ( Kingma & Welling , 2019 ) , qφ ( y|x ) is the posterior distribution of y when taking y as the latent variable . Here we only derive the bound for x ∈ Ds . A detailed analogy of the standard UNMT objective and the ELBO objective is presented in Table 1 . As you can see , both objectives have the same reconstruction error terms but different regularization terms : for ELBO , the model is optimized to stay close with the language model via the KL loss . Worth noting that , we are not the first to make a connection between marginal maximization and the standard UNMT training . In He et al . ( 2020 ) , they have already proposed an ELBO formulation for unsupervised sequence transduction task . However , they tend to focus on replacing the standard UNMT-training-style objective function with the ELBO objective , and propose several critical tricks such as Gumbel softmax and self-reconstruction for making ELBO really work . Instead , we leverage ELBO mainly as an anology to the standard UNMT training objective , and through comparative study with other protocol variants , we can further understand why standard objective and its variants work or not , even though they all tend to have similar ELBO values . Details are in appendix A.4 . 2.3 AN INFORMATION-THEORETIC VIEW . If we denote Y ′ =M ( X ) as the random variable ( r.v . ) generated by the modelM over X . Therefore , if ( Y ′ , X ) gradually contains more and more bilingual knowledge , the model will be able to generate better translations , eventually leading to the success of UNMT training . Suppose c is a constant predefined by users which controls the satisfactory level for translation performance , we give a definition to formalize the success of UNMT training from an information-theoretic viewpoint . Definition 2.1 . If I ( Y ′ , X ) > c after training , we say that UNMT training is successful ; otherwise , we say that UNMT training fails . ( Caveat , c is concetual quantity , we never instantiate its value . ) Suppose p ( x , y′ ) is the true distribution of 〈x , y′〉 , and pt→s ( x | y′ ) is an estimator of p ( x | y′ ) . We obtain the following two conditions for success of training UNMT based on the above definition 2.1 . Proposition 1 . ( Sufficient condition ) If Ep ( x , y′ ) log pt→s ( x|y′ ) ≥ c − H ( X ) , then UNMT training will be successful . Proof . Based on Definition 2.1 and the definition of mutual information ( MI ) and Jensen ’ s inequality , we can derive the following inequality ( Pimentel et al. , 2020 ) : I ( X , Y ′ ) = H ( X ) −H ( X | Y ′ ) ≥ H ( X ) −Hpt→s ( X | Y ′ ) = H ( X ) + Ep ( x , y′ ) log pt→s ( x | y′ ) ≥ c ( 4 ) Since the sufficient condition relies on true distribution p ( x , y′ ) which is unknown in practice , we sample ( x , y′ ) from the empirical distribution ofX and ps→t ( y′|x ) as approximation . Then the ideal sufficient condition is reduced to a practical one : If ∑ x Eps→t ( y′|x ) log pt→s ( x|y′ ) ≥ c − H ( X ) , then UNMT training will be successful . Since MI is symmetric , we can also have similar formula regarding s→ t direction with Ep ( x′ , y ) log ps→t ( y|x′ ) ≥ c−H ( Y ) . They together connect success of training to the BT loss in Eq . 2 : a lower BT loss is more likely to make UNMT training successful . Furthermore , if we denote the encoder output as a r.v. , Z = enc ( X ) , we can obtain the following necessary condition : Proposition 2 . ( Necessary condition ) . If UNMT training is successful , then I ( X , Z ) ≥ c. Proof . Following the Data Processing Inequality ( Cover & Thomas , 1991 ) , we have the following inequality that holds all the time : I ( X , Z ) ≥ I ( X , Y ′ ) ≥ c , ( 5 ) with the Markov chain ( or data processing order ) X enc−−→ Z dec−−→ Y ′ . In subsequent experiments , we follow Pimentel et al . ( 2020 ) and estimate I ( X , Z ) = H ( X ) − H ( X|Z ) by calculating H ( X ) through a statistical 1-gram language model and H ( X|Z ) through probing ( Conneau et al. , 2018 ) respectively . For estimating I ( X , Y ′ ) , we use the token-by-token point-wise mutual information ( PMI ) over some pseudo bitext as a surrogate to the sentence-bysentence MI . The detailed estimation methods are presented in appendix A.5 .

Summary: This paper performs an ablative study on the two components involved in training unsupervised MT systems: 1) back-translation loss, 2) denoising autoencoding loss. It links the reconstruction loss to ELBO (where the q distribution is a back-translation model). It shows that the original loss with both the components is important for unsupervised MT and ELBO needs to be augmented with denoising autoencoding loss to be effective at training unsupervised MT models.