Dual Reconstruction: a Unifying Objective for Semi-Supervised Neural Machine Translation (2020)

Abstract: While Iterative Back-Translation and Dual Learning effectively incorporate monolingual training data in neural machine translation, they use different objectives and heuristic gradient approximation strategies, and have not been extensively compared. We introduce a novel dual reconstruction objective that provides a unified view of Iterative Back-Translation and Dual Learning. It motivates a theoretical analysis and controlled empirical study on German-English and Turkish-English tasks, which both suggest that Iterative Back-Translation is more effective than Dual Learning despite its relative simplicity.

## introduction taking advantage of monolingual training data via back-translation @xcite , iterative back-translation @xcite or dual learning @xcite has become a de facto requirement for building high quality neural machine translation (nmt) systems @xcite . however, these methods rely on unrelated heuristic optimization objectives, and it is not clear what their respective strengths and weaknesses are, nor how they relate to the ideal but intractable objective of maximizing the marginal likelihood of the monolingual data (i.e., p θ (y) = x p θ (y | x)q(x) given target sentences y, an nmt model p θ (y | x), and the prior distribution q(x) on source x). instead of proposing new methods, this paper sheds new light on how these established techniques work and how to use them. we introduce a dual reconstruction objective to theoretically ground the comparison of semi-supervised training strategies that leverage monolingual data from both source and target languages (figure 1 ). in section 3, we show that, under some assumptions, this objective remarkably shares the same global optimum as the intractable marginal likelihood objective where the model's marginal distribution p θ (y) coincides with the target sentence distribution p(y). we also show that iterative back-translation (ibt) and dual learning can be viewed as different ways to approximate its optimization. theory suggests that ibt approximates the dual reconstruction objective more closely than the more complex dual learning approach, and in particular that dual learning's additional language model loss is redundant. we investigate whether these differences matter in practice by conducting the first controlled empirical comparison of back-translation, ibt, and dual learning in high-resource (wmt de-en), low-resource (wmt tr-en), and cross-domain settings @xcite . results support our theory that the additional language model loss and policy gradient estimation in dual learning is redundant and show that ibt outperforms the more complex dual learning algorithm in terms of translation quality. furthermore, we also compare different optimization strategies used in ibt to better balance translation quality against the computational cost. ibt and dual learning exploit large monolingual corpora which represent source and target language distributions better than the limited parallel corpora. back-translation trains the source-totarget translation model p θ (y | x) by maximizing the conditional log-likelihood of target language sentences y given pseudo source sentences x inferred by a pre-trained target-to-source translation model q φ (x | y) given y. ibt optimizes the dual translation models p θ (y | x) and q φ (x | y) via backtranslation in turn, both for semi-supervised @xcite @xcite and unsupervised mt @xcite . dual learning takes the view of cooperative game theory where dual models collaborate with each other to learn to reconstruct the observed source and target monolingual sentences, and is widely used for semi-supervised @xcite , unsupervised @xcite , and zero-shot multilingual nmt @xcite . concretely, dual learning optimizes p θ (y | x) and q φ (x | y) jointly by reconstructing the original target sentence y using p θ (y | x) given the source x inferred by q φ (x | y), and vice versa. the reconstruction loss is augmented with a language model loss and used to update both reconstruction and inference models via policy gradient @xcite . while dual learning and ibt each improve bleu over @xcite @xcite , they have not been compared directly to each other. @xcite interpret back-translation as a variational approximation where the pseudo source x can be viewed as a latent vari-able and the target-to-source model q φ (x | y) is an inference network that approximates the posterior distribution p θ (x | y). furthermore, they explain ibt as a way to better approximate the true posterior distribution with the target-to-source model. however, it is unclear how their heuristic objective relates to the ideal objective of maximizing the model's marginal likelihood of the target language monolingual data. more recently, @xcite connect back-translation and the language model loss in dual learning to the variational lower-bound (elbo) of the marginal likelihood objective. we introduce a more direct connection. ## empirical study we evaluate on six translation tasks ( ## summary of contributions we contribute theoretical and empirical results that improve our understanding of the connection between two seemingly distant semi-supervised training strategies for nmt: iterative back-translation (ibt) and dual learning. on the theory side, we define a dual reconstruction objective which unifies semi-supervised nmt techniques that exploit source and target monolingual text. we prove that optimizing this objective leads to the same global optimum as the intractable marginal likelihood objective, where the model's marginal distribution coincides with the prior language distribution while also maximizing the model's mutual information between source and target. ibt approximates this objective more closely than dual learning, despite the more complex objective and update strategies used in the latter. we present a systematic empirical comparison of back-translation, ibt, and dual learning on six tasks spanning high-resource, low-resource, and cross-domain settings. results support the theory that the lm loss and policy gradient estimation are unnecessary in dual learning, and show that ibt achieves better translation quality than dual learning. analysis confirms that the mutual information constraint required to reach an interesting dual reconstruction optimum is satisfied in practice. these findings lead us to recommend batch-level ibt to quickly boost model performance at early training stages and epoch-level ibt to further improve quality. our theory also suggests future directions for improving unsupervised mt via more effective methods to maximize the model's mutual information between source and target, and the potential of applying our dual reconstruction objective to other sequence-to-sequence tasks. the 2018 conference on empirical methods in natural language processing, pages 3612-3621. association for computational linguistics. zhirui zhang, shujie liu, mu li, ming zhou, and enhong chen. 2018. joint training for neural machine translation models with monolingual data. in proceedings of the thirty-second aaai conference on artificial intelligence, pages 555-562. aaai press.