Dual Learning: Theoretical Study and Algorithmic Extensions

Dual learning has been successfully applied in many machine learning applications including machine translation , image-to-image transformation , etc . The high-level idea of dual learning is very intuitive : if we map an x from one domain to another and then map it back , we should recover the original x . Although its effectiveness has been empirically verified , theoretical understanding of dual learning is still missing . In this paper , we conduct a theoretical study to understand why and when dual learning can improve a mapping function . Based on the theoretical discoveries , we extend dual learning by introducing more related mappings and propose highly symmetric frameworks , cycle dual learning and multipath dual learning , in both of which we can leverage the feedback signals from additional domains to improve the qualities of the mappings . We prove that both cycle dual learning and multipath dual learning can boost the performance of standard dual learning under mild conditions . Experiments on WMT 14 English↔German and MultiUN English↔French translations verify our theoretical findings on dual learning , and the results on the translations among English , French , and Spanish of MultiUN demonstrate the efficacy of cycle dual learning and multipath dual learning . 1 INTRODUCTION . Most machine learning tasks can be formulated as learning a mapping from one domain to another one , like image classification ( from image to label ) , neural machine translation ( from source language to target language ) , speech recognition ( from voice to text ) , etc . Among them , quite a few tasks are of dual forms , like image classification v.s . image generation ( from label to image ) , the neural machine translation between two languages ( English→French v.s . French→English ) , speech recognition v.s . speech synthesis ( from text to voice ) , etc . Such duality can be utilized to improve the model qualities . One prominent framework is dual learning , first proposed by He et al . ( 2016 ) for machine translation and then applied to many other applications like image translation ( Kim et al. , 2017 ; Zhu et al. , 2017 ) , question answering and generation ( Tang et al. , 2017 ) , etc . In dual learning , two mapping functions between two domains are trained simultaneously so that one function is close to the inverse of the other . The intuition is that , if we translate a sentence from English to French and then translate the obtained French sentence back to English , we should get the same sentence or a very simlar one . Dual learning is of great interest because it can accommodate any unidirectional architecture , e.g . a transformer ( Vaswani et al. , 2017 ) , and provide a performance boost . Moreover , dual learning can be used in semi-supervised learning , which is highly desirable since deep neural networks are generally thirst for labeled data . Despite of the empirical success of dual learning , the theoretical understanding is largely missing . In this paper , we conduct both theoretical analyses and empirical studies to answer the following questions : Q1 : Why and when does dual learning improve a mapping function ? Q2 : Can we further improve the performance of a mapping function ? 1.1 OUR CONTRIBUTIONS . Our contributions are in two folds : a theoretical study of dual learning and the frameworks of cycle dual learning and multipath dual learning , which subsume dual learning as a special case . Without loss of generality , we take machine translation as the example for the study and algorithm presentation . Dual learning . We take a novel statistical approach to model the problem . Suppose there are two baseline translators between two language spaces , one forward and one backward . Based on our Theorem 1 , dual learning outperforms both baseline translators as long as the baseline translators are better than random mappings under natural assumptions . Empirical studies show that an improvement is observed even if the reconstruction is far from perfect . Cycle dual learning and multipath dual learning . We propose cycle dual learning and multipath dual learning frameworks by extending dual learning . Both frameworks use dual learning as the basic building block and leverage a third , a fourth , or more languages to help boost the translator qualities between the original two languages . We prove that under mild conditions , the two symmetric frameworks outperform dual learning ( Theorems 2 and 3 ) . Our experiments on MultiUN dataset show a significant improvement ( 1.45 BLEU points , see Table 4 ) against dual learning . 1.2 RELATED WORK . Dual learning was first proposed by He et al . ( 2016 ) in the context of machine translation , where the two dual translators are updated in a reinforcement learning manner with the reconstructed distortion as the feedback signal . A similar approach proposed by Cheng et al . ( 2016 ) has the same high-level idea but a very different implementation . Since then dual learning architectures have been proposed for other applications including image processing ( Kim et al. , 2017 ; Zhu et al. , 2017 ) , sentiment analysis ( Xia et al. , 2017a ) , image segmentation ( Luo et al. , 2017 ) , etc . Built upon the dual learning framework , Xia et al . ( 2017b ) and Wang et al . ( 2018 ) consider the joint distribution constraint , which says the joint distribution of sample over two domains is invariant when computing from either domain . We relax this constraint for simplicity of analysis . Xia et al . ( 2018 ) proposed model-level dual learning , which shares components between the primary direction and the dual direction . Dual learning was also leveraged for unsupervised learning ( Lample et al. , 2018 ; Artetxe et al. , 2018 ) . Despite the vast number of works related to dual learning , none of them explain why it works or when it fails . Galanti et al . ( 2018 ) claim that dual learning does not circumvent the alignment problem , where a sentence is translated wrong by the forward translator but translated back to it by the backward translator . We show that the alignment problem occurs with a small probability under dual learning compared to the baseline translator , and this probability can be further reduced by our cycle dual learning and multipath dual learning . Furthermore , their hypothesis that the translator should not be too complex is not verified in the context of machine translation . 2 PRELIMINARIES . In this section , we introduce the notations and the essential assumptions used for theoretical analysis . A table summarizing the notations can be found in Appendix A . Let S1 , . . . , Sk be k language spaces , composed of sentences in each language . For any Si , we denote the distribution of sentences in Si by µ ( i ) and let X ( i ) be the random variable , i.e . Pr ( X ( i ) = x ) = µ ( i ) ( x ) . For any x ( i ) ∈ Si , we denote the correct translation in Sj by x ( j ) . In practice , we may select a threshold BLEU ( Papineni et al. , 2002a ) score , above which the translation is considered correct . Let Tij denote the baseline translator that translates from a sentence in Si to one in Sj , i.e . the desired mapping is Tij ( x ( i ) ) = x ( j ) . When a random sentence x ( i ) is sampled from Si according to µ ( i ) , it is possible that this sentence is translated incorrectly . We use accuracy of the translator pij , the accuracy of the translator , to describe the probability of translating a sentence correctly when this sentence is randomly sampled from Si according to µ ( i ) . Formally , pij = Prx ( i ) ∼µ ( i ) ( Tij ( x ( i ) ) = x ( j ) ) = Ex ( i ) ∼µ ( i ) [ 1Tij ( x ( i ) ) =x ( j ) ] = ∑ x ( i ) ∈Si 1Tij ( x ( i ) ) =x ( j ) µ ( i ) ( x ( i ) ) , where 1 is the indicator function . We sometimes omit the subscript x ( i ) ∼ µ ( i ) for simplicity . It is also easy to see that Pr ( Tij ( x ( i ) ) 6= x ( j ) ) = E [ 1Tij ( x ( i ) ) 6=x ( j ) ] = 1− pij . ( 1 ) In order to characterize reconstruction accuracy , we let X ( j , r ) denote the random variable that follows the distribution Tij ( X ( i ) ) where X ( i ) ∼ µ ( i ) . We define µ ( j , r ) ( x ) = Pr ( X ( j , r ) = x ) , and further define prji = PrX ( j , r ) ∼µ ( j , r ) ( Tji ( x ( j ) ) = x ( i ) ) = EX ( j , r ) ∼µ ( j , r ) [ 1Tji ( x ( j ) ) =x ( i ) ] =∑ x ( j ) ∈Sj 1Tji ( x ( j ) ) =x ( i ) µ ( j , r ) ( x ( j ) ) . The superscript r means “ reconstruction ” . The difference between prji and pji lies in the distributions of samples in space Sj . We assume that Tij and Tji are independent translators . This is natural because Tij and Tji are usually trained independently . Then , for any j 6= i , we have Pr ( Tij ( x ( i ) ) = x ( j ) , Tji ( x ( j ) ) = x ( i ) ) = pijp r ji . ( 2 ) Further , we make the following assumption . Assumption 1 . For any j 6= i , any y ( j ) ∈ Sj , E [ 1y ( j ) 6=x ( j ) , Tij ( x ( i ) ) =y ( j ) ] = 1−pij m , where m is a constant . The interpretation of this assumption is as follows . For all x ( i ) ∈ Si that are incorrectly translated into Sj , there are m possibly incorrect translations ( in Sj ) with positive probabilities . Assumption 1 states that each of the incorrect translations is chosen uniformly at random . Justification of Assumption 1 . It is generally not true in the real-world since the probabilities of different translations are usually very different . However , this assumption can be approximated by partitioning the possibly wrong translations into several clusters , where the probability of the translation falling into each cluster is close . The best case is that the incorrect translation with the largest probability pmax is treated as one cluster . All the other possibly incorrect translations are clustered so that the probability of each cluster is close to pmax . Then we have m ≤ 1−pijpmax . A small pmax ( a large m ) is desirable . Note that the correct translation is indeed also a group of translations if we set a threshold BLEU score , above which the translation is considered correct . To analyze the efficacy of dual learning , we consider the following two cases : ( Case 1 ) Tji ( Tij ( x ( i ) ) ) = x ( i ) ; ( Case 2 ) Tji ( Tij ( x ( i ) ) ) 6= x ( i ) . Dual learning will detect Case 2 and train the translators so that Case 2 is minimized . We use superscript d for the translators trained from dual learning . For any given xi ∈ Si which fails in reconstruction ( Case 2 ) , we define α = Pr ( T dij ( x ( i ) ) = x ( j ) , T dji ( T d ij ( x ( i ) ) ) = x ( i ) |Case 2 ) ( 3 ) β = Pr ( T dij ( x ( i ) ) 6= x ( j ) , T dji ( T dij ( x ( i ) ) ) = x ( i ) |Case 2 ) ( 4 ) γ = Pr ( T dji ( T d ij ( x ( i ) ) ) 6= x ( i ) |Case 2 ) , ( 5 ) where “ Case 2 ” denotes the condition Tji ( Tij ( x ( i ) ) ) 6= x ( i ) . Here α can be viewed as the probability of correcting the wrong translations by dual learning , β the probability of the occurrence of the alignment problem under Case 2 , and γ the probability of nonzero reconstruction error . γ models the imperfectness of dual learning , which should be zero in the ideal case . It is easy to see α+β+γ = 1 .

Summary: The paper addresses a means of boosting the accuracy of automatic translators (sentences) by training dual models (a.k.a. language A to B, B to A), multipath (e.g. A to B to C) and cyclical (e.g. A to B to C to A) while starting with well initialized models for translating simple pairs. The idea that additional errors are revealed and allow classifiers to adapt, thus boosting classifier performance, is appealing and intuitive. That a theoretical framework is presented to support this assumption is encouraging, but the assumptions behind this analysis (e.g. Assumption 1) are rather strong. Equation 2 assumes independence. Equations 3-5 can be presented more clearly. Not only are translation errors not uniform in multiclass settings, but they can be highly correlated - this being a possible pitfall of boosting approaches, seen as general classifier errors. The same strong assumptions permeate from the dual to the cyclical case. On the other hand, the (limited) empirical test presented, using a single dataset and baseline classifier method, does support the proposed improvement by boosting, albeit leading to improvements which are slight.