Neural machine translation ( NMT ) with recurrent neural networks , has proven to be an effective technique for end-to-end machine translation . However , in spite of its promising advances over traditional translation methods , it typically suffers from an issue of unbalanced outputs , that arise from both the nature of recurrent neural networks themselves , and the challenges inherent in machine translation . To overcome this issue , we propose an agreement model for neural machine translation and show its effectiveness on large-scale Japaneseto-English and Chinese-to-English translation tasks . Our results show the model can achieve improvements of up to 1.4 BLEU over the strongest baseline NMT system . With the help of an ensemble technique , this new end-to-end NMT approach finally outperformed phrasebased and hierarchical phrase-based Moses baselines by up to 5.6 BLEU points . Recurrent neural network ( RNN ) has achieved great successes on several structured prediction tasks ( Graves , 2013 ; Watanabe and Sumita , 2015 ; Dyer et al . , 2015 ) , in which RNNs are required to make a sequence of dependent predictions . One of its advantages is that an unbounded history is available to enrich the context for the prediction at the current time-step . Despite its successes , recently , ( Liu et al . , 2016 ) pointed out that the RNN suffers from a fundamental issue of generating unbalanced outputs : that is to say the suffixes of its outputs are typically worse than the prefixes . This is due to the fact that later predictions directly depend on the accuracy of previous predictions . They empirically demonstrated this issue on two simple sequence-to-sequence learning tasks : machine transliteration and grapheme-to-phoneme conversion . On the more general sequence-to-sequence learning task of machine translation ( MT ) , neural machine translation ( NMT ) based on RNNs has recently become an active research topic ( Sutskever et al . , 2014 ; Bahdanau et al . , 2014 ) . Compared to those two simple tasks , MT involves in much larger vocabulary and frequent reordering between input and output sequences . This makes the prediction at each time-step far more challenging . In addition , sequences in MT are much longer , with averaged length of 36.7 being about 5 times longer than that in grapheme-to-phoneme conversion . Therefore , we believe that the history is more likely to contain incorrect predictions and the issue of unbalanced outputs may be more serious . This hypothesis is supported later ( see Table 1 in § 4.1 ) , by an analysis that shows the quality of the prefixes of translation hypotheses is much higher than that of the suffixes . To address this issue for NMT , in this paper we extend the agreement model proposed in ( Liu et al . , 2016 ) to the task of machine translation . Its key idea is to encourage the agreement between a pair of target-directional ( left-to-right and right-to-left ) NMT models in order to produce more balanced translations and thus improve the overall translation quality . Our contribution is two-fold : • We introduce a simple and general method to address the issue of unbalanced outputs for NMT ( § 3 ) . This method is robust without any extra hyperparameters to tune and is easy to implement . In addition , it is general enough to be applied on top of any of the existing RNN translation models , although it was implemented on top of the model in ( Bahdanau et al . , 2014 ) in this paper . • We provide an empirical evaluation of the technique on large scale Japanese-to-English and Chinese-to-English translation tasks . The results show our model can generate more balanced translation results , and achieves substantial improvements ( of up to 1.4 BLEU points ) over the strongest NMT baseline ( § 4 ) . With the help of an ensemble technique , our new end-to-end NMT gains up to 5.6 BLEU points over phrase-based and hierarchical phrasebased Moses ( Koehn et al . , 2007 ) systems.1 2 Overview of Neural Machine Translation Suppose x = x 1 , x 2 , • • • , x m denotes a source sentence , y = y 1 , y 2 , • • • , y n denotes a target sen- tence . In addition , let x < t = x 1 , x 2 , • • • , x t-1 denote a prefix of x. Neural Machine Translation ( NMT ) directly maps a source sentence into a target within a probabilistic framework . Formally , it defines a conditional probability over a pair of sequences x and y via a recurrent neural network as follows : EQUATION where θ is the set of model parameters ; h t denotes a hidden state ( i.e. a vector ) of y at timestep t ; g is a transformation function from a hidden state to a vector with dimension of the target-side vocabulary size ; softmax is the softmax function , and [ i ] denotes the i th component in a vector . 2 Furthermore , h t = f ( h t-1 , c(x , y < t ) ) is defined by a recurrent function over both the previous hidden state h t-1 and the context c(x , y < t ) .3 Note that both h t and c(x , y < t ) have dimension d for all t. In this paper , we develop our model on top of the neural machine translation approach of ( Bahdanau et al . , 2014 ) , and we refer the reader this paper for a complete description of the model , for example , the definitons of f and c. The proposed method could just as easily been implemented on top of any other RNN models such as that in ( Sutskever et al . , 2014 ) . In this paper , we investigate the issue of unbalanced outputs suffered by recurrent neural networks , and empirically show its existence in the context of machine translation . To address this issue , we propose an easy to implement agreement model that extends the method of ( Liu et al . , 2016 ) from simple sequence-to-sequence learning tasks to machine translation . On two challenging JP-EN and CH-EN translation tasks , our approach was empirically shown to be effective in addressing the issue ; by generating balanced outputs , it was able to consistently outperform a respectable NMT baseline on all test sets , delivering gains of up to 1.4 BLEU points . To put these results in the broader context of machine translation research , our approach ( even without special handling of unknown words ) achieved gains of up to 5.6 BLEU points over strong phrase-based and hierarchical phrase-based Moses baselines , with the help of an ensemble technique .

Challenge: Recurrent neural network-based machine translation models suffer from unbalanced outputs, meaning quality degrades as they generate outputs.
Approach: They propose to apply a generic hyperparameter-free agreement model for neural machine translation which can be used in any neural models.
Outcome: The neural machine translation models with the agreement model generate better balanced outputs and significantly outperform baseline models.