Improved Training Techniques for Online Neural Machine Translation

1 INTRODUCTION . Sequence-to-Sequence ( S2S ) models are state-of-the-art for tasks where source and target sequences have different lengths , including automatic speech recognition , machine translation , speech translation , text-to-speech synthesis , etc . The most common models are composed of an encoder that reads the entire input sequence , while a decoder ( often equipped with an attention mechanism ) iteratively produces the next output token given the input and the partial output decoded so far . While these models perform very well in the typical offline decoding use case , few studies consider how S2S models are affected by low-latency constraints , and which architectures and strategies are the most efficient . Low-latency decoding is desirable for applications such as online speech recognition , and as-you-type machine translation . In such scenarios , the decoding process starts before the entire input sequence is available , and the output sequence is produced in an on-the-fly manner . However , if we consider for instance machine translation , online prediction generally comes at the cost of reduced translation quality and more research is needed to reach the grail of natural and high-quality online speech-to-speech interpretation . In this paper we consider deterministic “ wait-k ” decoders that are state of the art for low-latency decoding ( Ma et al. , 2019 ; Zheng et al. , 2019b ) . These decoders first read k tokens from the source , after which they proceed to alternatingly produce a target symbol and read another source symbol . We compare two architectures to implement such models : one based on a 2D-convolutional sequence-to-sequence model ( Elbayad et al. , 2018 ) , and one based on the attention-based transformer architecture ( Vaswani et al. , 2017 ) . For these models , we investigate the impact of the choice of k when training the models , and when using them to generate translations . For the transformer model , we also consider the effect of updating the hidden states of previous target symbols based on the full source context that is available at any moment . These updates are inspired from the 2D convolutional model , where such “ updates ” are an inherent consequence of the architecture . In summary , our contributions are the following : 1 . We compare transformer and 2D convolutional architectures for online machine translation . 2 . We propose improved training techniques for wait-k by first using uni-directional encoders and training across multiple values of k 3 . We boost the translation quality in test-time by updating the prefix hidden-states in transformer decoders . 4 . We set a new state of the art for online translation on WMT15 German-English , improving over the recent state-of-the-art results by Ma et al . ( 2019 ) and Zheng et al . ( 2019b ) across a full range of latency levels . The rest of this paper is organized as follows : Section 2 presents related work on low-latency machine translation ; Section 3 details our low-latency models , which we evaluate in Section 4 for German-English machine translation on the IWSLT14 dataset as well as the larger WMT15 dataset . Section 5 concludes the paper . 2 RELATED WORK . In order to study differences between translated and simultaneously interpreted text , He et al . ( 2016a ) produced a parallel corpus between both and showed that human interpreters regularly apply several tactics to reduce translation latency , including sentence segmentation and passivization . In machine translation , after pioneering work from Fügen et al . ( 2007 ) , Yarmohammadi et al . ( 2013 ) and He et al . ( 2015 ) proposed methods to increase the alignment monotonicity for statistical machine translation . To allow for a more flexible segmentation , Grissom et al . ( 2014 ) introduced a trainable segmentation module that decides whether to write or read tokens based on the prediction of the final verb and the next token in the source sequence . Similarly , Oda et al . ( 2015 ) use output of a syntax-based statistical translation system to find the optimal segmentation strategy that maximizes the quality of the translations via greedy search and dynamic programming . One of the first works on online translation to use attention-based sequence-to-sequence models is that of Cho & Esipova ( 2016 ) , which uses manually designed non-trainable waiting criteria that dictate whether the model should commit to a read/write operation . The neural transducer of Jaitly et al . ( 2016 ) reads equally-sized chunks of the source sequence and generates output sub-sequences of variable lengths , each ending with a special token marking the end of the writing . For training , a single segmentation is chosen to optimize the likelihood of the full output sequence . Raffel et al . ( 2017 ) propose an alternative to attention using monotonic alignments . Whereas attention requires access to the full source sequence to compute the weights , monotonic alignments enable linear time computation of the weights and online decoding . Another line of research treats general alignments of source and target sequence as a latent variable . Inspired from the HMM word alignment model used in statistical machine translation ( Vogel et al. , 1996 ) , the segment-to-segment neural transducer model of Yu et al . ( 2016 ) integrates a latent alignment variable into an LSTM-based sequence-to-sequence model where the transition probabilities are conditioned on the encoder-decoder hidden states . During training the alignment is marginalized out with a forward-backward algorithm ( Rabiner , 1989 ) . However , their approach is used as an alternative to attention-based models , not for low-latency translation . Luo et al . ( 2017 ) introduce a recurrent sequence-to-sequence model with binary stochastic decision variables to either emit the next output token or advance in encoding the source sequence with a unidirectional LSTM that jointly encodes the partially generated output sequence . The stochastic decisions are optimized using a standard policy gradient reinforcement learning approach . Gu et al . ( 2017 ) propose a trainable agent emitting read/write decisions modeled as a recurrent neural network fed with the encoder and decoder current hidden states . In their framework , a left-to-right recurrent sequence-to-sequence model is first pre-trained on the full bi-texts and then fine-tuned with policy gradient to optimize a reward balancing the quality and the latency of the translations . Dalvi et al . ( 2018 ) used a static decoding algorithm that starts with k read operations then alternates between blocks of l write operations and l read operations . Albeit simple , this approach outperforms the information based criteria of Cho & Esipova ( 2016 ) and allows for complete control of the translation delay . Their attempt to integrate incrementality in the training by pre-aligning the sourcetarget sequences and then attending over a constrained span of source positions failed to improve the translation quality . The work of Press & Smith ( 2018 ) , although not tackling the simultaneous translation task , allows for emitting target tokens before reading the full input sequence . Similar to the incremental training of Dalvi et al . ( 2018 ) , it requires pre-aligning the bitexts . More recently , Ma et al . ( 2019 ) trained a sequence-to-sequence model based on the transformer architecture ( Vaswani et al. , 2017 ) with an integrated “ wait-k ” agent that first reads k source tokens then alternate single read-writes , similar to Dalvi et al . ( 2018 ) but using l “ 1 . Wait-k approaches , were found most effective by Zheng et al . ( 2019b ) when trained for the specific k that is used to generate translations . This , however , requires training separate models for each potential value of k used for translation . As an alternative , Zheng et al . ( 2019a ; b ) both learn an adaptive policy to produce read/write decisions in a pre-trained offline translation model . Zheng et al . ( 2019a ) train the model for decoding along two wait-k paths and Zheng et al . ( 2019b ) use supervised training based on an oracle read/write sequence derived from the pre-trained offline translation model . Different latency levels are achieved by thresholding the policy ’ s confidence , however , on most regimes their model performs worse than using transformer models directly trained for wait-k online translation . In our work , we also use wait-k decoders , but unlike Ma et al . ( 2019 ) ; Zheng et al . ( 2019a ; b ) we opted for uni-directional encoders and show that they are efficient to train in an online setup and can achieve better results if we update the prefix decoder states during inference . 3 LOW-LATENCY SEQUENCE-TO-SEQUENCE MODELING . Let px , yq be a pair of source-target sequences of respective lengths |x| and |y| . Low-latency decoding consists of executing a sequence of interleaved reading and writing operations , consuming tokens from the source x and producing tokens of the target y. Low-latency decoding paths can be represented on a |y| ˆ |x| grid , where we advance from the top left to the bottom right in a total of |x| read steps and |y| write steps . See Figure 1 for an illustration . We consider two recent sequence-to-sequence models for low-latency decoding : the transformer architecture of Vaswani et al . ( 2017 ) and the 2D convolutional architecture of Elbayad et al . ( 2018 ) . We describe in Section 3.1 how each of these architectures is adapted for the task of simultaneous translation . In Section 3.2 we describe how these architectures can be trained for online decoding , based on one or more decoding paths . 3.1 LOW-LATENCY DECODING ARCHITECTURES . To formalize the low-latency decoding process we model it as a sequence of |y| write steps . At step t P t1 , . . . , |y|u we decode the t-th target token yt , conditioning on the target prefix yăt and the source sequence read up that point xďzt . The value of zt is dictated by the decoding path . The decoder computes the distribution over the next target token given the source and target contexts , pθpyt|yăt , xďzt , zătq , ( 1 ) where θ denotes the parameters of the writer . It depends on the architecture of the decoder whether , given zt , the prediction of yt further depends on the full decoding zăt path that led up to this point . 3.1.1 CONVOLUTION-BASED LOW-LATENCY DECODER . In order for the decoder to be trained efficiently for different source/target context sizes , we build upon the “ pervasive attention ” architecture of Elbayad et al . ( 2018 ) . Their machine translation model uses masked 2D convolutions across a source-target grid , as in Figure 1 . The masked convolutions ensure that only information from past target tokens is used to predict the next one . Here we adapt the masking pattern so as to construct the field-of-view in a way that it only extends over previous positions in both source and target dimensions . More concretely , we use a 11ˆ11 filter where only the top-left 6ˆ6 weights are non-zero . In this manner features computed by a sequence of convolutional layers at a position pt , jq in the grid can be used to define pθpyt|yăt , xďjq . As input , at each site pt , jq , the network takes the concatenation of the ( sub- ) word embedding of the t-th token in target and the j-th token in the source sequence . The input is then progressively transformed along a number of convolutional network layers . Let H “ phtjq for 1 ď t ď |y| and 1 ď j ď |x| be the final features of the masked convolutional neural network . Elbayad et al . ( 2018 ) apply max-pooling across the source dimension to map H to a fixed-sized vector for each target position . To define pθpyt|yăt , xďjq in our low-latency model , we use ftj “ PoolpHďt , ďjq , ( 2 ) where Poolp¨q poolsHďt , ďj in the source and/or target direction , or neither in which case ftj “ htj . Preliminary experiments with the pervasive attention architecture showed that max-pooling across the source dimension only is the best option . For the remainder of this paper we have : ftj “ Max-poolpHt , ďjq . ( 3 ) From there we generate the emission probabilities by linearly projecting the feature ftj to the dimension of the output vocabulary with the matrix W , followed by a soft-max normalization : pθpyt|yăt , xďjq “ softmaxpWftjq . ( 4 ) Due to the 2D convolutional structure of the network , the path leading up to position pt , jq in the network does not impact the prediction of yt . Therefore , we drop the dependence on zăt in Eq . ( 4 ) .

Summary: This work apply the wait-k decoding policy on the 2D CNN-based architecture and transformer.  In the transformer-based model the author proposed to recalculate the decoder hidden states when a new source token arrives. The author also suggested to train with multiple k at the decoder level with shared encoder output. The experiments showed that the transformer model provide the best quality on IWSLT14 En-De, De-En, and WMT15 De-EN.