Towards Stream Translation: Adaptive Computation Time for Simultaneous Machine Translation (2020)

Abstract: Simultaneous machine translation systems rely on a policy to schedule read and write operations in order to begin translating a source sentence before it is complete. In this paper, we demonstrate the use of Adaptive Computation Time (ACT) as an adaptive, learned policy for simultaneous machine translation using the transformer model and as a more numerically stable alternative to Monotonic Infinite Lookback Attention (MILk). We achieve state-of-the-art results in terms of latency-quality tradeoffs. We also propose a method to use our model on unsegmented input, i.e. without sentence boundaries, simulating the condition of translating output from automatic speech recognition. We present first benchmark results on this task.

## introduction simultaneous machine translation (mt) must accomplish two tasks: first, it must deliver correct translations on incomplete input as early as possible, i. e. before the source sentence is completely spoken. second, in a realistic usage scenario, it must deal with unsegmented input, either speech directly or automatic transcriptions without punctuation or sentence boundaries. until now, staged models @xcite , which have a separate component to insert punctuation @xcite achieved the best results in this task. in this paper, we will present the first step towards an endto-end approach. in recent years, a number of approaches for neural simultaneous machine translation have been proposed. they generally build on the common encoder-decoder framework @xcite , with the decoder deciding at each step whether to output a target language token based on the currently available information (write) or to wait for one more encoder step in order to have more information available (read). in order to do this, the decoder relies on a wait policy. the published policies can be broadly divided into two categories: ## background as @xcite point out, most previous work in simultaneous machine translation focuses on segmenting continuous input into parts that can be translated, whether it is utterances speech or sentences for text @xcite @xcite @xcite . for statistical machine translations, some approaches for stream translation without segmentation were known @xcite . the more recent neural simultaneous mt approaches simply take this segmentation as given and focus on translating simultaneously within a sentence. several approaches @xcite @xcite try to predict the whole target sentence in advance, before the input is complete. it may be possible to extend such approaches to work on an input stream, but they have the undesirable property of overriding their old output, which can make reading the translation difficult to follow for a human. @xcite train the wait policy as an agent with reinforcement learning, considering the pre-trained and fixed mt system as part of the environment. such an agent could learn to also predict the end of sentences and thus extend to stream translation, but it would be effectively the same as an explicit segmentation. cho and esipova (2016) and ma et al. (2019a) each define their own fixed policy for simultaneous mt. wait-k in particular is attractive because of its simplicity and ease of training. however, we believe that for very long input streams, an adaptive policy is necessary to make sure that the decoder never "falls behind" the input stream. most recently, the best results are produced by monotonic attention approaches @xcite , in particular @xcite . their approach uses rnns, whereas we would like to use the state-of-the-art transformer architecture @xcite . unfortunately, we were unable to transfer their results to the transformer, largely due to numerical instability problems. @xcite claim to have done this, but we were unable to reproduce their results either. we therefore propose our own, more stable, architecture based on adaptive computation time (act, graves (2016)) in the offline case, the encoder has access to all inputs at once and the attention has access to all encoder hidden states. the standard soft attention calculates the context vector as a linear combination of all hidden states: here, energy could be a multi-layer perceptron or, in the case of transformer, a projection followed by a dot product. in the simultaneous case, there are additional constraints: each encoder state must only depend on the representations before it and the inputs up to the current one as input becomes available incrementally. in addition, we require a wait policy which decides in each step whether to read another encoder state or to write a decoder output. each read incurs a delay, but gives the decoder more information to work with. we denote the encoder step at which the policy decides to write in decoder step i as n (i). note that this kind of discrete decision-making process is not differentiable. some approaches using reinforcement learning have been proposed @xcite , but we will focus on the monotonic attention approaches. ## conclusion and future work we have presented adaptive compuation time (act) for simultaneous machine translation and demonstrated its ability to translate continuous, unsegmented streams of input text. to the best of our knowledge, this is the first end-to-end nmt model to do so. while stream translation model still loses a lot of performance compared to the sentence-based models, we see this as an important step towards end-to-end simultaneous stream translation. we see several possibilites for future work on this model: training the whole model in one training rather than the multiple rounds of pre-training may be possible by gradually introducing the latency loss during training. perhaps the latency decisions can be improved by adding extra layers to the act attention module. but most importantly, we believe the model must be adapted to the speech domain