A General Framework for Adaptation of Neural Machine Translation to Simultaneous Translation (2020)

Abstract: Despite the success of neural machine translation (NMT), simultaneous neural machine translation (SNMT), the task of translating in real time before a full sentence has been observed, remains challenging due to the syntactic structure difference and simultaneity requirements. In this paper, we propose a general framework for adapting neural machine translation to translate simultaneously. Our framework contains two parts: prefix translation that utilizes a consecutive NMT model to translate source prefixes and a stopping criterion that determines when to stop the prefix translation. Experiments on three translation corpora and two language pairs show the efficacy of the proposed framework on balancing the quality and latency in adapting NMT to perform simultaneous translation.

## introduction simultaneous translation @xcite @xcite @xcite @xcite , the task of producing a partial translation of a sentence before the whole input sentence ends, is useful in many scenarios including outbound tourism, international summit and multilateral negotiations. different from the consecutive translation in which translation quality alone matters, simultaneous translation trades off between translation quality and latency. the syntactic structure difference between the source and target language makes simultaneous translation more challenging. for example, when translating from a verb-final (sov) language (e.g., japanese) to a verb-media (svo) language (e.g., english), the verb appears much later in the source sequence than in the target language. some premature translations can lead to significant loss in quality @xcite . recently, a number of researchers have endeavored to explore methods for simultaneous translation in the context of nmt @xcite . some of them propose sophisticated training frameworks explicitly designed for simultaneous translation @xcite . these approaches are either memory inefficient during training @xcite or with hyper-parameters hard to tune @xcite . others utilize a full-sentence base model to perform simultaneous translation by modifications to the encoder and the decoding process. to match the incremental source context, they replace the bidirectional encoder with a left-to-right encoder @xcite @xcite or recompute the encoder hidden states @xcite . on top of that, heuristic algorithms @xcite or a read/write model trained with reinforcement learning @xcite @xcite or supervised learning @xcite are used to decide, at every step, whether to wait for the next source token or output a target token. however, these models either cannot directly use a pretrained consecutive neural machine translation (cnmt) model with bidirectional encoder as the base model or work in a sub-optimal way in the decoding stage. in this paper, we study the problem of adapting neural machine translation to translate simultaneously. we formulate simultaneous translation as two nested loops: an outer loop that updates input buffer with newly observed source tokens and an inner loop that translates source tokens in the buffer updated at each outer step. for the outer loop, the input buffer can be updated by an asr system with an arbitrary update schedule. for the inner loop, we translate using the pretrained cnmt model and stop translation with a stopping controller. such formulation is different from previous work @xcite @xcite which define simultaneous translation as sequentially making interleaved read or write decisions. we argue that our formulation is better than the previous one in two aspects: (i) our formulation can better utilize the available source tokens. under previous formulation, the number of source tokens observed by the cnmt model is determined by the number of read actions that has been produced by the policy network. it is likely that the cnmt model does not observe all the available source tokens produced by the asr system. in contrast, the cnmt model observes all the available source tokens when performing inner loop translation in our framework. (ii) previous formulation makes t η +t τ read or write decisions regardless of the asr update schedule, where t η and t τ are source sentence and translation length, respectively. for an asr system that outputs multiple tokens at a time, this is computational costly. consider an extreme case where the asr system outputs a full source sentence at a time. previous work translates with a sequence of t η +t τ actions, while we translate with a sequence of t τ decisions (t τ -1 continue and 1 stop). under our proposed framework, we present two schedules for simultaneous translation: one stops the inner loop translation with heuristic and one with a stopping controller learned in a reinforcement learning framework to balance translation quality and latency. we evaluate our method on iwslt16 german-english (de-en) translation in both directions, wmt15 english-german (en-de) translation in both directions, and nist chineseto-english (@xmath0. the results show our method with reinforced stopping controller consistently improves over the de-facto baselines, and achieves low latency and reasonable bleu scores. ## background given a set of source-target sentence pairs x m , y * m m @xmath1, a consecutive nmt model can be trained by maximizing the log-likelihood of the target sentence from its entire source side context: the distribution of next target word is defined as: in consecutive nmt, once obtained, the encoder hidden states h and the decoder hidden state z t are not updated anymore and will be reused during the entire decoding process. ## simultaneous nmt in snmt, we receive streaming input tokens, and learn to translate them in real-time. we formulate simultaneous translation as two nested loops: the outer loop that updates an input buffer with newly observed source tokens and the inner loop that translates source tokens in the buffer updated at each outer step. more precisely, suppose at the end of outer step s -1, the input buffer is x s-1 = {x 1 , ..., x η[s-1] }, and the output buffer is y s-1 = {y 1 , ..., y τ [s-1] }. then at outer step s, the system translates with the following steps: the simultaneous decoding process continues until no more source tokens are added in the outer loop. we define the last outer step as the terminal outer step s, and other outer steps as non-terminal outer steps. for the outer loop, we make no assumption about the value of c s , while all previous work assumes c @xmath2. this setting is more realistic because (i) increasing c s can reduce the number of outer steps, thus reducing computation cost; (ii) in a real speech translation application, an asr system may generate multiple tokens at a time. for the inner loop, we adapt a pretrained vanilla cnmt model to perform partial translation with two important concerns: ## related work a number of works in simultaneous translation divide the translation process into two stages. a segmentation component first divides the incoming text into segments, and then each segment is translated by a translator independently or with previous context. the segmentation boundaries can be predicted by prosodic pauses detected in speech @xcite , linguistic cues @xcite , or a classifier based on alignment information @xcite and translation accuracy @xcite @xcite . some authors have recently endeavored to perform simultaneous translation in the context of nmt. niehues et al. (2018 ); arivazhagan et al. (2020) adopt a re-translation approach where the source is repeatedly translated from scratch as it grows and propose methods to improve translation stability. cho and esipova (2016); dalvi et al. (2018); ma et al. (2018) introduce a manually designed criterion to control when to translate. satija and pineau (2016); gu et al. (2017); alinejad et al. (2018) extend the criterion into a trainable agent 1 2 3 4 5 6 7 8 9 吴邦国 出席 签字 仪式 并 在 协议 上 签字 le wu bangguo attended the signing ceremony and signed the agreement tn wu bangguo attended the signing ceremony and signed the agreement greedy wu bangguo attended the signing ceremony and signed the agreement ref wu bangguo attends signing ceremony and signs agreement nato does not want to break agreements with russia le die nato möchte keine abkommen mit russland brechen tn die nato will keine abkommen mit russland brechen greedy die nato möchte keine abkommen mit russland brechen ref nato will vereinbarungen mit russland nicht brechen figure 9: translation examples from the test set of nist @xmath3. we compare le with @xmath4. greedy and ref represent the greedy decoding result from consecutive translation and the reference, respectively. in a reinforcement learning framework. however, these work either develop sophisticated training frameworks explicitly designed for simultaneous translation @xcite or fail to use a pretrained consecutive nmt model in an optimal way @xcite @xcite @xcite . in contrast, our work is significantly different from theirs in the way of using pretrained consecutive nmt model to perform simultaneous translation and the design of the two stopping criteria. ## conclusion we have presented a novel framework for improving simultaneous translation with a pretrained consecutive nmt model. the basic idea is to translate partial source sentence with the consecutive nmt model and stops the translation with two novel stopping criteria. extensive experiments demonstrate that our method with trainable stopping controller outperforms the state-of-the-art baselines in balancing between translation quality and latency.