Opportunistic Decoding with Timely Correction for Simultaneous Translation (2020)

Abstract: Simultaneous translation has many important application scenarios and attracts much attention from both academia and industry recently. Most existing frameworks, however, have difficulties in balancing between the translation quality and latency, i.e., the decoding policy is usually either too aggressive or too conservative. We propose an opportunistic decoding technique with timely correction ability, which always (over-)generates a certain mount of extra words at each step to keep the audience on track with the latest information. At the same time, it also corrects, in a timely fashion, the mistakes in the former overgenerated words when observing more source context to ensure high translation quality. Experiments show our technique achieves substantial reduction in latency and up to +3.1 increase in BLEU, with revision rate under 8% in Chinese-to-English and English-to-Chinese translation.

## introduction simultaneous translation, which starts translation before the speaker finishes, is extremely useful in many scenarios, such as international conferences, travels, and so on. in order to achieve low latency, it is often inevitable to generate target words with insufficient source information, which makes this task extremely challenging. recently, there are many efforts towards balancing the translation latency and quality with mainly two types of approaches. on one hand, @xcite propose very simple frameworks that decode following a fixed-latency policy such as waitk. on the other hand, there are many attempts to learn an adaptive policy which enables the model to decide read or write action on the fly using various techniques such as reinforcement learning @xcite ; grissom ii * these authors contributed equally. y @xmath0 a n e i f x 4 9 f 4 n t 6 m 9 / l o x s h 3 9 u a x j i 9 v l j k u m @xmath1: besides y t , opportunistic decoding continues to generate additional w words which are represented as ŷ w t . the timely correction only revises this part in future steps. different shapes denote different words. in this example, from step t to t + 1, all previously opportunistically decoded words are revised, and an extra triangle word is generated in opportunistic window. from step t + 1 to t + 2, two words from previous opportunistic window are kept and only the triangle word is revised. et al., 2014), supervised learning over pseudooracles @xcite , imitation learning @xcite , model ensemble @xcite or monotonic attention @xcite . though the existing efforts improve the performance in both translation latency and quality with more powerful frameworks, it is still difficult to choose an appropriate policy to explore the optimal balance between latency and quality in practice, especially when the policy is trained and applied in different domains. furthermore, all existing approaches are incapable of correcting the mistakes from previous steps. when the former steps commit errors, they will be propagated to the later steps, inducing more mistakes to the future. inspired by our previous work on speculative beam search @xcite , we propose an opportunistic decoding technique with timely correction mechanism to address the above problems. as shown in fig. 1 , our proposed method always decodes more words than the original policy at each step to catch up with the speaker and reduce the latency. at the same time, it also employs a timely correction mechanism to review the extra outputs from previous steps with more source context, and revises these outputs with current preference when there is a disagreement. our algorithm can be used in both speech-to-text and speech-to-speech simultaneous translation @xcite @xcite . in the former case, the audience will not be overwhelmed by the modifications since we only review and modify the last few output words with a relatively low revision rate. in the later case, the revisable extra words can be used in look-ahead window in incremental tts @xcite . by contrast, the alternative retranslation strategy @xcite will cause non-local revisions which makes it impossible to be used in incremental tts. we also define, for the first time, two metrics for revision-enabled simultaneous translation: a more general latency metric revision-aware average lagging (ral) as well as the revision rate. we demonstrate the effectiveness of our proposed technique using fixed @xcite and adaptive @xcite policies in both chineseto-english and english-to-chinese translation. ## preliminaries full-sentence nmt. the conventional fullsentence nmt processes the source sentence @xmath2, ..., x n ) with an encoder, where x i represents an input token. the decoder on the target side (greedily) selects the highest-scoring word y t given source representation h and previously generated target tokens, @xmath3, ..., y t-1 ), and the final hypothesis @xmath4, ..., y t ) with y @xmath5: simultaneous translation. without loss of generality, regardless the actual design of policy, simultaneous translation is represented as: timely correction. in order to deliver the correct information to the audience promptly and fix previous mistakes as soon as possible, we also need to review and modify the previous outputs. at step t + 1, when encoder obtains more information from x g(t) to x g(t+1) , the decoder is capable to generate more appropriate candidates and may revise and replace the previous outputs from opportunistic decoding. more precisely, ŷ w t and y t+1 • ŷ w-1 t+1 are two different hypothesis over the same time chunk. when there is a disagreement, our model always uses the hypothesis from later step to replace the previous commits. note our model does not change any word in y t from previous step and it only revise the words in ŷ w t . modification for adaptive policy. for adaptive policies, the only difference is, instead of committing a single word, the model is capable of generating multiple irreversible words. thus our proposed methods can be easily applied to adaptive policies. correction with beam search. when the model is committing more than one word at a time, we can use beam search to further improve the translation quality and reduce revision rate @xcite . the decoder maintains a beam b k t of size b at step t, which is ordered list of pairs figure 2 : the decoder generates target word y 4 = "his" and two extra words "welcome to" at step @xmath6. when the model receives x 9 at step @xmath7, the decoder immediately corrects the previously made mistake "welcome" with "agreement" and emits two additional target words ("to president"). the decoder not only is capable to fix the previous mistake, but also has enough information to perform more correct generations. our framework benefits from opportunistic decoding with reduced latency here. note though the word "to" is generated in step @xmath8, it only becomes irreversible at step @xmath9. hypothesis, probability , where k denotes the k th step in beam search. at each step, there is an initial beam b 0 @xmath10, 1 ]. we denote one-step transition from the previous beam to the next as when the opportunistic decoding window is w at decoding step t, we define the beam search over w + 1 (include the original output) as follows: ## revision-aware al and revision rate we define, for the first time, two metrics for revision-enabled simultaneous translation. ## conclusions we have proposed an opportunistic decoding timely correction technique which improves the latency and quality for simultaneous translation. we also defined two metrics for revision-enabled simultaneous translation for the first time.