Re-translation versus Streaming for Simultaneous Translation (2020)

Abstract: There has been great progress in improving streaming machine translation, a simultaneous paradigm where the system appends to a growing hypothesis as more source content becomes available. We study a related problem in which revisions to the hypothesis beyond strictly appending words are permitted. This is suitable for applications such as live captioning an audio feed. In this setting, we compare custom streaming approaches to re-translation, a straightforward strategy where each new source token triggers a distinct translation from scratch. We find re-translation to be as good or better than state-of-the-art streaming systems, even when operating under constraints that allow very few revisions. We attribute much of this success to a previously proposed data-augmentation technique that adds prefix-pairs to the training data, which alongside wait-k inference forms a strong baseline for streaming translation. We also highlight re-translation’s ability to wrap arbitrarily powerful MT systems with an experiment showing large improvements from an upgrade to its base model.

## introduction in simultaneous machine translation, the goal is to translate an incoming stream of source words with as low latency as possible. a typical application is speech translation, where we often assume the eventual output modality to also be speech. in a speech-to-speech scenario, target words must be appended to existing output with no possibility for revision. the corresponding translation task, which we refer to as streaming translation, has received considerable recent attention, generating custom approaches designed to maximize quality and minimize latency (cho and esipova, 2016; gu et al., * equal contributions 2017; @xcite . however, for applications where the output modality is text, such as live captioning, the prohibition against revising output is overly stringent. the ability to revise previous partial translations makes simply re-translating each successive source prefix a viable strategy. compared to streaming models, re-translation has the advantage of low latency, since it always attempts a translation of the complete source prefix, and high final-translation quality, since it is not restricted to preserving previous output. it has the disadvantages of higher computational cost, and a high revision rate, visible as textual instability in an online translation display. when revisions are an option, it is unclear whether one should prefer a specialized streaming model or a re-translation strategy. in light of this, we make the following contributions: (1) we evaluate a combination of retranslation techniques that have not previously been studied together. (2) we provide the first empirical comparison of re-translation and streaming models, demonstrating that re-translation operating in a very low-revision regime can match or beat the quality-latency trade-offs of streaming models. (3) we test a 0-revision configuration of re-translation, and show that it is surprisingly competitive, due to the effectiveness of data augmentation with prefix pairs. re-translation was originally investigated by @xcite , and more recently extended by @xcite , who propose a suitable evaluation framework, and use it to assess inference-time re-translation strategies for speech translation. we adopt their inference-time heuristics to stabilize re-translation, and extend them with prefix training from @xcite . where they experiment on ted talks, compare only to vanilla re-translation and use proprietary nmt, we follow recent work on streaming by using wmt training and test data, and provide a novel comparison to streaming approaches. ## metrics we adapt the evaluation framework from @xcite , which includes metrics for latency, stability, and quality. where they measure latency with a temporal lag, we adopt an established token lag that does not rely on machine speed. our evaluation is built around a prefix translation list (ptl), which can be generated for any streaming or re-translation system. for each token in the source sentence (after merging subwords), this list stores the tokenized system output. ## re-translation methods to evaluate re-translation, we build up the source sentence one token at a time, translating each resulting source prefix from scratch to construct the ptl for evaluation. ## experiments we use standard wmt14 english-to-french (enfr; 36.3m sentences) and wmt15 german-to-english (deen; 4.5m sentences) data. for enfr, we use newstest 2012+2013 for development, and newstest 2014 for test. for deen, we validate on newstest 2013 and report results on newstest 2015. we use bpe @xcite on the training data to construct a 32k-type vocabulary that is shared between the source and target languages. ## conclusion we have presented the first comparison of retranslation and streaming strategies for simultaneous translation. we have shown re-translation with low levels of erasure (@xmath0.2) to be as good or better than the state of the art in streaming translation. also, re-translation easily embraces arbitrary improvements to nmt, which we have highlighted with large gains from an upgraded base model. in our setting, re-translation with no erasure reduces to wait-k inference, which we have shown to be much more effective than previously reported, so long as the underlying nmt system's training data has been augmented with prefix pairs. due to its simplicity and its effectiveness, we suggest retranslation as a strong baseline for future research on simultaneous translation.