Anticipation-Free Training for Simultaneous Machine Translation (2022)

Abstract: Simultaneous machine translation (SimulMT) speeds up the translation process by starting to translate before the source sentence is completely available. It is difficult due to limited context and word order difference between languages. Existing methods increase latency or introduce adaptive read-write policies for SimulMT models to handle local reordering and improve translation quality. However, the long-distance reordering would make the SimulMT models learn translation mistakenly. Specifically, the model may be forced to predict target tokens when the corresponding source tokens have not been read. This leads to aggressive anticipation during inference, resulting in the hallucination phenomenon. To mitigate this problem, we propose a new framework that decompose the translation process into the monotonic translation step and the reordering step, and we model the latter by the auxiliary sorting network (ASN). The ASN rearranges the hidden states to match the order in the target language, so that the SimulMT model could learn to translate more reasonably. The entire model is optimized end-to-end and does not rely on external aligners or data. During inference, ASN is removed to achieve streaming. Experiments show the proposed framework could outperform previous methods with less latency.

## introduction simultaneous machine translation (simulmt) is an extension of neural machine translation (nmt), aiming to perform streaming translation by outputting the translation before the source input has ended. it is more applicable to real-world scenarios such as international conferences, where people could communicate fluently without delay. however, simulmt faces additional difficulties compared to full-sentence translation -such a model needs to translate with limited context, and the different word order between languages would training target reordered output 他 在下午 打了個盹 他 在下午 打了個盹 (he) (in the afternoon) (took a nap) output input 他 在下午 打了個盹 @xcite (in the afternoon) (took a nap) he took a nap in the afternoon ## proposed method for a source sentence @xmath0, x 2 , ..., x |x| ⟩ and a target sentence @xmath1, y 2 , ..., y |y| ⟩, in order to perform simulmt, the conditional probability of translation p(y|x) is modeled by the prefix-toprefix framework @xcite . formally, ## conclusion we proposed a framework to alleviate the impact of long-distance reordering on simultaneous translation. we apply our method to the ctc model and show that it improves the translation quality and latency, especially english to chinese translation. we verified that the asn indeed learns the correct alignment between source and target. besides, we showed that a single encoder can perform simultaneous translation with competitive quality in low latency settings and enjoys the speed advantage over wait-k transformer.