Simultaneous Neural Machine Translation with Prefix Alignment (2022)

Abstract: Simultaneous translation is a task that requires starting translation before the speaker has finished speaking, so we face a trade-off between latency and accuracy. In this work, we focus on prefix-to-prefix translation and propose a method to extract alignment between bilingual prefix pairs. We use the alignment to segment a streaming input and fine-tune a translation model. The proposed method demonstrated higher BLEU than those of baselines in low latency ranges in our experiments on the IWSLT simultaneous translation benchmark.

## introduction simultaneous machine translation (simulmt) is a task to start outputting translation before observing the whole input sentence. simulmt is more difficult than the translation with the whole input sentence because it cannot use the latter part of the sentence as context. simulmt has to decide whether to wait for more input or to output partial translation using the input so far, in real-time. the translation quality should become better if we can use longer inputs and vice versa. we have to handle such a trade-off between the quality and latency of the translation by decision policies to choose the next action between read (waiting for the next input segment) and write (outputting a translation segment) for a given input-output history @xcite . neural machine translation (nmt) models used for simulmt can be roughly categorized into policy-dependent and policy-independent. a policy-dependent model is trained with the constraints given by the policy, in order to translate an input prefix into an output prefix. @xcite proposed a simple method with a fixed policy called wait-k, where the nmt first takes k read actions followed by alternating write and read actions until the end of the translation output. @xcite proposed a joint training framework for flexible policies and the corresponding nmt model using a latency-augmented loss function and monotonic infinite lookback (milk) attention. in contrast, a policy-independent model is a standard nmt model to translate the whole input into the whole output and used for simulmt along with a given policy in the inference. we can share one nmt model for different policies, so the quality-latency trade-off can be controlled easily. @xcite achieved some latency reduction with a small loss in bleu by the use of a fixed policy called static-rw. @xcite also applied their wait-k policy using a sentencebased nmt model, called test-time wait-k. @xcite proposed a flexible policy to predict segment boundaries in an input. once a boundary is found, the segment is translated using a sentence-based nmt model. the model based on their segmentation demonstrated better results in quality-latency trade-off than those using wait-k and milk in chinese-to-english simulmt. @xcite proposed another flexible policy using simple rules with syntactic constituent label prediction and showed better performance than mubased simulmt in english-to-japanese. one problem in the use of a policy-independent model in simulmt is the difference between training and inference conditions; the nmt model is trained in the sentence level but is used to translate the prefix of a sentence in inference. this causes unexpectedly long translation and hurts the quality of simulmt @xcite . to mitigate the problem, we propose a method for data augmentation to fine-tune a policy-independent nmt model to the problem of prefix-to-prefix translation, called bilingual prefix alignment. we use a pre-trained sentence-based nmt model to align source language prefix and target language prefix of sentences in the training corpus and collect prefix translation pairs. the proposed method demonstrated higher bleu than baselines in low latency ranges, in our simulmt experiments using iwslt english-to-japanese and english-to-german datasets. ## related work the problem of simulmt has been tackled for a decade. in early attempts using statistical machine translation, decision policies were combined with the beam search decoding @xcite . @xcite used phrase reordering probabilities used in phrasebased statistical machine translation for their decision policy. in later years, feature-based learned policies were proposed. @xcite proposed a feature-based policy optimization to maximize bleu. syntactic features also successfully used for the policies @xcite . recently, most simulmt studies are based on nmt, and such methods can output more fluent translation than before. among nmt-based simulmt studies, one major approach is to train an nmt model optimized for given or jointly-learned policies. wait-k @xcite is a very simple fixed policy that waits for k input tokens first. @xcite proposed an ensemble of different wait-k-based models for adaptive simulmt. to make the policies more flexible, latency-augmented loss functions are used to jointly optimize accuracy and latency in the training of the simulmt model @xcite @xcite . another approach employs such policies only in inference, using a standard sentence-based nmt model. fixed policies can be applied to this approach easily @xcite . @xcite proposed greedy decoding with policies conditioned by the decoder's prediction, called wait-if-worse and wait-if-diff. @xcite proposed a rule-based policy using incremental prediction of the syntactic constituents. to learn segmentation policies from the bilingual corpus, reinforcement learning-based methods were proposed @xcite @xcite . it is a straightforward way to optimize latency and accuracy jointly, but its training process is relatively complex and sometimes unstable. instead of the joint learning of a segmentation policy and policy-dependent model, @xcite proposed a method to find oracle read and write actions using a pre-trained nmt model. zhang et al. (2020) this work is motivated by @xcite and @xcite and extends them with bilingual prefix alignment using a pre-trained nmt model. our method finds appropriate segment boundaries based on the similarity between reference and translation hypothesis for given prefix segments in a different way from @xcite . we also fine-tune the pre-trained nmt model using the bilingual prefix pairs, which is a more sophisticated way than @xcite foot_0 . ## simultaneous machine translation a sentence-level nmt is formulated as follows, letting @xmath0, x 2 , ..., x n be an input sentence and @xmath1, y 2 , ..., y m be its translation: simulmt takes a prefix of the input for its incremental decoding, formulated as follows: in this work, we use chunk-based incremental decoding @xcite , in which we translate an input prefix from the beginning. it is similar to an approach called re-translation @xcite ), but we force the decoder to follow already translated output prefixes in the same way as the teacher forcing in nmt training. step 1 i step 2 i bought step 3 i bought a step 4 i bought a pen 0.9 > 0.5 私は 0.2 < 0.5 0.3 < 0.5 0.7 > 0.5 私はペンを買った read source words boundary prediction translation step 5 i bought a pen . 0.7 > 0.5 私はペンを買った。 figure 1: the translation process of the proposed method from english to japanese. the threshold of boundary probability is 0.5 in this case. the underlined part is the forced output prefix. pre-trained nmt model and a bilingual corpus for fine-tuning the model for simulmt. the proposed method consists of the following steps: 1. collect prefix translation pairs using the pretrained model 2. find reference prefixes corresponding to the prefix translation pairs 3. train a boundary prediction model 4. fine-tune the nmt model their details are described in the following subsections. ## experimental setup we conducted experiments on english-to-german (en-de) and english-to-japanese (en-ja) simultaneous translation to compare the proposed method with the baselines in the quality-latency trade-off. ## analysis 7.1 effect of pa-based nmt fine-tuning for the detailed analyses, we investigated the performance of the chunk-based simulmt without the fine-tuning using the bilingual prefix pairs. here, only the boundary predictor was used to segment the input for the chunk-based simulmt. figures 7 , 8 , 9, and 10 show the results by the proposed method with the pre-trained nmt model (paoff-0 and paoff-1). they clearly show 0.0 2.5 5.0 7.5 10.0 12.5 15.0 17.5 al 1.0 1.2 1.4 1.6 1.8 2.0 ratio pa-off-0 pa-off-1 wait-k mu-0 mu-1 iclp full-sentence figure 8: length ratio and average lagging (en-de) without pa-based nmt fine-tuning the proposed method does not work well without fine-tuning the nmt model; it resulted in a longer translation length so bleu decreased due to the brevity penalty. these results suggest the segmentation policy in the chunk-based simulmt should match the prefix translation models because a fullsentence translation model often generates a toolong translation result for a short prefix input. ## conclusion we proposed a method to train the neural simulmt model by extracting bilingual prefix pairs by prefix alignment. the proposed method outperformed the baselines in quality-latency trade-off in englishto-german simultaneous translation but showed mixed results in english-to-japanese. we investigated the results in detail and found the difference in the translation length made a large effect on the results, caused by the performance of the sentencelevel nmt model and the word order difference. in future work, we extend the method to work for language pairs with the large word order differences such as english-japanese, in the wide range of al. the proposed method to extract source prefixes can be adapted to speech input. we applied this method to speech-to-text simultaneous machine translation system submitted to the iwslt 2022 evaluation campaign @xcite .