Translation-based Supervision for Policy Generation in Simultaneous Neural Machine Translation (2021)

Abstract: In simultaneous machine translation, finding an agent with the optimal action sequence of reads and writes that maintain a high level of translation quality while minimizing the average lag in producing target tokens remains an extremely challenging problem. We propose a novel supervised learning approach for training an agent that can detect the minimum number of reads required for generating each target token by comparing simultaneous translations against full-sentence translations during training to generate oracle action sequences. These oracle sequences can then be used to train a supervised model for action generation at inference time. Our approach provides an alternative to current heuristic methods in simultaneous translation by introducing a new training objective, which is easier to train than previous attempts at training the agent using reinforcement learning techniques for this task. Our experimental results show that our novel training method for action generation produces much higher quality translations while minimizing the average lag in simultaneous translation.

## introduction simultaneous machine translation focuses on the real-time translation of the stream of utterances in the source language to the target language. the essence of simultaneous translation imposes a trade-off between translation quality and the delay in the delivery of the translated utterances. finding the optimal segments in the input stream is one important task to balance the delay and the translation quality. segmentation based on the input sentence structure @xcite @xcite has been explored in previous work. the other approach to reach the optimal segmentation strategy (policy) is to define the segmentation problem in reinforcement learning framework @xcite @xcite in which the segmentation agent chooses among two possible actions of read (waiting to receive more input) and write (producing an output based on the available input inventory). the segmentation agents can either use a fixedlatency policy @xcite or use an adaptive policy @xcite . in the former, the agent waits for receiving a certain number of input tokens and performs pairs of one write and one read afterward. in the latter, the agent looks for a specific signal in input to permit a write action; a signal that might come from a parsing feature @xcite , a stochastic classifier @xcite , or the attention module of the translation model @xcite . recently, imitation learning has been considered to train adaptive policies. this thread of research focuses on designing supervised oracle agents that can compute the optimal sequence of read/write actions; a sequence that leads the model to produce the most similar translation to that of an offline translation model 1 . @xcite and @xcite use pairs of (input, reference) sentences to train the oracle agent. in this work, we create such an oracle agent using a fully trained neural machine translation model. we do not use reference translations to improve our agent and the translation model is not fine-tuned based on the performance of the agent. in spite of that, we are successful at lowering the latency of simultaneous translation below all previous methods across many different language pairs while retaining a competitive translation quality. our results also demonstrate the applicability of our policy in augmenting existing simultaneous neural translation approaches and improving their 1 a model that waits to receive all the input stream tokens and then starts to translate. such a model can provide more accurate translations in comparison to simultaneous translation model that performs the task using partial and incomplete information. table above. the reference action sequence of reads (r) and writes (w) can be created based on the occupied cells in each column. r i want to study computer science w ich möchte informatik studieren figure 1: example of partial translations table. our oracle action trajectory is indicated by the red dash line and the green cells are the words that our policy choose to write. the subset of input words at each row is extended @xmath0. the remaining parts of the paper are as follows. in section 2, we formally define our imitation learning problem and describe our solution to it. in section 3, we examine our provided solution, and in section 4, we examine the solution and provide experimental results and the analysis of our results. section 5 compares our work to the related work and section 6 concludes our work. ## supervised approach in simultaneous machine translation, we aim to receive the input sequence @xmath1, . . . , x j } incrementally and to transform it to the translated tokens @xmath2, . . . , y i } as accurately and as fast as possible (by producing the output while reading the input). our supervised framework contains two main components: the interpreter which takes subsets of the input sequence x @xmath3, . . . , x j } and generates partial translations y @xmath4, . . . , y j m j } foot_0 ; and the agent which decides whether to send the next input subset x j+1 to the interpreter or not, based on the currently generated output tokens (and possibly other useful information) which represents the state of the interpreter. ## results and analysis we compare the performance of our system against two baselines: for both of these settings, we will use the evalwait-k @xcite policy, where each written word is exactly k words behind the source side. we compare the performance of our model against the state-of-the-art (sota) in section 4.2. ## conclusion we present a novel idea for generating optimal segments in simultaneous translation by comparing the output of a simultaneous system for translation with an offline translation model. this provides us with oracle action sequences that we can use to train an agent used to produce read and write actions for simultaneous translation. our experimental results show that by using an offline translation component, our agent can generate better policies in terms of translation quality and delay compared to our baselines. we also show that our agent can be trained by previously proposed translation components and generate better policies compared to what they have reported before. i want to study computer @xmath5 figure 5 : the first table shows an example of our oracle action sequence. in the middle table, we switch the second write action to a read action. we will continue reading until we observe the same word, then we can come back to the original path. in this scenario, the final translation does not change. in the last table, we distort the 5th read into a write action. here, the generated translation is slightly changed as we are writing the wrong word.