Wait-info Policy: Balancing Source and Target at Information Level for Simultaneous Machine Translation (2022)

Abstract: Simultaneous machine translation (SiMT) outputs the translation while receiving the source inputs, and hence needs to balance the received source information and translated target information to make a reasonable decision between waiting for inputs or outputting translation. Previous methods always balance source and target information at the token level, either directly waiting for a fixed number of tokens or adjusting the waiting based on the current token. In this paper, we propose a Wait-info Policy to balance source and target at the information level. We first quantify the amount of information contained in each token, named info. Then during simultaneous translation, the decision of waiting or outputting is made based on the comparison results between the total info of previous target outputs and received source inputs. Experiments show that our method outperforms strong baselines under and achieves better balance via the proposed info.

## introduction simultaneous machine translation (simt) @xcite @xcite outputs the translation while receiving the source sentence, aiming at the trade-off between translation quality and latency. therefore, a policy is required for simt to decide between waiting for the source inputs (i.e., read) or outputting translations (i.e., write), the core of which is to wisely balance the received source information and the translated target information. when the source information is less, the model should wait for more inputs for a high-quality translation; conversely, when the translated target information is less, the model should output translations for a low latency. existing simt policies, involving fixed and adaptive, always balance source and target at the token level, i.e., treating each source and target token equally when determining read/write. fixed policies decide read/write based on the number of received source tokens @xcite , such as wait-k policy @xcite simply considers each source token to be equivalent and lets the target outputs always lag the source inputs by k tokens, as shown in figure 1 (a). fixed policies are always limited by the fact that the policy cannot be adjusted according to complex inputs, making them difficult to get the best trade-off. adaptive policies predict read/write according to the current source and target tokens @xcite and thereby get a better trade-off, but they often ignore and under-utilize the difference between tokens when deciding read/write. besides, existing adaptive policies always rely on complicated training @xcite or additional labeled data @xcite @xcite , making them more computationally expensive than fixed policies. treating each token equally when balancing source and target is not the optimal choice for simt policy. many studies have shown that different words have significantly different functions in translation @xcite @xcite , often divided into content words (i.e., noun, verb, • • • ) and function words (i.e., conjunction, preposition, • • • ), where the former express more important meaning and the latter is less informative. accordingly, tokens with different amounts of information should also play different roles in the simt policy, where more informative tokens should play a more dominant role because they bring more information to simt model @xcite . therefore, explicitly differentiating various tokens rather than treating them equally when determining read/write will be beneficial to developing a more precise simt policy. in this paper, we differentiate various source and target tokens based on the amount of information they contain, aiming to balance received source information and translated target information at the information level. to this end, we propose wait-info policy, a simple yet effective policy for simt. as shown in figure 1 (b), we first quantify the amount of information contained in each token through a scalar, named info, which is jointly learned with the attention mechanism in an unsupervised manner. during the simultaneous translation, read/write decisions are made by balancing the total info of translated target information and received source information. if the received source information is more than translated target information by k info or more, the model outputs translation, otherwise the model waits for the next input. experiments and analyses show that our method outperforms strong baselines and effectively quantifies the information contained in each token. ## related work simt policy recent policies fall into fixed and adaptive. for fixed policy, @xcite proposed wait-k policy, which first read k source tokens and then read/write one token alternately. @xcite proposed an efficient multipath training for wait-k policy to randomly sample k during training. @xcite proposed future-guide training for wait-k policy, which introduces a full-sentence mt to guide training. @xcite proposed a char-level wait-k policy. @xcite proposed a mixtureof-experts wait-k policy to develop a universal simt model. for adaptive policy, @xcite trained an agent to decide read/write via reinforcement learning. arivazhagan et al. (2019) proposed milk, which predicts a bernoulli variable to determine read/write. ma et al. (2020) proposed mma to implement milk on transformer. (zhang and feng, 2022c) proposed dual-path simt to enhance mma with dual learning. zheng et al. (2020) developed adaptive wait-k through heuristic ensemble of multiple wait-k models. miao et al. (2021) proposed a generative framework to generate read/write decisions. zhang and feng (2022a) proposed gaussian multi-head attention to decide read/write based on alignments. previous policies always treat each token equally when determining read/write, ignoring the fact that tokens with different amounts of information often play different roles in simt policy. our method aims to develop a more precise simt policy by differentiating the importance of various tokens when determining read/write. information modeling in nmt linguistics divides words into content words and function words according to their information and functions in the sentence. therefore, modeling the information contained in each word is often used to improve the nmt performance. @xcite and @xcite used the word frequency to indicate how much information each word contains, and the words with lower frequencies contain more information. @xcite and @xcite found that the norm of word embedding is related to the token information in nmt. @xcite and @xcite argued that the attention mechanism for different types of word should be different, where the attention distribution of content word tends to be more concentrated. our method explores the usefulness of modeling information for simt policy, and proposes an unsupervised method to quantify the information of tokens through the attention mechanism, achieving good explainability. with source length n and the target sentence as @xmath0, • • • , y m ) with target length m. transformer @xcite is the most widely used architecture for full-sentence mt, consisting of an encoder and a decoder. encoder maps x to source hidden states @xmath1, • • • , z n ). decoder maps y to target hidden states @xmath2, • • • , s m ), and then performs translating. specifically, each encoder layer contains two sub-layers: self-attention and feed-forward network (ffn), while each decoder layer contains three sub-layers: self-attention, cross-attention and ffn. both self-attention and cross-attention are implemented through the dotproduct attention between query q and key k, calculated as: to differentiate various tokens when determining read/write, we quantify the amount of information contained in each source and target token, named info. as shown in figure 2 , we propose infoaware transformer to jointly learn the quantified info with the attention mechanism in an unsupervised manner. then based on the quantified info, we propose wait-info policy to balance the received source information and translated target information. the details are as follows. info quan)zer inputs emb. & pos. emb. & pos. outputs (shifted right) linear so1max output probablities source info target info info quan)zer info-aware self-a6en7on feed forward info-aware self-a6en7on info-consistent cross-a6en7on ## analysis we conduct extensive analyses on wait-info policy. unless otherwise specified, all results are reported on @xmath3. ## case study to study the specific improvement of the proposed wait-info policy compared to the wait-k policy, we conduct a case study in figure 8 . in wait-k, the model is forced to wait for a fixed 5 tokens be-fore translating, which makes the model either too aggressive or too conservative in different cases @xcite . as shown in this case, at the beginning of translation, when translating 'grahams', 2 source tokens are enough to translate, but wait-k policy forces the model to wait for 5 tokens, resulting in unnecessary waiting. when translating the noun 'surgery', the model should have waited until receiving 'operation', but the model was forced to output in advance, resulting in the wrong translation 'educated' (marked in green). in wait-info, this weakness is ameliorated by quantifying the information in each token rather than considering each token equally. first of all, we find the proposed info can effectively distinguish different tokens, where the content words often get larger info, such as 'sich', @xcite in english, thereby being more important to the simt policy. owing to the quantified info, when translating the 'surgery', the model recognized that the previous 'der' (i.e., determiner in german) does not contain enough info, so the model continues to wait for the 'operation' and thereby generates the correct translation 'surgery' (marked in red). overall, in wait-info policy, tokens with larger info, such as verbs and nouns, play a more important role in the model's decision of read/write, making it easier to ensure that those content words are read before translating. ## conclusion in this paper, we quantify the information in tokens and propose a wait-info policy accordingly. experiments show the superiority of our method on simt tasks and good explainability of the quantified info.