Enhanced Simultaneous Machine Translation with Word-level Policies (2023)

Abstract: Recent years have seen remarkable advances in the field of Simultaneous Machine Translation (SiMT) due to the introduction of innovative policies that dictate whether to READ or WRITE at each step of the translation process. However, a common assumption in many existing studies is that operations are carried out at the subword level, even though the standard unit for input and output in most practical scenarios is typically at the word level. This paper demonstrates that policies devised and validated at the subword level are surpassed by those operating at the word level, which process multiple subwords to form a complete word in a single step. Additionally, we suggest a method to boost SiMT models using language models (LMs), wherein the proposed word-level policy plays a vital role in addressing the subword disparity between LMs and SiMT models. Code is available at https://github.com/xl8-ai/WordSiMT.

## introduction simultaneous machine translation (simt) commences the translation process while simultaneously receiving the input, making it an effective approach for applications that require minimal latency such as simultaneous interpretation or live broadcast. the development of a novel policy is central to research efforts in simt. this policy dictates the translation process by determining whether to execute a read or write action at each step of the process. neural simt models, like offline neural machine translation (nmt) models, commonly employ byte pair encoding (bpe) @xcite or similar techniques to encode an input sentence into a sequence of tokens. typically, a single read or write action of a simt policy is responsible for handling an encoded token, which may sometimes be a word but often a subword. the development of bpe-based simt models and their policies has resulted in researchers focusing on working at the subword level. the performance analysis and implementation of many simt systems, to the best of our knowledge, have been carried out on encoded sequences of source and target subwords, rather than on the original source and target sentences 1 . this has led to two critical issues that need to be addressed. the first issue is the lack of a standardized tokenization and encoding scheme, meaning that different implementations may employ varying token sequences to encode identical text. this variability can impact latency evaluation results and complicate score comparisons across different systems. the second issue is the missed opportunity to process more source tokens before writing each target token without added latency. for a bpe-based simt model, the input must be received on a wordby-word basis to ensure proper encoding of each word into a sequence of subwords. consequently, when the model encodes a word and performs a read to process only a subword, it delays the reading of the remaining subwords without any benefit in actual latency foot_1 , and may adversely impact t rem end ously▁ challenging .▁ le▁ de▁ coût▁ t rem end ement▁ difficile .▁ réponse▁ était▁ très▁ difficile .▁ t rem end ously▁ challenging .▁ token-level (@xmath0, token-@xmath1.38, word-@xmath2.05) word-level (@xmath3, token-@xmath4.4, word-@xmath5.0) src: tgt: src: tgt: figure 2 : an exemplary case depicting the difference between token-level and word-level wait-k policies and their average lagging (al) scores. the token-level model begins translating in the middle of reading "tremendously" and fails to recover from the incorrectly translated target prefix. on the other hand, the word-level model processes "tremendously" in a single read action and produces a correct translation. in this paper, we show that establishing the unit of measuring and operating simt systems at the word level, rather than the subword level, offers a viable solution to these issues. specifically, to tackle the first issue, we propose word-level latency metric calculation for measuring the latency of simt systems. this not only enables consistent comparisons between different systems but also provides a more accurate reflection of the actual latency experienced in simt applications that display translation results word by word. to address the second issue, we illustrate that an existing token-level policy can be transformed into a word-level policy that inherently overcomes the second issue, resulting in improved performance. word-level policies take into consideration word boundaries and perform a read or write action to sequentially process a sequence of tokens that form a complete word. this conversion process can be applied to any token-level policies, and our experiments reveal that state-of-the-art fixed and adaptive policies exhibit significantly better performance when transformed into their word-level counterparts. notably, these word-level policies often outperform token-level policies, even when evaluated using a token-level latency metric, due to their enhanced utilization of input and output tokens. additionally, to boost translation accuracy, we suggest incorporating a pre-trained language model (lm) into a simt model, where the word-level policy plays a crucial role as a pivotal component. one of the major hurdles in utilizing an lm for simt is the vocabulary mismatch between the simt model and the lm. the difficulty of handling subword disparities when utilizing an lm for a downstream task is widely acknowledged @xcite , and it becomes particularly problematic in simt, as inconsistent subwords between the lm and simt model make processing the same source or target prefix challenging. our study demonstrates that our proposed word-level policy effectively tackles this challenge, enabling a successful integration of lms into simt systems. ## proposed methods in this section, we propose the concept of employing a word-level latency metric and outline our conversion process for translating token-level policies into their word-level equivalents. additionally, we present an integration of lm into simt, highlighting the advantages of utilizing word-level policies. ## analysis to validate the effectiveness of word-level policies from multiple angles, we conduct several analyses on various settings. all the experiments were conducted on wmt de.en with transformer-big unless specified otherwise. ## conclusions this paper explores the potential benefits of wordlevel operations in simt systems. we propose word-level latency calculation to ensure fair and accurate latency comparisons. we introduce a conversion process that transforms token-level policies into word-level policies, enabling the processing of multiple subwords that form a word within a single read or write action. additionally, we propose the integration of lm-fused attention, which combines an autoregressive lm model into simt models with word-level policies. experimental results demonstrate the superiority of word-level policies compared to token-level policies, as well as the effectiveness of the lm integration. our findings highlight the crucial role of word-level policies in the integration process.