Enhanced Simultaneous Machine Translation with Word-level Policies
Authors: Kang Kim
Hankyu Cho
Venue: EMNLP 2023 Findings (2023)
Keywords: Simultaneous Machine Translation
word-level policies
TL;DR: Word-level policies are better than token-level policies in many aspects for Simultaneous Machine Translation.

Abstract: Recent years have seen remarkable advances in the field of Simultaneous Machine Translation (SiMT) due to the introduction of innovative policies that dictate whether to READ or WRITE at each step of the translation process. However, a common assumption in many existing studies is that operations are carried out at the subword level, even though the standard unit for input and output in most practical scenarios is typically at the word level. This paper demonstrates that policies devised and validated at the subword level are surpassed by those operating at the word level, which process multiple subwords to form a complete word in a single step. Additionally, we suggest a method to boost SiMT models using language models (LMs), wherein the proposed word-level policy plays a vital role in addressing the subword disparity between LMs and SiMT models. Code is available at https://github.com/xl8-ai/WordSiMT.

Enhanced Simultaneous Machine Translation with Word-level Policies
Kang Kim", Hankyu Cho"
XL8 Inc. (kai, matthew)@x18. ai
Abstract
sec
end
chillaningg
Recent years have remarkable advances in the field of Simultaneous Machine Transla- tion (SiMT) due the introduction of inno- vative policies that dictate whether to READ or WRITE at each step of the translation pro- cess. However, common assumption in many existing studies is that operations are carried out at the subword level, even though the stan- dard unit for input and output in most prac- tical scenarios is typically at the word level. This paper demonstrates that policies devised and validated at the subword level are sur- passed by those operating at the word level, which process multiple subwords form complete word in single step. Additionally, we suggest a method to boost SiMT models using language models (LMs),, wherein the proposed word-level policy plays vital role addressing the subword disparity between LMs and SiMT models. Code is available https://g/ thub. com/x18-oa 18-ai ai/MordSiT..
dimicle
Figure I: Illustration of both token-level Wait-1 (red arrow lines) and word-level Wait-1 (green arrow lines) policies. The symbol at the end of token indi- cates a word boundary.
The development of BPE-based SiMT models and their policies has resulted researchers focus- ing on working at the subword level. The perfor- mance analysis and implementation of many SiMT systems, to the best of our knowledge, have been carried out on encoded sequences of source and target subwords, rather than on the original source and target sentences! This has led two critical issues that need be addressed.
1 Introduction
The first issue is the lack of standardized tok- enization and encoding scheme, meaning that dif- ferent implementations may employ varying token sequences to encode identical text. This variability can impact latency evaluation results and compli- cate score comparisons across different systems. The second issue is the missed opportunity process more source tokens before writing each tar- get token without added latency. For BPE-based SiMT model, the input must received on a word- by-word basis to ensure proper encoding of each word into a sequence of subwords. Consequently, when the model encodes word and performs a READ to process only subword, it delays the reading of the remaining subwords without any ben- efit in actual latency2. and may adversely impact
Simultaneous Machine Translation (SiMT) com- mences the translation process while simultanc- ously receiving the input, making effective ap- proach for applications that require minimal latency such as simultaneous interpretation or live broad- cast. The development of novel policy is central to research efforts in SiMT. This policy dictates the translation process by determining whether to execute READ WRITE action each step of the process.
Neural SiMT models, like offline Neural Ma- chine Translation (NMT) models, commonly em- ploy Byte Pair Encoding (BPE) (Sennrich et al., 2016) or similar techniques to encode an input sen- tence into sequence of tokens. Typically, a single READ or WRITE action of SiMT policy is re- sponsible for handling encoded token, which may sometimes be word but often subword.
'We provide a list reference works pertaining to this case in "Here we assume that the prncessing time of READ is negligible, compared to the interval between receiving words.
"qqual contribution.

---

Token-level (le-3, token-AL-3.38, word-AL-1.05)
Word-level [l==1
token-AL=2.4 -L-1-A
SRC:
ouslyy....
challenging
rem
end
ously -
challenging
TGT:
Réponse
difficlle
and
ement
difficle
Figure 2: An exemplary case depicting the difference between token-level and word-level Wait-k policies and their Average Lagging (AL) scores. The token-level model begins translating in the middle reading "Tremendously" and fails recover from the incorrecll translated target prefix. On the other hand, the word-level model processes "Tremendously" in single READ action and produces correct translation.
translation quality by causing the model to rely on incomplete representations extracted from partially- read words. Similarly, performing a WRITE to generate a subword earlier than the remaining sub- words does not necessarily reduce latency, as the subword must wait for complete word to be gen- erated before it can be displayed.
and the LM. The difficulty of handling subword disparities when utilizing an LM for downstream task is widely acknowledged (Liu et al., 202 la Wang et al., 2022), and it becomes particularly problematic in SiMT, as inconsistent subwords be- tween the LM and SiMT model make processing the same source target prefix challenging, Our study demonstrates that our proposed word-level policy effectively tackles this challenge, enabling successful integration of LMs into SiMT systems.
In this paper, show that establishing the unit of measuring and operating SiMT systems at the word level, rather than the subword level, offers a viable solution to these issues. Specifically, to tackle the first issue, we propose word-level la- tency metric calculation for measuring the latency of SiMT systems. This not only enables consistent comparisons between different systems but also provides more accurate reflection of the actual latency experienced in SiMT upplications that dis- play translation results word by word.
Related Work
2.1 Simultaneous Machine Translation
SiMT systems that employ a fixed policy utilize a pre-defined sequence READ and WRITE opera- tions for each source sentence. STATIC-RW (Dalvi et al., 2018) and Wait-k (Ma et al., 2019a) policies first read k source tokens, then alternate between reading and writing single token. Elbayad et al. (2020) propose the multi-path training of Wait-k model train a single model that supports different values at test time. Zhang et al. (2021) improve Wait-k policy using knowledge distillation from an offline MT model, while Zhang and Feng (2021) suggest a Mixture-of- predictions from multiple k values are combined inside a single model.
To address the second issue, we illustrate that an existing token-level policy can be transformed into a word-level policy that inherently overcomes the second issue, resulting in improved performance. Word-level policies take into consideration word boundaries and perform a READ or WRITE action to sequentially process a sequence of tokens that form complete word. This conversion process can be applied any token-level policies, and our experiments reveal that state-of-the-arr fixed and adaptive policies exhibit significantly better per- formance when transformed into their word-level counterparts. Notably, these word-level policies often outperform token-level policies, even when evaluated using a token-level latency metric, due to their enhanced utilization of input and output tokens.
Wait-k Policy where
In contrast, research efforts adaptive policies focus on the development of dynamic decision- making processes for READ/WRITE actions. Cho and Esipova (2016) firstly introduce model-based adaptive criteria for Neural SiMT. Gu et al. (2017) propose to learn a policy by using reinforcement learning. Raffel et al. (2017) introduce Monotonic Attention that ensures monotonic alignment in the attention mechanism. Succeeding works improve it by extending the alignment window (Chiu and Raf- fel. 2017; Arivazhagan et al., 2019), extending monotonic multi-head attention (MMA) (Ma al., 2019b) or learning transposed policies between the forward and backward models. (Zhang and Feng,
Additionally, to boost translation accuracy, suggest incorporating pre-trained language model (LM) into SiMT model, where the word-level pol- icy plays a crucial role as a pivotal component. One the major hurdles in utilizing an LM for SiMT is the vocabulary mismatch between the SiMT model

---

Given source sentence x (11, 22; ...,,,, the goal of a SiMT model is to generate target sen- tence of y = (1/ (2.m. while minimizing la- tency metries. A SiMT model's policy, represented by the variable gr. determines the number of source tokens to process before predicting target token 2022c). Zheng et al. (2020) derive a policy by com- posing Wait-k models trained with different values of k. Zhang and Feng (2022a) model to predict the alignment between each target token and the source token. Zhang and Feng (2022b) measure accumulated information from source tokens to de- cide whether to write the current target token.
original tokens due token fragmentation. Addi- tionally, their approach falls under target-side LM integration, overlooking the potential advantages of source-side LM integration.
In this paper, we demonstrate an effective way of integrating source-side LM into SiMT systems, offering more versatile solution that can be in- tegrated into most existing neural SiMT models. Building upon previous rescarch conducted in of- fline MT (Zhu al., 2020), we introduce essential modifications, with a particular focus on word-level policies as a pivotal component. The effective man- agement of vocabulary mismatches between the LM and the SiMT model is contingent upon the successful implementation of a word-level SiMT policy, a key aspect that we address in our study.
Despite significant advancements, the impact of operating policies at the word level has not been thoroughly explored in existing works, which have mainly focused on developing and evaluating sys- tems at the token level. In this paper, we address this gap by demonstrating that implementing vari- ous types of policies at the word level consistently outperforms their token-lever counterparts.
2.2 Utilizing pre-trained LM for MT
3 Proposed Methods
Since the successful utilization of Transformer- based LMs pre-trained on large text corpora for downstream NLP tasks (Devlin et al., 2019; Liu al., 2019; Lample and Conneau, 2019), the uti- lization of these models for MT has become sig- nificant research area. Several studies have demon- strated the effectiveness of incorporating encoder- only LMs into NMT models. Weng et al. (2020); Yang et al. (2020) combine the LM representations with the encoder's representation using a gating mechanism. Zhu et al. (2020) propose attention between BERT and both the encoder and decoder. Weng al. (2022) leverage mBERT as an encoder and introduce a decoder that attends to grouped representations the encoder output.
In this section, we propose the concept of employ- ing a word-level latency metric and outline our con- version process for translating token-level policies into their word-level equivalents. Additionally, we present integration of LM into SiMT, highlight- ing the advantages of utilizing word-level policies.
3.1 Preliminaries
Another research direction focuses develop- ing LMS with the encoder-decoder architecture designed for NMT as the target downstream task (Lewis et al., 2020; Liu et al., 2020). These models show improvements particularly for low-resource language pairs. To enhance their adaptation for MT, various methods have been proposed, including fine-tuning specific parts of the LMs (Cooper Stick- land et al., 2021), reducing domain mismatch and overestimation (Wang al., 2022) and mitigating the copying behavior (Liu al., 2021b).
Vi. Then the probability of generating given x is formulated follows:
ly p(ylx) = II S Nzy
where 0 is the model's parameters which are com- monly optimized with cross-entropy loss. Transformer encoder-decoder model (Vaswani et al., 2017) is currently the most widely used ar- chitecture for SiMT. To avoid redundant encod- ing of the input sequence after each READ op- eration, the encoder is typically modified to en- code the source tokens unidirectionally (Elbayad et al., 2020). Alternatively, more advanced tech- niques like the recurrent Linear Transformer (Ka- hardipraja et al., 2021) or Partial Bidirectional En- coding (Iranzo Sanchez et al., 2022) can be adopted to enhance the encoding capabilities further.
The integration of pre-trained LMs into SiMT remains an underexplored area of research. To date, Indurthi et al. (2022) is the only related study are aware of. It improves MMA by integrating the LM's prediction of the next target token, which is encoded using their model's vocabulary before being inputted into the model. However, this ap- proach sacrifices the semantic coherence of the

---

During the evaluation of SiMT systems, transla- tion quality commonly assessed in conjunction with the latency required for generating translations. Various metrics have been proposed to calculate la- tency scores, with Average Lagging (AL) (Ma et al., 2019a) being the most commonly used metric.
X
X
X
X
X;
X
X;
3.2 Measuring latency based on the word level
Token lovel unidiructoral encoding Inbra Lidiractionl encoding
As detailed A.I, a substantial body of prior re- search work assesses the performance of SiMT systems by utilizing a latency metric on encoded source tokens under different tokenization and en- coding schemes. This practice results in each system being evaluated non-identical token se- quences for the same dataset, thereby making challenging to accurately compare scores across different systems.
Figure 3: Comparison of masking in the token-level unidirectional attention (left) and the intra-word bidi- rectional encoding (right). Word boundaries are repre- sented by vertical/horizontal bars on each axis.
decision-makinn criterion while ensuring uninter- rupted reading of an entire word when initiating the reading of token.
Similarly to the word-level READ, we design a word-level WRITE to balance READ and WRITE actions throughout the translation. To achieve this, we can modify such that it writes until it pro- duces token that ends with an end-of-word sym- bol. We define w; that satisfies this as follows:
To address this, we propose word-level latency score calculation by considering the word bound- aries in token-level sequences. Specifically, when the first token of source word is processed through a READ operation, we consider as reading the corresponding word. Similarly, when the last token target word is written via a WRITE operation, we consider it as writing that word. By doing so, the latency scores are calculated consistently, re- gardless of the tokenization and encoding of the input. This ensures that results from different sys- tems be compared fairly.
b; == min 2 inje B
(3)
Tis
if = b b,
W/ ==
(4)
0h-1, otherwise
where By denotes the indices of the target words' last tokens and b; represents the index of the final token in the word that includes yi- By employ- ing wy in place of r; (or gi). ensure that the policy consistently composes entire words with- out interruptions from any READ actions. This approach effectively reduces latency by facilitat- ing faster writing of certain tokens compared to the original policy, thereby compensating for the increased latency resulting from word-level READ operations. Figure provides a visual comparison between word-level and token-level policies in the context of Wait-1, with the word-level policy en- compassing both word-level READ and WRITE operations.
3.3 Word-level SiMT policies
The proposed word-level policy restricts a SiMT policy's 's transition from READ to WRITE or vice versa to occur exclusively the boundaries of words. Any token-level policy can be transformed operate the word-level by following the con- version process we outline below.
Concretely, we ensure that a word-level policy does not write target token in the middle of read- ing sequence of source tokens that make up word. To accomplish this word-level READ, delay gi until reaches the nearest source word boundary. We formally define r; that has a refined value of gi based on the word boundaries in x as follows:
3.4 Intra-word bidirectional encoding
Unidirectional encoding in SiMT is vital for man- aging computational complexity and training effi ciency. However, it has an inevitable consequence of weakening the source sequence representations compared to bidirectional encoding. This is an ad- ditional factor contributing to the lower translation
:= = min 2 9i AjE Bs}
Here, By denotes the indices of the source words' last tokens. Substituting r; for gi as a policy trans- forms into another policy that upholds the same

---

WRITE
WRITE
rem end ement_.
rem end ement_ difficlle
Decoder
Decoder
rem
end
ement
difficile
READ
READ
H
He
H
LM
Encoder
LM
Encoder
Tre mend ously
rem end
ously
Tre mend ously cchallenging
T rem end ously challenging
LM vocab
In-domain vocab
LM vocab
In-domain vocab
Source stream:
Tremendously
Source stream:
challleggngg
Figure 4: Illustration of LM-fused attention with the word-level Wait- specific stage are highlighted in red. When source word is received, it is independently encoded by the LM's vocabulary and the in-domain vocabulary. The hidden activations of the word from the LM is then utilized in the encoder. The decoder generates sequence of tokens for a word by using both the LM and encoder activations.
Word and model activations processed at
quality of SiMT compared to offline models, along with the early translation from partial inputs.
other option to explore methods to bridge vo- cabulary gaps (Kim et al., 2019; Sato et al., 2020; Liu et al., 2021a), but they are either validated only in certain transfer learning scenarios require an additional training phase to train adapters or fine- tuning the entire LM using pre-training data.
To mitigate this issue, we utilize technique called intra-word bidirectional encoding. At the word level, this approach involves unidirectional encoding for each word in the input sentence, mean- ing past words cannot attend to future words. How- ever, at the subword level, subwords within the same word can attend to each other, allowing past subwords to attend to future subwords within the same word. Since READ operates at the word level in word-level policies, this encoding does not re- quire recomputation during each WRITE operation. It only necessitates single forward pass, similar to token-level unidirectional encoding. However, it can produce better encoded representation by en- abling attention to more tokens An example mask- ing enable intra-word bidirectional encoding is depicted in Figure 3.
In this paper, we introduce a method for lever- aging LM in manner that facilitates the inte- gration of an off- the-the-shelf LM into SiMT model, utilizing word-level policy, regardless of vocab- ulary mismatches and the internal structure of the SiMT model. Specifically, we employ an LM fused attention for both the encoder and decoder, follow- ing the approach outlined in (Zhu et al., 2020), but with two notable modifications.
Firstly, we replace BERT with decoder-only auto-regressive LM (Radford et al., 2019; 9; Lin et al., 2022) for unidirectional encoding of the input, aligning with SiMT models for efficient training and inference. Secondly, the attention between the SiMT model and the LM occurs when both mod- els exccute a word-level READ for an input word. This ensures they interact only when they process an equal amount of input prefix, naturally resolv- ing the synchronization issue. Additionally, as they align at every word boundary, the SiMT model can operate independently with vocabulary derived from in-domain data, while the LM continues to use its original vocabulary. Unlike methods target- ing specific SiMT models (Indurthi al., 2022), our approach can benefit any Neural SiMT model with any decoder-only LM. Figure illustrates the proposed integration of the LM with word-level 1
3.5 Integration of LM into SiMT through word-level policies
In this subsection, we showcase an additional ben- efit of word-level policies when integrating an LM into SiMT system. One of the key challenges in this integration is the vocabulary mismatch be- tween the LM and the SiMT model, which hinders ensuring that both models process equal amount of input prefix at each translation step.
One possible solution is to use the LM's vocab- ulary for the SiMT model. However, the LM's training data may not align well with the specific domain targeted by the SiMT system (Wang et al., 2022). This can result in suboptimal vocabulary for the SiMT model compared vocabulary ob- tained from in- nomaiii data (Liu et al., 2021a). la). An-

---

Wirt-k token-ievel
Mel inkerdleed
TTST token-ieed
Wisze wondleeeel
MoC werd-lose
7TST wonddond
Wart wond --weed----e----- LM
Mrl: werd lovel LM
TTST word wond-looee -IM
32
Wod-AL
Wod-AL
Wod.AL
(a) En-Fr Wait-k Transformer small
(b) En-Fr MoF Transformer small
(c) En-Fr ITST Transformer small
80.00
24-
Woi-k wontlevel
Waz-l loken-lvell
MiE ukendevd
MrG word-lor
36
IIST token-leed
ITST ITST wondlevel
Wat-l wonbleeel- LM
MrE word-oord 4
STST wondeleni -M
(d) Dc-En Wait-k Transformer big
(c) De-En MoE Transformer big
De-En ITST Transformer big
Figure 5: Results translation quality v.s. word-level AL.
4 Experiments
fixed Experts weights and then fine-tuned with dy- namic Experts weights. The word-level Wait-k policy of different k is applied to each expert.
4.1 Datasets
We use the following two datasets for experiments: IWSLT 17 English French (En-Fr) (Cettolo et al., 2017) consists of 242k pairs split into 233k training, 890 validation, and 8,597 test pairs. WMT 15 German to English (De-En) comprises 4.5 million pairs. We use newstest2013 (3,000 pairs) for validation and newstest2015 (2,169 pairs) for testing.
ITST (Zhang and Feng, 2022b): A SoTA SiMT model equipped with Information-Tranee Transport- based policy that quantifies information weights from each source the current target token. To imple- ment word-level ITST, we convert the number source tokens required for the first target token each target word into the corresponding number of source words using Equation 2. Once the required number of source words is read, we complete the translation of the word. Additionally, calculate the latency cost at the word level.
4.2 Experimental settings
Our methods are evaluated on the following sys- tems, all of which are based on Transformer (Vaswani et al., 2017) with unidirectional encoding.
We compare each system by training both token- level and word-level models, with and without an LM. For models with an LM,
XGLM-564M (Lin al., 2022) and employ the two-stage train- ing in which we first train the SiMT model with- out the LM and initialize the encoder and decoder from the LM-fused model with the pre-trained weights (Zhu et al., 2020). We also tested the sin- gle stage training where all trainable parameters are trained jointly with the LM from scratch. The difference of these strategies discussed in Sec- tion 5.3. We tokenized and encoded the input using sentencepiece (Kudo and Richardson, 2018) and applied BPE with a vocabulary size of 32k. We use sacreBLEU for BLEU calculation (Post, 2018). For models with token-level policies, we trained
Wait-k (Ma et al., 2019a): A model operating under Wait-k policy. A single model trained for all k values by randomly sampling k during training (Elbayad et al., 2020). For the word-level Wait-k policy, we define it as reading the first k words and then alternating between reading one source word and writing target word.
MoE Wait-k (Zhang and Feng. 2021): A Mixture-of-Experts model initially trained with
"Technically, the word-level policy derived from the token- level Wait-k through the conversion process in Section 3.3 can wait between and & tokens, depending on the input encoding. Therefore, it is not equivalent to the word-level Wait-k policy we define here, which always waits for k words.

---

models with the official implementations** 5 and implemented word-level policies based on these im- plementations. More training details are described in A.2.
20-
25-
WW (aogosod)
4.3 Main results
with bnteaved Infrnetioal cacodag
vilaor inte word
The performance of each system is compared in Figure 5. Notably, when measuring latency at the word level, the word-level policy proves to be highly effective for all three systems across differ- ent latency levels and datasets, resulting in superior performance. The only exception is observed the En-Fr ITST models that demonstrate similar levels of performance. The incorporation of an LM using the proposed LM-fused attention further enhances the performance for all word-level configumm guratuuttinn. This observation highlights the suitability of word- level policies for the LM-fused attention approach and underscores the effectiveness leveraging an LM to enhance SiMT systems.
Word
(a)
Figure 6: Ablation studies for word-level policies. (a): Comparison of word-level Wait-k policies with different policies. (b): Comparison of word-level Wait-k models with and without the intra-word bidirectional encoding.
target tokens without considering word boundaries.
The results are presented in Figure 6 (a). The the word-level policy (WW) consistently outperforms TW across all latency settings. This is attributed to its imbalance between the number of source and target prefixes processed in each step. Addition- ally, WT achieves a minimal AL of approximately 3.6, indicating that is well-suited for scenar- ios that require low latency. Lastly, TkTk shows significantly worse performance than WW, sug- gesting that reading or writing few consecutive tokens without considering semantic boundaries offers no benefits, unlike word-level policies.
Notably, as depicted in Figure 11, the word-level policies also outperform or compete with the token- level policies in token-level latency. This can be attributed the enhanced token representations der the word-level policy, thanks the contextual information provided by all other tokens belonging to the same word for each token.
5 Analysis
5.1.2 Effeets of intra-word bidirectional encoding
To validate the effectiveness of word-level policies from multiple angles, we conduct several analyses on various settings. All the experiments were con- ducted on WMT En with transformer-bie unless specified otherwise.
In order to assess the impact of the proposed intra- word bidirectional encoding, we trained wevd-level Wait-K models with and without it and compared the accuracy of the two models across different AL settings. The results are presented in Figure 6 (b). Remarkably, the model equipped with the intra- word bidircctional encoding consistently achieved higher BLEU scores compared to the model with- out it, across all tested latency settings. This pro- vides strong evidence of the effectiveness of the intra-word bidirectional encoding in enhancing SiMT performance.
5.1
Ablation study
5.1.1
Effects of Word-level READ and WRITE
To gain insights into the functionality of word-level READ and WRITE actions, we trained Wait-k mod- els with various policy settings and conducted per- formance comparison. Specifically, we examined models with the following policy settings:
5.2 Effectiveness of word-level policies for LM In this subsection, we aim to explore the signifi- cance of word-level policies in leveraging LM for SIMT. We compare different configurations based three factors:
WW: word-level READ and WRITE.
TW: token-level READ and word-level WRITE.
WT: word-level READ and token-level WRITE.
TkTk: simpler baseline policy which involves alternating reading k source tokens and writing k
Word vs. Token: The policy type that the model
'Efficient Wait-k: http:://gituu..cm/lllaaaaaaaaaa
'MoE Wait-k: http:/ghhhh.oooo//////////eteete Waitk TTTT: http:://gtthub.cc/ http://gituub.co///m/lpprrs
operates with.
In-domain vocab vs. LM vocab: Whether the

---

32-
30.
28.
11.35
-3
26
24
22
donnas fooothlts
20
WorkAL
Wend-A.
18
Vokee Litanted
8
(a)
Word-AL
Figure 8: Translation accuracy comparison of LM- attention models with different training configurations. (a): En-Fr Transformer small (b) De-En Transformer big
Figure Comparison of models with different LM integration.
model uses an in-domain vocabulary obtained from the in-domain training data or uses the LM's vo- cabulary for the source language. Note that the use of "in-domain vocab" is specific to the "Word" configuration due the vocabulary mismatch. LM-fused attention vs. LM embed: Whether the model incorporates an L.M using the L.-fused attention or replacing the embedding layer of the encoder with the LM's embedding (Xu et al., 2021). The latter approach uses "LM vocab" by design. Figure 7 showcases the results. The models with word-level policies consistently outperform those with token-level policies by a significant mar- gin in both LM-fused attention and LM embed- ding scttings, underscoring the importance of word- level policy adoption for effective LM integration. The top-performing configuration is the proposed vocab-Woo demonstrating that the highest translation accuracy is achieved when the SiMT model operates with in-domain vocabulary. Additionally, it is evident that the "LM embed" approach performs notably worse than the proposed LM-fused attention, further affirming the latter's superiority.
naturally less pronounced compared to XGLM due the difference in model size. Moreover, although models trained using the single-stage training gen- erally exhibit lower performance compared to those trained using the two-stage training. they still out- perform models without the LM for most configu- rations. This indicates that the LM-fused attention is applicable to various types of LMs and remains effective when using the single-stage training strategy. This flexibility allows users to choose training approach and model configuration that aligns best with their desired accuracy goals and computational constraints.
5.4 Policy quality comparison
To assess the accuracy of policies in determining when to read or write, we adopt the methodology of estimating the quality of policy in prior research (Zhang and Feng, 2022b,c; Guo et al., 2022). We measure the quality of a policy by analyzing the proportion of aligned source words received before translating on RWTH De-En alignment dataset
To ensure accurate word-level alignment calcu- lation, we consider an aligned source word is read before writing ground truth (GT) target word if the last token of the source word is read before the first token of the target word is written. The results of this analysis are presented in Figure 9, It is observed that word-level policies, both for ITST and Wait-k, exhibit better alignments across most latency settings. This suggests that word-level policies contribute to revising premature WRITE actions by guiding the model to read the remain- ing tokens of the aligned word, without negatively impacting the model's latency.
Effects of LM-fused attention with various 5.3
LMs and training configurations
To assess the effectiveness and broad applicabil- ity of our proposed LM integration, we conducted experiments on the IWSLT17 En-Fr dataset with two decoder-only LMs of different sizes: the 137M parameter GPT-2 model (Radford et al., 2018) and the XGLM-564M model. Additionally, explore the option of training models in the single train- ing stage instead of the two-stage training. The results, presented in Figure 8, demonstrate that GPT-2 model also exhibits improved performance with fM--uued attention, although their impact is
'hurps//wwwwio.nndd wwwp:/wwiiiiooooa......mmm. aachen.dc/golde machen.deggolddllggmment

---

100
translation. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 1313-1323, Florence, Italy. Association for Computational Linguisties.
Alignnicnts are read
95
90
Ozan Caglayan, Julia Ive, Veneta Haralampieva, Pranava Madhyastha, Loic Barrault, and Lucia Spe- cia. 2020. Simultaneous machine translation with visual context. In Proceedings of the 2020 Confer- ence on Empirical Methods in Natural Language Processing (EMNLP), pages 2350-2366 Online. As- sociation for Computational Linguistics.
K5
WaF-k token level
Wai-k wonl-leve
ITST toleu-Jevel
ISS word-iee
15
4
16
12
Word-AL
Figure 9: The percentage of aligned source words read before the translation of the target words started.
Mauro Ccttolo, Marcello Federico, Luisa Bentivogli, Jan Nichues, Sebastian Stüker, Katsuhito Sudoh, Koichiro Yoshino, and Christian Federmann. 2017. Overview of the IWSLT 2017 evaluation campaign. In Proceedings of the 14th International Conference on Spoken Language Translation, pages 2-14, Tokyo, Japan. International Workshop on Spoken Language Translation.
6 Conclusions
This paper explores the potential benefits of word- level operations in SiMT systems. We propose word-level latency calculation to ensure fair and accurate latency comparisons. We introduce con- version process that transforms token-level policies into word-level policies, enabling the processing of multiple subwords that form word within a single READ or WRITE action. Additionally, we propose the integration L.M-fseed attention, which com- bines an autoregressive LM model into SiMT mod- els with word-level policies. Experimental results demonstrate the superiority of word-level policies compared to token-level policies, well as the effectiveness of the LM integration. Our findings highlight the crucial role of word-level policies in the integration process.
Chung-Cheng Chiu and Colin Raffel. 2017, Mono- tonic chunkwise attention. a7Xiv:1712.05332.
arXiv preprint
Kyunghyun Cho and Masha Esipova. 2016. Can ral machine translation do simultaneous translation? arXiv preprint 1606:::6000000111111
Asa Cooper Stickland, Xian Li, and Marjan Ghazvinine- jad. 2021. Recipes for adapting pre-trained monolin- gual and multilingual models to machine translation. In Proceedings of the 16th Conference of the Euro- pean Chapter of the Association for Computational Linguistics: Main Volume, pages 3440-3453, Online. Association for Computational Linguisties.
Fahim Dalvi, Nadir Durrani, Hassan Sajjad, and Stephan Vogel. 2018. Incremental decoding and training methods for simultancous translation in neural ma- chine translation. In Proceedings of the 2018 Con- ference of the North American Chapter of the Asso- ciation for Computational Linguistics: Human Lan- guage Technologies, Volume (Short Papers), pages 493-499, New Orleans, Louisiana. Association for Computational Linguistics.
7 Limitations
While the proposed word-level policy implementa- tion is widely applicable most existing SiMT sys- tems, it is important to note that systems utilizing languages with a writing style that lacks spaces or other delimiters between words or sentences (e.g., Chinese) are unable to derive benefits from this approach. Furthermore, it is important to consider that while the proposed LM-fused attention proves effective in enhancing translation quality across all latency levels, integrating large LM may ne- cessitate faster compute capability fulfill the low-latency demands of the SiMT task.
Jacob Devlin, Ming-Wer Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language under- standing. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Humun Lunguage Tech- nologies, Volume (Long and Short Papers), pages 4171-4186, Minneapolis, Minnesota. Association for Computational Linguistics.
Maha Elbayad, Laurent Besacier, and Jakob Verbeek. 2020. Efficient Wait-k Models for Simultaneous Ma- chine Translation. In Proc. Interspeech 2020, pages 1461-1465.
References
Naveen Arivazhagan, Colin Cherry, Wolfgang Macherey, Chung-Chere Cheng Chiu, Semih Yavuz, Ruom- ing Pang. Wei Li, and Colin Raffel. 2019. Monotonic infinite lookback attention for simultaneous machine
Jiatao Gu, Graham Neubig, Kyunghyun Cho, and Vic- tor O.K. Li. 2017. Learning to translate in real-time

---

with neural machine translation. In Proceedings of
Guillaume Lample and Alexis Conneau. 2019. Cross- lingual language model pretraining, arXiv preprint arXiv:1901 1901.07291.
the 15th Conference of the European Chapter of the Association for Computational Linguisties: Volume
1, Long Papers, pages 1053-1062, Valencia, Spain. Association for Computational Linguistics.
Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Veselin Stoyanov, and Luke Zettlemoyer. 2020. BART: Denoising
Shoutao Guo, Shaolei Zhang, and Yang Feng. 2022. Turning fixed to adaptive: Integrating post-evaluation into simultaneous machine translation. In Findings of the Association for Computational Linguistics: EMNLP 2022, pages 2264-2277 Abu Dhabi, United Arab Emirates. Association for Computational Lin- guistics.
pre-training for natural language generation, translation, and com- prehension. In Proceedings of the 58th Annual Meet- ing of the Association for Computational Linguistics. pages 7871-7880, Online. Association for Computa- tional Linguistics.
Xi Victoria Lin, Todor Mihaylov, Mikel Artetxe, Tianlu Wang, Shuohui Chen, Daniel Simig, Myle Oit, Na- man Goyal, Shruti Bhosale, Jingfei Du, Ramakanth Pasunuru, Sam Shleifer, Punit Singh Koura, Vishrav Chaudhary, Brian O'Horo, Jeff Wang. Luke Zettle- moyer, Zornitsa Kozareva, Mona Diab, Veselin Stoy- anov, and Xian Li. 2022. Few-shot learning with multilingual generative language models. In Proceed- ings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 9019-9052, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics.
Shoutao Guo, Shaolei Zhang, and Yang Feng. 2023, Learning optimal policy for simultaneous machine translation via binary search. In Proceedings of the 61 Anmual Meeting of the Association for Compu- lational Linguistics (Volume 1: Long Papers), pages 2318-2333, Toronto, Canada. Association for Com- putational Linguistics.
Sathish Reddy Indurthi. Mohd Abbas Zaidi, Beomseok Lee, Nikhil Kumar Lakumarapu, and Sangha Kim. 2022. Language model augmented monotonic atten- tion for simultaneous translation. In Proceedings of the 2022 Conference of the North American Chap- ter of the Association for Computational Linguistics: Human Language Technologies, pages 38-45, Seat- tle, United States. Association for Computational Linguistics.
Xin Liu, Baosong Yang. Dayiheng Liu, Haibo Zhang, Weihua Luo. Min Zhang, Haiying Zhang, and Jin- song Su. 2021a. Bridging subword gaps in pretrain- finetune paradigm for natural language generation. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Lan- guage Processing (Volume 1: Long Papers), pages 6001-6011, Online. Association for Computational Linguistics.
Javier Iranzo Sanchez, Jorge Civera, and Alfons Juan- Císcar. 2022. From simultaneous to streaming ma- chine translation by leveraging streaming history. In Proceedings of the 60th Annual Meeting of the As- sociation for Computational Linguistics ( (Volume 1: Long Papers), pages 6972-6985, Dublin, Ireland. As- sociation for Computational Linguistics.
Xuebo Liu, Longyue Wang, Derek F. Wong, Liang Ding, Lidia S. Chao, Shuming Shi, and Zhaopeng Tu. 202 On the copying behaviors of pre-training for neural machine translation. In Findings of the Association for Computational Linguistics: ACL- IJCNLP 2021, pages 4265-42755 Online. Association for Computational Linguistics.
Patrick Kahardipraja, Brielen Madureira, and David Schlangen. 2021. Towards incremental transformers: An empirical analysis of transformer models for in- cremental NLU. In P Proceedings of the 2021 Confer- ence on Empirical Methods in Natural Language Pro- cessing. pages 1178-17 1789,119, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics.
Yinhan Liu, Jiatao Gu, Naman Goyal, Xian Li, Sergey Edunov, Marjan Ghazvininejad, Mike Lewis, and Luke Zettlemoyer. 2020. Multilingual denoising pre- training for neural machine translation. Transac- tions of the Association for Computational Linguis- tics, 8:726-742.
Yunsu Kim, Yingho Gao, and Hermann Ney. 2019. Ef- fective cross-lingual transfer of neural machine trans- lation models without shared vocabularies. In Pro- ceedings of the 57th Annual Meeting of the Asso- ciation for Computational Linguistics, pages 1246 1257, Florence, Italy. Association for Computational Linguistics.
Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Man- dar Joshi, Danqi Chen, Omer Levy, Mike Lewis. Luke Zettlemoyer, and Veselin Stoyanov. 2019. Roberta: A robustly optimized bert pretraining ap- proach. arXiv preprint arXiv: 16692
Taku Kudo and John Richardson. 2018 SentencePiece:
Mingbo Ma, Liang Huang, Hao Xiong, Renjic Zheng, Kaibo Liu, Baigong Zheng, Chuanqiang Zhang, Zhongjun He, Hairong Liu, Xing Li, Hua Wu, and Haifeng Wang. 2019a. STACL: Simultaneous trans- lation with implicit anticipation and controllable la- tency using prefix-toprred framework. In Proceed- ings of the 57th Annual Meeting of the Association for
A simple and language independent subword tok- enizer and detokenizer for neural text processing. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pages 66-71, Brussels, Belgium. Association for Computational Linguistics.

---

Computational Linguisties, pages 3025-3036, Flo- rence, Italy, Association for Computational Linguis- tics.
Wenxuan Wang. Wenxiang Jiao, Yongchang Hao, Xing Wang, Shuming Shi, Zhaopeng Tu, and Michael Lyu. 2022. Understanding and improving sequence-to- sequence pretraining for neural machine translation. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistces (Volume 1: Long Paperss, pages 2591-2600 Dublin, Ireland. Association for Computational Linguistics.
Xutai Ma. Juan Pino, James Cross, Liezl Puzon, and Jiatao Gu. 2019b. Monotonic multihead attention. arXiv preprint arXiv:1909 1 12406.
Matt Post. 2018. A call for clarity in reporting BLEU scores. In Proceedings of the Third Conference on Machine Translation: Research Papers, pages 186- 191, Brussels, Belgium. Association for Computa- tional Linguistics.
Rongxiang Weng. Heng Yu, Shujian Huang, Shanbo Cheng, and Weihua Luo. 2020. Acquiring knowledge from pre- pre-trained model to neural machine translation. In Proceedings of the AAAI conference on artificial intelligence, volume 34, pages 9266-9273.
Alec Radford, Karthik Narasimhan, Tim Salimans, llya Sutskever, et al. 2018. Improving language under- standing by generative pre-training.
Rongxiang Weng, Heng Yu, Weihua Luo, and Min Zhang. 2022. Deep fusing pre-trained models into neural machine translation. In Proceedings of the AAAI Conference on Artificial Intelligence, vol- ume 36, pages I 1468-11476.
Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. 2019. Language models are unsupervised multitask learners. OpenAl blog. 188)9
Haoran Xu, Benjamin Van Durme, and Kenton Mur- ray. 2021. BERT, mBERT, or BiBERT? a study on contextualized embeddings for neural machine trans- lation. In Proceedings of the 2021 Conference Empirical Methods in Natural Language Processing. pages 6663-6675, Online and Punta Cana, Domini- can Republic. Association for Computational Lin- guistics.
Colin Raffel, Minh-Thang Luong, Peter J. Liu, Ron J. Weiss, and Douglas Eck, 2017. Online and linear- time attention by enforcing monotonic alignments. In Proceedings of the 34th International Conference on Machine Learning, volume 70 of Proceedings of Machine Learning Research, pages 2837-2846. PMLR.
Shoetsu Sato, Jin Sakuma, Naoki Yoshinaga, Masashi Toyoda, and Masaru Kitsuregawa. 2020. Vocabu- lary adaptation for domain adaptation in neural ma- chine translation. In Findings of the Association for Computational Linguistics: EMNLP 2020, pages 4269-4279, Online. Association for Computational Linguisties.
Jiacheng Yang, Mingxuan Wang, Hao Zhou, Chengqi Zhao, Weinan Zhang, Yong Yu, and Lei Li. 2020. Towards making the most of bert in neural machine translation. In Proceedings of the AAAI conference on artificial intelligence, volume 34, pages 9378- 9385.
Felix Schneider and Alexander Waibel. 2020. Towards stream translation: Adaptive computation time for simultaneous machine translation. In Proceedings of the 17th International Conference on Spoken Lan- guage Translation, pages 228-236, Online. Associa- tion for Computational Linguistics.
Shaolei Zhang and Yang Feng. 2021. Universal simul- tancous machine translation with mixture-of-experr ef-experts wait-k policy. In Proceedings of the 2021 Confer- ence on Empirical Methods in Natural Language Pro- cessing. pages 7306-7317, Online and Punta Cana. Dominican Republic. Association for Computational Linguistics.
Rico Sennrich, Barry Haddow, and Alexandra Birch. 2016. Neural machine translation of rare words with subword units. In Proceedings of the 54th Annual Meeting of the Association for Computational Lin- guistics (Volume I: Long Papers), pages 1715-1725, Berlin, Germany. Association for Computational Lin- guistics.
Shaolei Zhang and Yang Feng. 2022a. Gaussian multi- head attention for simultaneous machine translation. In Findings of the Association for Computational Linguistics: ACL 2022, pages 3019-3030, Dublin, Ireland, Association for Computational Linguistics.
Ashish Vaswami, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. Attention all you need, Advances in neural information processing systems, 30.
Shaolei Zhang and Yang Feng. 2022b. Information- transport-based policy for simultaneous translation. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 992- 1013, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics.
Shushu Wang, Jing WU, Kai Fan, Wei Luo, Jun Xiao, and Zhongqiang Huang. 2023. Better simultaneous translation with monotonic knowledge distillation. In Proceedings of the 6l Anmual Meeting of the Association for Computational Linguistics ( (Volume Long Papers), pages 2334-2349, Toronto, Canada. Association for Computational Linguistics.
Shaolei Zhang and Yang Feng. 2022c. Modeling dual read/write paths for simultaneous machine transla- tion. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics ( (Vol- une 1: Long Papers). pages 1461-2477, Dublin, Ireland, Association for Computational Linguistics.

---

Shaolei Zhang and Yang Feng. 2023. Hidden markov transformer for simultaneous machine translation. arXiv preprint arXiv:2333.000257
tings. The details of hyperparameter settings for each model is in Table
To ensure optimal performance of the ITST mod- els, we conducted multiple training runs for each model configuration and dataset. We observed per- formance fluctuations across the training runs. To select the best-performing models for both token- level and word-level policies, we repeated the train- ing process three times and selected the checkpoint with the lowest validation loss for each configura- tion.
Shaolei Zhang. Yang Feng, and Liangyou Li. 2021. Future- gatrreeguieddddde incremental transformer for simulta- neous translation. Proceedings of the AAAI Con- ference on Artificial Intelligence, volume 35, pages 11428-14436.
Shaolei Zhang, Shoutao Guo, and Yang Feng. 2022. Wait-info policy: Balancing source and target in- formation level for simultaneous machine translation. In Findings of the Association for Computational Linguistics: EMNLP 2022, pages 22449,2663, Abu Dhabi, United Arah Emirates. Association for Com- putational Linguistics.
For LM integrated models, the intra-word bidi- rectional encoding was not applied to the LM to prevent the need for fine-tuning.
Baigong Zheng, Kaibo Liu, Renjic Zheng, Mingbo Ma, Hairong Liu, and Liang Huang. 2020. Simultane- translation policies: From fixed adaptive. In Proceedings of the 58th Annual Meeting of the Asso- ciation for Computational Linguistics, pages 2847- 2853, Online. Association for Computational Lin- guistics.
A.3 Examples and discussions
In Table 2, an illustrative case is presented to demonstrate the distinction between token-level policy (token-level Wait-1) and word-level pol- icy (word-level Wait-1). At Step 2, the token-level model predicts the target token "B" after process- ing the subword "B" for the word "Beine" (which means "leg" in German). Subsequently, it fails to recover from this incorrect prediction and contin- ues by predicting "ody as the next token, even after processing the remaining token "eine_ ". In contrast, the word-level model accurately predicts "legs" (encoded "leg s _")) after processing the complete word 'Beine' (encoded as 'B eine_') '). Furthermore, the token model also makes erro- ncous predictions for all steps when processing the word 'blutiberströmtt. (encoded as 'bl ut über ström t : meaning "covered in blood" in Ger- man), while the word-level model accurately pre- dicts correct word in single step.
Jinhua Zhu, Yingce Xia, Lijun Wu, Di He, Tao Qin. Wengang Zhou, Houqiang Li, and Tie-Ya Yan Liu. 2020. Incorporating bert into neural machine translation. arXiv preprint arXiv:2002.06823.
A Appendix
A.1
Token-level latency evaluation in previous works
Many previous works in the field of SiMT have employed token-level latency calculation as a fun- damental metric for evaluating system performance. In the first group of papers, which includes (Ari- vazhagan et al., 2019; Ma et al., 2019b; Schneider and Waibel, 2020; Zheng et al., 2020; Zhang and Feng. 2022b), researchers explicitly stated their use of token-level latency calculation or incorporated token-level scores in their analyses. On the other hand, the second group, comprising (Zhang and Feng, 20226; Elbayad et al., 2020; Zhang and Feng, 2022c, 2021, 2022a; Zhang et al., 2022; Guo et al., 2022; Zhang and Feng, 2023; Guo et al., 2023; Caglayan et al., 2020; Wang et al., 2023), can be identified by their publicly available official imple- mentations, which incorporate token-level latency evaluation code. This underscores the prevalence of token-level latency calculation in assessing the effectiveness SiMT systems in the cutting-edge SiMT research works.
Another case be observed in Table 3, where the token-level model initiates the translation be- fore fully reading the word "Irgendetwas" (encoded as "Irgen det was "). As a consequence, it pro- duces an incorrect translation. On the other hand, the werd-level model accurately translates the sen- tence by processing the complete word before mak- ing any predictions.
A.4
Table of main results
A.5 Graphs of main results
A.2 More training details
For Transformer model configurations, we follow the Efficient Wait-k (Elbayad et al., 2020)'s set-

---

Hyperparameter
transformer--malll
transformer-base
transformer-big
attention heads embedding dime ffn embeding dim dropout encoder layers decoder layers Ir Ir scheduler optimizer clip-norm
16
256
512
1024
1024
2048
4096
0.3
0.3
0.3
6
6
6
6
5e-4
5e-4
5e-4
inverse sqrt Adam (0.9, 0.98)
inverse sqrt Adam (0.9, 0.98)
inverse sqrt
Adam (0.9, 0.98)
warmup-updates
4000
4000
4000
warmup-init-Ir
1e-7
1e-7
le-7
weight decay
0
label smoothing
0.1
0.
0.1
max tokens
32768
524288
1048576
Table 1: Hyperparameters of each Transformer model.
Sre:
Meine
B eine waren ut
üibe (,
Ref:
My
leg 8 were covered
blood.
Step
Streaminig Input
Token Wait-1 Output
Word Wait-l Output
Meine
My
My
2
Meine
B
B
3
Meine
B eine
ody
leg
4
Meine
B cine
waren
was
were
5
Meine
B eine
waren bl
blue
6
Meine
B cinc
waren
bl ut
and
7
Meine
B eine
waren
bl ut über
my
8
Meine
B cinc
waren
bl ut üher ström
leg
9
Meine
B cinc
bl ut ütee ström
10
Meine B cine
waren
bl ut üher ström
blood ri dden
bloo dy
Table 2: An example case from WMT De En test set (transformer-bo for Token-level Wait-1 (token-AL.. 2.56. word-AL: 1.69) and Word-level Wait-l (token-AL: 1.44, word-Al.: 1.0).
Sre:
Ir gen det was
lag in
Luft
Ref:
Some thing was
Step
Streaminig Input
Token Wait-1 Output
Word Wait-l Output
Ir
Ir
Ir gen
gen
Ir gen det
gen
4
Ir gen det was
gen
Some thing
5
Ir gen det was
lag
gen
lay
Ir gen det was
lag
in
Ir gen det
lag
in
der
the
Ir gen det
lag
der
Luft Luft
Ir gen det
lag
fact
WHS
the
Table 3: An Eexample case from WMT De.En test set (transformer-biig for Token-level Wait-1 (token-AL: 2.69, word-AL: 4.25) and Word-level Wait-l (token-AL: 2.38, word-AL: 1.0).

---

15
Bi
Waze leken-evell
MiE uken-lvd
IIST teven-leee
Wat-1 wonblevel
MrG work
ITST wonblerd
Wat-l wonbleedd- LM
MrE wod & LM
ITST wonddord -M
12
(a) De-En Wait-k Transformer base
(b) De-En MoE Transformer base
(c) De-En ITST Transformer base
Figure 10: Results of translation quality v.s. word-level AL, De-En transformer-base.
42
12-
34
12
12
TokanAL
Tokeo
Token-A
(a) En-Fr Wait-k Transformer small
(b) En-Fr MoE Transformer small
(c) En-Fr ITST Transformer small
BELEUU
telen-sroell wond-level
Inkendlerl
FIST
weed
TTST wond-lovel
Wast I word level
weed level LM
TIST word level -LM
TokmsAA
Token-AL
TokmsAL
(d) De-En Wait-k Transformer base
(c) Dc-En MoE Transformer base
(f) De-En ITST Transforner base
B0.20
BERU
29
Wistok lokendevel
MrC tnketoloee
TSTT token-level TTST word lroel
Wort- 2 wond-level level
Mrl weed lnvel
Wor-- wonf-leell- IM
Mrl werd lovel LM
35
TIST wond-lood -IM
Token AL
(g) De-Fn Wait-k Transformer big
(h) De-En MoF Transformer big
De-En ITST Transformer big
Figure 1l: Results of translation quality v.s. token-level AL.

---

Token-level Wait-k
Token-level MoE Wait-k
k
token AL
word AL
BLEU
token AL
word AL
BLEU
1.95
1.91
32.32
1.90
1.88
32.20
3 3.83
3.43
38.90
3.83
3,42
38.92
5
5.67
4.82
40.68
5
5.69
4.84
40.51
7
7.51
6.23
40.95
7.53
6.25
40.83
9
9.28
7.58
41.06
9.31
7.60
40.92
11 10.91
8.82
41.14
11
10.94
8.85
40.95
13
12.39
9.95
41.12
13
12.42
9.97
40.96
Word-level Wait-k
Word-level MoE Wait-k
k
token AL
word AL
BLEU
k
token AL
word AL
BLEU
1 2.24
1.74
32.43
2.01
1.60
32.01
3 4.18
3.32
39.36
4.18
3.32
39.28
5
6.55
5.15
41.09
6.57
5.17
40.77
7 8.85
6.94
41.48
8.86
6.95
41.03
9 10.92
8.57
41.57
10.94
8.58
41.11
11
12.75
9.98
41.58
12.76
9.99
41.10
Word-level Wait-k w/ LM
Word-level MoE Wait-k w/ LM
k
token AL
word AL
BLEU
k
token AL
word AL
BLEU
1 2.18
1.74
33.47
2.17
1.72
33.01
3 4.23
3.37
40.83
4.22
3.37
40.46
5 6.58
5.19
42.78
6.56
5.17
42.35
7 8.86
6.97
43.00
8.84
6.95
42.61
9
10.94
8.58
43.08
10.91
8.57
42.66
11
12.76
10.00
43.09
11
12.74
9.99
42.67
Table 4: Main results of transformer-smon Waik-k
Table 5: Main results of transformer-small MoE
models on IWLST17 En.Fr Fr dataset.
Waik-k models IWSLT17 En.Fr dataset.

---

Token-level ITST
6
token AL
word AL
BLEU
0.2
3.33
3.05
37.81
0.3
3.59
3.25
38.64
0,4
4.14
3,70
39.40
Token-level Wait-k
0.5
5.18
4.59
40.19
token AL
word AL
BLEU
0.6
6.71
5.92
40.84
0.52
0.75
17.63
0.7
8.62
7.63
41.27
2.16
1.91
23.24
0.75
9.56
8.45
41.34
3.92
3,05
26.14
0.8
10.72
9.41
41.42
7
5.77
4.23
27.46
0.85
12.11
10.53
41.42
9
7.76
5.50
28.77
0.9
13.84
11.80
41.42
11
9.64
6.69
29.37
Word-level ITST
13
11.49
7.86
29.88
8
token AL
word AL
BLEU
15
13.18
8.92
29.89
0.2
3.27
2.64
37.25
17
14.79
9.93
30.19
0.3
3.46
2.80
37.88
Word-level Wait-k
0.4
3.91
3.21
38.69
k 2.96
token AL
word AL
BLEU
0.5
4.67
3,91
39.57
1.84
24.02
0.6
5.81
4.96
40.44
5.33
3.43
28.08
0.7
7.29
6.28
40.94
8.08
5.23
29.35
0.75
8.21
7.05
41.13
10.82
7.01
29.87
0.8
9,41
8.02
41.36
13.47
8.72
30.17
0.85
11.19
9.37
41.50
11
15.88
10.25
30.38
0.9
13.36
11.00
41.53
Word-level Wait-k w/ LM
Word-level ITST w/ LM
k
token AL
word AL
BLEU
8
token AL
word AL
BLEU
2.96
1.84
24.71
0.2
3.25
2.61
38.30
5.27
3.38
28.49
0.3
3.30
2.65
38.75
8.05
5.20
29.95
0.4
3.48
2.81
39.61
10.81
7.00
30.56
0.5
3.94
3.23
40.50
9
13.44
8.69
30.74
0.6
4.80
4.04
41.35
11
15.85
10.24
30.95
0.7
6.02
5.19
41.88
0.75
6.76
5.86
42.09
Table 7: Main results of transformer-base Waik-k
0.8
7.72
6.67
42.30
models on WMT15 De. En dataset.
0.85
8.99
7.68
42.56
0.9
10.90
9.16
42.63
Table 6: Main results of transformer-small] ITST
models on IWSLT17 En.Fr dataset.

---

Token-level ITST
8
token AL
word AL
BLEU
0.2
2.45
2.08
22.19
0.3
2.89
2.38
24.15
0.4
3.82
2.96
26.21
Token-level MoE Wait-k
0.5
5.23
3.92
28.19
k
token AL
word
BLEU
0.6
7.28
5.50
29.25
1.17
1.11
17.28
0.7
9.87
7.48
29.47
3
2.50
2.09
23.29
0.75
11.66
8.81
29.76
5
4.17
3.18
25.95
0.8
13.79
10.38
29.79
7
6.00
4.36
27.86
0.85
16.60
12.11
29.90
9
7.81
5.51
28.81
0.9
19.81
13.85
29.71
11
9.62
6.66
29.53
Word-level ITST
13
11.42
7.80
29.66
token AL
word AL
BLEU
15
13.15
8.89
29.81
0.2
4.13
2.64
26.81
14.80
9.91
29.79
0.3
4.19
2.69
27.01
Word-level MoE Wait-k
0.4
4.39
2.81
27.47
k
token AL
word AL
BLEU
0.5
5.09
3.30
28.04
1 3.12
1.91
23.59
0.6
6.43
4.28
28.87
3 5.29
3.39
27.75
0.7
8.60
5.93
29.54
5 8.03
5.19
29.29
0.75
9.91
6.92
29.54
7 10.79
6.98
30.14
0.8
11.54
8.09
29.67
9 13.38
8.65
30.39
0.85
13.83
9.61
29.82
11 15.81
10.2
30.6
0.9
16.75
11.38
29.93
Word-level MoE Wait-k w/
LM
Word-level ITST w/ LM
k
token AL
word AL
BLEU
8
token AL
word AL
BLEU
1 3.08
1.92
24.97
0.2
4.15
2.64
27.74
3 5.32
3.44
28.66
0.3
4.24
2.70
27.99
5 8.08
5.23
30.21
0.4
4.55
2.92
28.44
7 10.87
7.04
30.63
0.5
5.33
3.49
29.15
9 13.47
8.73
30.9
0.6
6.77
4.63
29.97
11
15.88
10.26
31.0
0.7
8.57
6.08
30.49
Table 8:
Main results of transformer-base MoE
0.75
9.51
6.80
30.53
Waik-k models WMT15 De.En dataset.
0.8
10.71
7.67
30.60
0.85
12.32
8.72
30.68
0.9
14.28
9.93
30.71
Table 9: Main results transformer-bass ITST mod- els on WMT15 De. En dataset.

---

Token-level Wait-k
Token-level MoE Wait-k
k
token AL
word
BLEU
token AL
word AL
BLEU
-0.13
0.39
19.20
0.86
0.94
18.44
3 1.80
1.71
24.01
2.19
1.92
24.07
5
3.74
2.95
26.86
3.90
3.04
26.70
7
5.81
4.26
28.86
7
5.70
4.19
28.47
9
7.80
5.53
29.96
9
7.69
5.47
29.59
11
9.72
6.74
30.62
11
9.60
6.67
30.36
13
11.51
7.87
30.91
13
11.44
7.83
30.51
15
13.20
8.95
31.16
15
13.14
8.91
30.56
17
14.84
9.97
31.31
17 14.75
9.91
30.65
Word-level Wait-k
Word-level MoE Wait-k
k
token AL
word AL
BLEU
k
token AL
word AL
BLEU
1 2.55
1.59
24.63
2.86
1.76
24.79
3 5.11
3.30
28.82
5.24
3.37
28.95
5 7.95
5.16
30.43
7.98
5.15
30.26
7 10.77
6.99
31.35
10.77
6.97
31.01
9 13.37
8.678
31.86
13.37
8.66
31,47
11 15.80
10.22
32.03
15.80
10.20
31.55
Word-level Wait-k w/ LM
Word-level MoE Wait-k w/ LM
k
token AL
word AL
BLEU
k
token AL
word AL
BLEU
1 2.89
1.79
26.21
2.60
1.60
25.26
3 5.21
3.36
30.16
5.19
3.31
29.03
5
7.98
5.17
31.36
8.01
5.16
30.79
7 10.82
7.01
32.29
10.83
6.99
31.66
9
13.42
8.71
32.58
9
13.43
8.67
31.82
11
15.74
10.16
32.45
11
15.83
10.21
32.17
Table 10: Main results of transformer-big Waik-k models on WMT15 En dataset.
Table 11: Main results of transformer-big MoE Waik- models on WMT15 En dataset.

---

Token-level ITST
8
token AL
word AL
BLEU
0.2
1.49
1.55
23.22
0.3
2.41
2.12
24.75
0,4
3.93
3,06
26.67
0.5
5.99
4.39
28.23
0.6
9.12
6.57
29.85
0.7
12.69
9.32
31.02
0.75
14.72
10.75
31.28
0.8
16.95
12.23
31.54
0.85
19.06
13.49
31.70
0.9
21.26
14.69
31.84
Word-level ITST
8
token AL
word AL
BLEU
0.2
3.69
2.35
26.33
0.3
3.79
2.42
26.62
0.4
3.94
2.54
26.62
0.5
4.69
3.02
27.55
0.6
6.60
4.27
28.99
0.7
10.28
6.77
30.41
0.75
12.50
8.34
30.95
0.8
14.22
9.65
31.54
0.85
15.90
10.81
31.56
0.9
18.13
12.23
31.61
Word-level ITST w/ LM
8
token AL
word AL
BLEU
0.2
3.97
2.53
28.12
0.3
4.00
2.55
28.13
0.4
4.10
2.62
28.29
0.5
4.39
2.82
28.92
0.6
5.37
3.46
29.66
0.7
7.85
5.13
31.18
0.75
9.73
6.48
31.77
0.8
11.90
8.04
32.10
0.85
14.04
9.57
32.34
0.9
16.14
10.97
32.62
Table 12: Main results of transformer-bip ITST mod- els on WMT15 En dataset.


Reviews:
[{"review_id": "gy961FhTpg", "reviewer": "Reviewer_tsh7", "summary": "", "questions": "1. What are some potential solutions or alternative approaches that could be explored to address the limitation of the proposed method in SiMT systems for languages with a writing style that lacks spaces or other delimiters between words or sentences, such as Chinese?", "limitations": "", "rating": 4, "confidence": 4, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your positive feedback and valuable comments.\n\n### Speed experiments\nFollowing the reviewer's suggestion, we have added speed comparison between models with and without an LM to assess the potential impact of the LM integration to the latency of an SiMT system. Below is the results (Measured on a 3090 GPU):\n\n| Model | Speed (Words/sec)  |  Relative speed w.r.t. baseline|\n|:---:|:---:|:---:|\n| Transformer-small  | 72.0   | 1.0  |  \n| Transformer-small + GPT2  | 47.6   | 0.66  | \n| Transformer-big  | 63.2   | 1.0  |  \n| Transformer-big + XGLM  | 43.8  | 0.69  |  \n\nResults show that models with an LM is up to 34% slower than baseline models. However, they are still faster than the speed of natural conversation (160 words per min) [1].\n\n### Potential solutions or alternative approaches for languages without space\nFor languages that have well-developed segmentation methods like 'jieba' for Chinese and 'nagisa' for Japanese, we can initially tokenize input sentences into words using the segmenter and then apply BPE to further split them into subwords, similar to [2]. However, a limitation that remains in this approach is that, it requires different systems to use the same segmenter for fair comparisons between them.\n\n[1] Towards an integrated understanding of speaking rate in conversation (Yuan et al., ICSLP 2006) \\\n[2] The University of Edinburgh’s Neural MT Systems for WMT17 (Sennrich et al., WMT 2017)", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "Strengths of the paper:\n- Proposes a novel method of integrating language models (LM) into Simultaneous Machine Translation (SiMT) systems\n- Offers a more versatile solution that can be integrated into most existing neural SiMT models\n- LM-fused attention proves effective in enhancing translation quality across all latency levels\n- Proposed word-level policy plays a crucial role in effectively managing the vocabulary mismatch between the LM and model", "reasons_to_reject": "Weaknesses of the paper:\n- The proposed method may not be applicable to languages with a writing style that lacks spaces or other delimiters between words or sentences, such as Chinese. Experiments like translation direction of En-Zh should be included in this paper.\n- Integrating a large LM may require a faster compute capability to fulfill the low-latency demands of the SiMT task.Speed experiments should be involved in this paper.", "questions_to_authors": "1. What are some potential solutions or alternative approaches that could be explored to address the limitation of the proposed method in SiMT systems for languages with a writing style that lacks spaces or other delimiters between words or sentences, such as Chinese?", "paper_topic_and_main_contributions": "The paper addresses the problem of the assumption in many existing studies that operations are carried out at the subword level, even though the standard unit for input and output in most practical scenarios is typically at the word level. The paper demonstrates that policies devised and validated at the subword level are surpassed by those operating at the word level, which process multiple subwords to form a complete word in a single step. The main contributions of this paper are the demonstration of the superiority of word-level policies over subword-level policies in SiMT and the proposal of a method to boost SiMT models using language models.", "excitement": 4, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "Vx1614fGcC", "reviewer": "Reviewer_CvhJ", "summary": "", "questions": "Question A: How are the chunk boundaries defined? what are the parameters governing the model that takes the splitting decision? Obviously, the chunk size affects the quality of the bidirectional encoding, have you explored this point in your experiments? I guess bidirectional encoding is applied to all chunks, except for the last one in which unidirectional encoding would be applied, am I right?", "limitations": "", "rating": 4, "confidence": 3, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your thorough review and constructive criticism. We have revised our paper to address your suggestions and comments. More details are elaborated below.\n\n### Improving the presentation and discussion of results\nIn the main result (Figure 5), We relocated the Transformer-base plot and token-level AL plots to the Appendix, and now have dedicated plots for each policy:  Wait-k, MoE, and ITST. With these changes, the total number of plots remains at 6 (3 policies * 2 datasets), but each plot features only 3 curves of baseline, word-level policies with and without an LM. Now it is a lot easier to identify the relative impact of word-level policies and adding an LM to them. (In short, both almost always improve results, but in general adding an LM bring more improvements)\n\n### Explanation for the word-level chunk attention\nWe acknowledge the concerns raised by two reviewers regarding the misleading naming. We have now renamed it **'intra-word bidirectional encoding.'** At the word level, this approach involves unidirectional encoding for each word in the input sentence, meaning past words cannot attend to future words. However, at the subword level, subwords within the same word can attend to each other, allowing past subwords to attend to future subwords within the same word. Since READ operates at the word level in word-level policies, this encoding does not require recomputation during each WRITE operation. It only necessitates a single forward pass, similar to token-level unidirectional encoding. However, it can produce a better encoded representation by enabling attention to more tokens.\n\n### Missing references on discussing the encoding of the input sequence\nWe have added these to the discussion in the paper like the following: \"To minimize redundancy in encoding input sequences following READ operations, one can consider adaptations to the encoder for enhanced efficiency. Techniques such as the recurrent Linear Transformer (Kahardipraja et al., 2021), Partial Bidirectional Encoding (Sanchez et al., 2022), and the widely adopted unidirectional token encoding (Elbayad et al., 2020) are among the available options.\"\n\n### Tables 4-12 in the appendix\nThese tables present precise values corresponding to the main results depicted in Figure 5. Given that SiMT evaluation predominantly entails graphical comparisons of latency-accuracy curves, including these tables allows a more accurate comparison and recreating the same curves. Demonstrating relative improvements over the baseline in these tables poses challenges due to the varying latency (AL) scores associated with each data point. JFYI, we adopted the same table format other papers have used.\n\n### WW, TW, WT, but Read-k-Write-k in Figure 6 (a)\nWe chose not to use 'TT' for Read-k-Write-k to prevent confusion. According to our existing nomenclature of WW, TW, and WT, 'TT' should represent the token-level Wait-k, which is distinct from Read-k-Write-k. We acknowledge the need for a more consistent and reader-friendly naming scheme and have decided to change it to 'TkTk'.\n\n### Cleaning Figure 8 (a)\nFollowing your advice, we reduced the number of curves from 7 to 5 in Fig. 8 (a) by removing the GPT2-medium results and putting them in the appendix.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "The improvements in translation quality are consistent and significant across k values in wait-k policies.", "reasons_to_reject": "The presentation and discussion of results need to be improved to put into perspective what contributions are having a more significant impact than others and under which conditions. For example, Figure 5 shows too many plots and too many curves, I would discard Transformer base plots and I would select those curves that clearly show the main conclusions that can be extracted from the experiments: word-level policies improve the results and adding a LM does much more. Those conditions that do not help much can be mentioned in the discussion. However, it would be nice to know what the contribution is for the word-level policies and the LMs and under which conditions one may contribute more than the other, if this is the case.\n\nWord-level chunk attention is not clear to me since little detail is provided about how chunks are defined.\n\nTo sum up, the results are good and extensive, but are not easy to read in order to draw the main conclusions. The authors should have devoted more time to prepare the presentation of the results to ease the task of the reader.", "questions_to_authors": "Question A: How are the chunk boundaries defined? what are the parameters governing the model that takes the splitting decision? Obviously, the chunk size affects the quality of the bidirectional encoding, have you explored this point in your experiments? I guess bidirectional encoding is applied to all chunks, except for the last one in which unidirectional encoding would be applied, am I right?", "paper_topic_and_main_contributions": "This paper shows that moving from token-based to word-level policies in Simultaneous Machine Translation (SiMT) brings about improvements in translation quality leveraging all subwords that are part of the word being read and not writing in the middle of a word. In addition, and more importantly, it eases the integration of external LMs in order to improve SiMT models, and this is shown applying LM-fused attention previously applied in NMT. Finally, word-level policies allows for a fair comparative evaluation independently of the subword algorithm being applied. The results show a consistent improvements up to approximately 15%.", "excitement": 4, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {"Missing_References": "When discussing the encoding of the input sequence in lines 227-232, I think it is worth mentioning two additional references:\n\n@inproceedings{iranzo-sanchez-etal-2022-simultaneous,\n    title = \"From Simultaneous to Streaming Machine Translation by Leveraging Streaming History\",\n    author = \"Iranzo Sanchez, Javier  and\n      Civera, Jorge  and\n      Juan-C{\\'\\i}scar, Alfons\",\n    booktitle = \"Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)\",\n    month = may,\n    year = \"2022\",\n    address = \"Dublin, Ireland\",\n    publisher = \"Association for Computational Linguistics\",\n    url = \"https://aclanthology.org/2022.acl-long.480\",\n    doi = \"10.18653/v1/2022.acl-long.480\",\n    pages = \"6972--6985\",\n    abstract = \"Simultaneous machine translation has recently gained traction thanks to significant quality improvements and the advent of streaming applications. Simultaneous translation systems need to find a trade-off between translation quality and response time, and with this purpose multiple latency measures have been proposed. However, latency evaluations for simultaneous translation are estimated at the sentence level, not taking into account the sequential nature of a streaming scenario. Indeed, these sentence-level latency measures are not well suited for continuous stream translation, resulting in figures that are not coherent with the simultaneous translation policy of the system being assessed. This work proposes a stream-level adaptation of the current latency measures based on a re-segmentation approach applied to the output translation, that is successfully evaluated on streaming conditions for a reference IWSLT task\",\n}\n\n@inproceedings{kahardipraja-etal-2021-towards,\n    title = \"Towards Incremental Transformers: An Empirical Analysis of Transformer Models for Incremental {NLU}\",\n    author = \"Kahardipraja, Patrick  and\n      Madureira, Brielen  and\n      Schlangen, David\",\n    booktitle = \"Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing\",\n    month = nov,\n    year = \"2021\",\n    address = \"Online and Punta Cana, Dominican Republic\",\n    publisher = \"Association for Computational Linguistics\",\n    url = \"https://aclanthology.org/2021.emnlp-main.90\",\n    doi = \"10.18653/v1/2021.emnlp-main.90\",\n    pages = \"1178--1189\",\n    abstract = \"Incremental processing allows interactive systems to respond based on partial inputs, which is a desirable property e.g. in dialogue agents. The currently popular Transformer architecture inherently processes sequences as a whole, abstracting away the notion of time. Recent work attempts to apply Transformers incrementally via restart-incrementality by repeatedly feeding, to an unchanged model, increasingly longer input prefixes to produce partial outputs. However, this approach is computationally costly and does not scale efficiently for long sequences. In parallel, we witness efforts to make Transformers more efficient, e.g. the Linear Transformer (LT) with a recurrence mechanism. In this work, we examine the feasibility of LT for incremental NLU in English. Our results show that the recurrent LT model has better incremental performance and faster inference speed compared to the standard Transformer and LT with restart-incrementality, at the cost of part of the non-incremental (full sequence) quality. We show that the performance drop can be mitigated by training the model to wait for right context before committing to an output and that training with input prefixes is beneficial for delivering correct partial outputs.\",\n}", "Typos_Grammar_Style_and_Presentation_Improvements": "Typos:\n\nL. 142: attetnion -> attention\nL. 216: , The goal -> , the goal\nL. 232: unidirectionally**.** (Elbayad et al., 2020)\nL. 147: missing words in \"Zhang and Feng (2022a) model to predict the alignment...\"\nL.422: BLEU calculation**.** (Post, 2018)\nTable 2: An Eexample case -> An example case\n\n\nPresentation improvements:\n\nTables 4-12 in the appendix could be more compacted and elaborated to ease the comparison across k values (since explored k values are different), relative improvements over the baseline would help to clearly see the impact of the different factors.\n\nFigure 6 (a) and its description in the text need to be improved to make the description more consistent, you are using WW, TW, WT, but Read-k-Write-k to refer to TT policy.\n\nI would discard Figure 6 (b), since it can be described with a sentence. In addition, this contribution is not clear to me since little detail is provided about how chunks are defined.\n\nFigure 8 (a) contains too many curves that cannot be read properly. I would reduce the number of curves discarding those less significant in terms of impact in the results and describe these minor effects in the text. In addition, from an aesthetic viewpoint, I would try to use same font size across figures in page8 and shorthen legend description to not occlude the curves (see Figure 8 (a))."}}, {"review_id": "l1wmwaRRVf", "reviewer": "Reviewer_5oLi", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your review, comments and opinions.\n\n### In-depth explanation and evidence supporting our claim regarding the evaluation and analysis of latency\nFirst of all, to clarify, our claim pertains specifically to SiMT for text-to-text translation, where both source and target sentences are typically BPE tokenized. We have updated our paper to make this distinction clear. \\\nWhile competitions ensure a fair comparison by defining a standardized input/output format that all submissions must adhere to, many individual research works report latency scores without such a standardized format. We did not claim that word-level latency calculations were never used. However, as shown in the below references, token-level AL has been more common in the recent papers and we proposed to use word-level AL for a fair comparison. We think this situation is analogous to the inconsistent BLEU calculation before the introduction of SacreBLEU, which was affected by variations in tokenization and normalization. Below we summarize a list of references that we have reviewed to confirm that the latency evaluations of the systems were conducted on encoded tokens:\n1. The MILk paper [1] is known to use BPE tokens for latency calculation. Examples and analysis in the paper are also based on tokens.\n2. In the MMA paper [2], a footnote in page 6 mentions they follow [1]: “Latency metrics are computed on BPE tokens for WMT15 De-En – consistent with Arivazhagan et al.”\n3. [3] also states that they report latency scores on tokenized inputs following [1].\n4. In [4], the example sentences and their latency scores are at the token level (page 5).\n5. In [5], all latency scores for text inputs are all measured at the token level.\n6.  [5, 6, 7, 8, 9, 10, 11, 12, 13, 14] are SiMT research works that we are aware of that share official implementations publicly. We analyzed and ran all the code, and confirmed that they all calculated the latency score from the subword sequences.\n7. We asked the main author of [15] how they calculated the AL scores and this is a quote from their reply: “Our latency(AL) scores were calculated on BPE tokens. We generally followed the training/ evaluation settings of  Multipath Wait-k and ITST, and BPE-level Average Lagging (AL) were used to measure latency.”\n8. SimulEval supports two types of “--eval-latency-unit”: “word” or “char”, and the default is “word”. However, the code relies on the user to input original source/target text and the given agent to calculate a word-level latency. If BPE tokens are given, it may return BPE-token level latency scores. More importantly, all the open-source implementations we mentioned above do not use SimulEval and instead use their own evaluation code.\n\nWe believe that the above evidence could advocate our claim and thus we respectfully request the reviewer to consider reevaluating our paper. We have added the above details to the appendix in the paper. We also revised the words \"most SiMT systems\" in our claim line 42 to \"many SiMT systems\".\n\n[1] Monotonic Infinite Lookback Attention for Simultaneous Machine Translation (Arivazhagan et al., ACL 2019) \\\n[2] Monotonic Multihead Attention (Ma et al., ICLR 2020) \\\n[3] Towards Stream Translation: Adaptive Computation Time for Simultaneous Machine Translation (Schneider & Waibel, IWSLT 2020) \\\n[4] Simultaneous Translation Policies: From Fixed to Adaptive (Zheng et al., ACL 2020) \\\n[5] Information-Transport-based Policy for Simultaneous Translation (Zhang & Feng, EMNLP 2022) \\\n[6] Efficient Wait-k Models for Simultaneous Machine Translation (Elbayad et al., INTERSPEECH 2020)  \\\n[7] Modeling Dual Read/Write Paths for Simultaneous Machine Translation (Zhang & Feng, ACL 2022) \\\n[8] Universal Simultaneous Machine Translation with Mixture-of-Experts Wait-k Policy (Zhang & Feng, EMNLP 2021) \\\n[9] Gaussian Multi-head Attention for Simultaneous Machine Translation (Zhang & Feng, Findings 2022) \\\n[10] Wait-info Policy: Balancing Source and Target at Information Level for Simultaneous Machine Translation (Zhang et al., Findings 2022) \\\n[11] Turning Fixed to Adaptive: Integrating Post-Evaluation into Simultaneous Machine Translation (Guo et al., Findings 2022) \\\n[12] Hidden Markov Transformer for Simultaneous Machine Translation (Zhang et al., ICLR 2023) \\\n[13] Learning Optimal Policy for Simultaneous Machine Translation via Binary Search (Guo et al., ACL 2023) \\\n[14] Simultaneous Machine Translation with Visual Context (Caglayan et al., EMNLP 2020) \\\n[15] Better Simultaneous Translation with Monotonic Knowledge Distillation (Wang et al., ACL 2023)\n\n### Existence of prior research using word-level policies and novelty of our work\nWe acknowledge the second reference provided by the reviewer, which introduces a component for determining chunk boundaries at the word level. This work is indeed relevant and will be included in our Related Work section. However, it's important to emphasize that our contribution lies not in being the first to introduce word-level policies but in our discovery that many existing policies, initially designed at the subword level, can benefit significantly from transitioning to the word level. Moreover, we demonstrate that incorporating a language model on top of these word-level policies further enhances their performance. This finding represents a significant contribution to the SiMT community, as it opens the door for improving existing systems with relatively minor modifications and highlights the pivotal role of word-level policies in effectively integrating language models into SiMT.\n\n### Naming issue with the “word-level chunk attention”\nWe agree with you and renamed it to **intra-word bidirectional encoding**. Please refer to our rebuttal for Reviewer 2 if you need more detailed explanation.\n\n### About the ethical concern of the reviewer that the authors are lying\nWe are not lying about anything. We hope the evidence we provided above could resolve both the reviewer’s technical and ethical concerns.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "One of the main claims of the paper that so far, the average lagging, one of the main ways of evaluating simultaneous MT systems, has happened on subword level. This claim is not true, and the idea to go from subword level to word level when computing the policy is not new at all either. The remaining part about using the LM is not enough for publication.\n\nSo overall, I see no reasons to accept the paper in its current form.", "reasons_to_reject": "The main claim of the paper that so far the average lagging was computed on subword level and not on word level is simply not true. In fact, SimEval by default uses word level.  This is absolutely necessary to do to have fair comparison between submissions to competitive evaluations/shared tasks like Workshop on Simultaneous Translation, and also Simultaneous speech translation track at the IWSLT conference. \n\nAlso, prior research definitely used word-level adaptive policies, like the phrases/chunks in one of the papers that I listed below - these were defined on the word level. \n\nYour use of word-level \"chunk attention\" is misleading - you seem to just attend to multiple subwords of the same word - this is not a chunk in any of the established meaning of this word. \n\nOverall, it is clear that the output of words, not subwords is expected in simultaneous MT, as they have to be presented to a user in form of words. How you read the input, in subwords, words, or directly from speech, or in characters - this is up to the architecture of the system, so there is nothing novel about going from subwords to words there. \n\nSo overall, I see no merit in the paper, as its main claims have no ground.", "questions_to_authors": "", "paper_topic_and_main_contributions": "The paper proposes to use policies like wait-k for simultaneous MT on word level rather than on subword level. The authors additionally suggest to use a language model (possibly with a different subword segmentation than the translation model) to further improve the quality of the translation; operating on word level makes it possible to apply the LM despite the differences in subword segmentation.", "excitement": 3, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Typos_Grammar_Style_and_Presentation_Improvements": "Line 493: \"all tested latency settings\": according to Fig. 6b, this is not true: in the lowest latency setting of 2, the system without \"chunk attention\" is better."}, "extra_text": {"Missing_References": "@inproceedings{papi2022does, title={Does Simultaneous Speech Translation need Simultaneous Models?}, author={Papi, Sara and Gaido, Marco and Negri, Matteo and Turchi, Marco}, booktitle={Findings of the Association for Computational Linguistics: EMNLP 2022}, pages={141--153}, year={2022} }\n\n@article{wilken2020neural, title={Neural Simultaneous Speech Translation Using Alignment-Based Chunking}, author={Wilken, Patrick and Alkhouli, Tamer and Matusov, Evgeny and Golik, Pavel}, journal={IWSLT 2020}, pages={237}, year={2020} }", "Justification_for_Ethical_Concerns": "From Line 42: \"The performance analysis and implementation of most SiMT systems, to the best of our knowledge, have been carried out on encoded sequences of source and target subwords, rather than on the original source and target sentences. This has led to two critical issues that need to be addressed.\"\n\nThis is simply not true! And because of this, the paper has no value! The authors did add \"to the best of our knowledge\", but actually, this is a known fact for everyone working on simultaneous MT. \n\n=== After the author's rebuttal, I acknowledge that some papers still use BPE-level evaluation of simultaneous MT systems. Still, I believe that it is clear that this is not the right thing to do - yet I don't have any ethical concerns anymore after the authors re-worded the paper."}}]

Author rebuttal:

Meta-review: The main contributions of this paper are porting subword-level policies for Simultaneous Machine Translation to the word level and proposing fusion with a language model.

Altogether the reviewers agree that the paper is technically sound. There are some concerns about the presentation, and one reviewer believes that it will have limited impact due to the main idea being trivial. 
However, given that (some) recent previous work has failed to explore this path, I believe that this only strengthens the merit of publishing the paper.
Decision: Accept-Findings