Simultaneous Machine Translation with Tailored Reference
Authors: Shoutao Guo
Shaolei Zhang
Yang Feng
Venue: EMNLP 2023 Findings (2023)
Keywords: Simultaneous Machine Translation
Machine Translation

Abstract: Simultaneous machine translation (SiMT) generates translation while reading the whole source sentence. However, existing SiMT models are typically trained using the same reference disregarding the varying amounts of available source information at different latency. Training the model with ground-truth at low latency may introduce forced anticipations, whereas utilizing reference consistent with the source word order at high latency results in performance degradation. Consequently, it is crucial to train the SiMT model with appropriate reference that avoids forced anticipations during training while maintaining high quality. In this paper, we propose a novel method that provides tailored reference for the SiMT models trained at different latency by rephrasing the ground-truth. Specifically, we introduce the tailor, induced by reinforcement learning, to modify ground-truth to the tailored reference. The SiMT model is trained with the tailored reference and jointly optimized with the tailor to enhance performance. Importantly, our method is applicable to a wide range of current SiMT approaches. Experiments on three translation tasks demonstrate that our method achieves state-of-the-art performance in both fixed and adaptive policies.

Simultaneous Machine Translation with Tailored Reference
Shoutao Guo 12, Shaolei Zhang 1.2 Yang Feng 1,2-
Key Laboratory of Intelligent Information Processing, Institute of Computing Technology, Chinese Academy of Sciences (ICT/CAS)
2 University of Chinese Academy of Sciences, Beijing, China guoshoutao22zéict..cc cn, zhangshaole 20z@ict, cn, fengyang@ict.ac. at..iitttc.c...
Abstract
Source Sentence
(watchiaa movie) (15)
(an)
(enjoyable) (activity)
Simultancous machine tramslation (SiMT) gen- erates translation while reading the whole source sentence. However, existing SIMT mod- els are typically trained using the same ref- erence disregarding the varying amounts of available source information at different la- tency. Training the model with ground-truth at low latency may introduce forced anticipations, whereas utilizing reference consistent with the source word order high latency results in performance degradation. Consequently, it is crucial to train the SiMT model with appropri- ate reference that avoids forced anticipations during training while maintaining high quality. this paper, we propose a novel method that provides tailored reference for the SiMT mod- els trained different latency by rephrasing the ground-trune trouth Specifically, we introduce the tailor, induced by reinforcement learning, to modify ground-truth to the tailored reference.
Gnund-Truth
is enjoyable activity to watch movie
Tailored Ref
Watching a movie is an enjoyable activity
Figure An example Chinese--ngliss parallel sen- tence. The SiMT model will be forced predict "an enjoyable activity' before reading corresponding source tokens. contrast, the tailored reference avoids forced anticipations while maintaining the original semantics.
different latency should exhibit excellent transla- tion performance.
Using an inappropriate reference train the SiMT model can significantly impact its perfor- mance. The optimal reference for the SiMT model trained at different latency varies. Under high la- tency, it is reasonable to train the SiMT model with ground-truth since the model can leverage suffi- cient source information (Zhang and Feng, 2022c), However, under low latency, the model is strained by limited source information and thus requires reference consistent with the source word order (Chen et al., 2021). Therefore, the SiMT model should be trained with corresponding appro- priate reference under different latency.
SiMT model is trained with the tailored reference and jointly optimized with the tai- lor to enhance performance. Importantly, our method applicable wide range cur- rent SiMT approaches. Experiments three translation tasks demonstrate that our method achieves state-of-the- performance in both fixed and adaptive policies'.
However, the existing SiMT methods, which em- ploy fixed adaptive policy, commonly utilize only ground-truth for training across different la- tency settings. For fixed policy (Ma et al., 2019; Elbayad et al., 2020; Zhang and Feng, 2021), the model generates translations based on the prede- fined rules. The SiMT models are often forced to anticipate target tokens with insufficient informa- tion or wait for unnecessary source tokens. For adaptive policy (Ma et al., 2020; Miao et al., 2021; Zhang and Feng. 2022b), the model can adjust its translation policy based on translation status. Nev- ertheless, the policy learning of SiMT model will gradually adapt to the given reference (Zhang et al., 2020). Consequently, employing only ground-truth
1 Introduction
Simultaneous machine translation (SiMT) (Gu al., 2017; Ma et al., 2019, 2020) generates the target sentence while reading in the source sen- tence. Compared to Full-sentence translation, faces a greater challenge because it has to make trade-off' between latency and translation quality (Zhang and Feng, 2022a). In applications, it needs to meet the requirements of different latency tol- erances, such online conferences and real-time subtitles. Therefore, the SiMT models trained
*Corresponding author: Yang Feng.
'Code is available at https: thub. wtt.tth........l...t/et/ com/ictnlp/ Tailored-Ref

---

for the SiMT models trained at varying latency lev- els can negatively impact overall performance, as it forces them to learn the identical policy, Further- more, Chen et al. (2021) adopts an offline approach to generate reference using the Full-sentence model for training the SiMT model at different latency, but this approach also imposes an upper bound on the performance of the SiMT model. Therefore, it is necessary to provide high-quality and appropriate reference for the models with different latency.
k
3
5
7
9
A AR[%]
28.17
8.68
3.49
1.12
0.49
Table 1: The AR()) on WMTI5 De->En test set at different latency. k belongs Wait-k policy (Ma al., 2019) and represents the number of tokens that the target sentence lags behind the source sentence.
at that latency. To achieve this, the tailor learns from non-anticipatory reference corresponding that latency, which can be generated by applying Wait-k policy to Full-sentence model (Chen et al., 2021). On the other hand, tailored reference should maintain high quality, which can be achieved by encouraging the tailor to learn from ground-truth. Therefore, we mcasure the similarity between the output of tailor and both non con-nnicipatoory reference and ground-truth, assigning them as separate wards. The tailor be optimized by striking balance between these two rewards. During train- ing, the SiMT model takes the output of the tailor as the objective and jointly optimized with the tailor. Additionally, our method is applicable to a wide range of SiMT approaches. Experiments on three translation tasks demonstrate that our method achieves state-of-the-art performance in both fixed and adaptive policies.
Under these grounds, aim to dynamically pro- vide an appropriate reference for training the SiMT models at different latency. In SiMT, the source in- formation available to the translation model varies with latency (Ma et al., 2019). Therefore, the appro- priate reference should allow the model to utilize the available information for predicting target to- kens accurately. Otherwise, it will result in forced anticipations, where the model predicts target to- kens in reference using insufficient source infor- mation (Guo et al., 2023). To explore the extent of forced anticipations when training the SiMT model with ground-trume truth at different latency, we introduce anticipation rate (AR) (Chen et al.. 2021). As shown in Table 1, the anticipation rate decreases the SiMT is trained with higher latency. Con- sequently, the reference requirements of the SiMT model vary at different latency. To meet the re- quirements, the appropriate reference should avoid forced anticipations during training and maintain high quality. Therefore, we propose to dynamically tailor reference, called tailored reference, for the training of SiMT model according to the latency, thereby reducing forced anticipations. We present an intuitive example of tailored reference in Figure 1. It can avoid forced predictions during training while maintaining the semanties consistent with the original sentence.
2 Background
For SiMT task, the model reads in the source tence (I1 (.,,,..,..)) with length J and generates the target sentence y (y/,..., yr) with length I based on the policy. To describe the policy, we de- fine the number of source tokens read when trans- lating y/ as Js- Then the policy can be represented .... gr). Therefore, the SiMT model can
as g = (g1...., be trained by minimizing the cross-entropy loss:
Therefore, we propose new method for pro- viding tailored reference SiMT models differ- ent latency. To accomplish this, we introduce the tailor, a shallow non-autoregressive Transformer Decoder (Gu et al., 2018), to modify ground-truth to the tailored reference. Since there is no explicit supervision to train the tailor, we quantify the re- quirements for the tailored reference two reward functions and optimize them using reinforcement learning (RL). On the one hand, tailored reference should avoid foreed anticipations, ensuring that the word reorderings between it and the source sen- tence can be handled by the SiMT model trained
Lsimt =
Σ log p(uf
X Yes),
(1)
where yf represents the ground-truth token. Our approach involves Wait-k (Ma et al., 2019),
HMT (Zhang and Feng, 2023b) and CTC training (Libovický and Helcl, 2018), we briefly intro- duce them.
Wait-k policy As the most widely used fixed policy, the model reads in tokens first and then alternates writing and reading a token. It can be formalized as:
= + --- J},
(2)

---

where J indicates the length of the source sentence,
Fround-Truth
Non Amticpatory Roference
HMT Hidden Markov Transformer (HMT), which derives from the Hidden Markov Model, is the current state-of-the-ar SiMT model. It treats the translation policy g as hidden events and the target sentence as observed events. During train- ing, HMT learns when to generate translation by minimizing the negative log- likelihood of observed events over possible policies:
R
Tailored
Transiation
Liimt
Reference
Collapse
Lineer Softiax
Decoder
Foed-Forward
Reference
Tailor
Cross-Aitention
x.,,
Chmt log Σ ply x. p(g)
Unidirectional Encoder
Self-Attention
CTC CTC (Graves et al., 2006) is applied in non- autoregressive translation (NAT) (Gu et al., 2018) due to its remarkable performance and no need for length predictor. CTC-based NAT will generate sequence containing repetitive and blank tokens first, and then reduce normal sentence based on the collapse function r-1 During training. CTC will consider all possible sequences a, which can be reduced to y using function
Embedding
Embedding
Upsamplc
Source Sentence
Ground-Truth
The architecture of method. The tailor module modifies ground-truth to the tailored reference, which serves as the training target for the SiMT model. The tailor is induced to optimize two rewards by rein- forcement learning.
Figure 2:
Lete - loo
Σ
p(a x)
(4)
of the encoder and modifies ground-truth while considering the word order of the source sentence. Finally, it transforms the output of tailor into the tai- lored reference by eliminating repetitive and blank tokens (Libovický and Helcl, 2018), 8). The tailored reference replaces the ground- truth the training objective for the SiMT model.
acl
where p(a x) is modeled by NAT architecture.
3 Method
In this section, we introduce the architecture of our model, which incorporates tailor into the SiMT model. To train the SiMT model with the tailor, we present a three-stage training method, which the SiMT model benefits from training with tailored reference and is optimized together with the tai- lor. During inference, the SiMT model generates translation according to the policy. The details introduced in the following subsections.
Given the lack of explicit supervision for train- ing the tailor, we quantify the requirements for tailored reference into two rewards and optimize the model through reinforcement learning. We pro- pose three-stage training method for the SiMT model with the tailor, the details of which will be presented in the next subsection.
3.1 Model Architecture
3.2 Training Method
We present the architecture of our method in Figure 2. Alongside the encoder and decoder, our method introduces the tailor module, which is responsible for generating a tailored reference for the SiMT model, utilizing the ground-tre as its input. Con- sidering the efficiency of generating tailored refer- ence, the tailor module adopts the structure of the non-autoreerressive Transformer decoder (Vaswani al., 2017). To enable the tailor generate tailored reference that not limited by the length truth,, it initially upsamples the ground- truth. Subsequently, it cross-aad cross-attends to the output
After incorporating tailor into the SiMT model. is essential to train the SiMT model with the assistance of tailor to get better performance. light of this, we quantify the requirements of the tailored reference into two rewards and propose a novel three-stage training method for the training of our method. First, train the SiMT model using ground-truth and equip the SiMT model with good translation capability. Subsequently, we use a pre-training strategy train the tailor, enabling establish a favorable initial state and converge faster. Finally, we fine-tune the tailor by optimiz-

---

ing the two rewards using reinforcement learning, where the output of the tailor serves as the train- ing target for the SiMT model after being reduced. In the third stage, the tailor and SiMT model are jointly optimized and share the output the of the encoder. Next, we describe our three-stage training method in detail.
where T' is the output length of tailor and a multi- ple of the length yf Subsequently, we can get the normal sequence s by applying collapse function r- to a and the distribution of s is calculated by considering all possible a:
(s( x,y')
Σ Pa(a x,y"). *ET(*)
(6)
Training the Base Model In our architecture, the
tailor cross-attends the output of the encoder to
To make the tailor learn from ground-truth, the tailor is optimized by minimizing the negative log- likelihood:
adjust ground-truth based on source information. As result, before training the tailor module, we need a well-trained SiMT model as the base model. In our method, we choose the Wait-k policy (Ma et al., 2019) and HMT model (Zhang and Feng, 2023b) as the base model for fixed policy and adap- tive policy, respectively. The base model is trained using the cross-entropy loss. Once the training of the base model completed, we optimize the tailor module, which can provide the tailored refer- ence for the SiMT models trained across different latency settings.
Ext = -og x,y'),
(7)
which can be efficiently calculated through dy- namic programming (Graves et al., 2006).
RL Fine-tuning After completing the pre- training, the tailor is already in a favorable initial state. We quantify the requirements for tailored reference two rewards and fine-tune the tailor using reinforcement learning. We then introduce the two reward functions.
Pre-training Tailor The tailor adopts the archi- tecture of a non-autoregressivee decoder (Gu et al., 2018). The non non-autoregressive architecture has demonstrated excellent performance (Qian al., 2020; Huang al., 2022). Importantly, it enables the simultaneous generation of target tokens across all positions, making highly efficient for rein- forcement learning. However, if we train the tailor using reinforcement learning directly, it will con- verge to a suboptimal state in which the tokens at each position are some frequent tokens (Shao et al., 2022). This behavior is attributed to the exploration-based nature of reinforcement learning, highlighting the need for favoralle initial state for the model (Lopes et al., 2012). Since the tailored reference modified from goound-truth, we let learn from ground-truth during pre-training and then fine-tune it using reinforcement learning. The details of pre-traine pre-training stage are introduced below.
On the one hand, the tailored reference should not force the model to predict the target tokens before reading corresponding source information, which means the SiMT model can handle the word reorderings between the tailored reference and the source sentence at that latency (Zhang et al., 2022). Therefore, make the tailor learn from non- anticipatory reference Yna- which is generated by applying the corresponding Wait-h policy to the senteneetence model. has the word order that matches the latency and maintains the original se- mantics (Chen et al., 2021). We employ reward Rna to measure the similarity between the output of tailor and non-anticipatory reference. On the other hand, the tailored reference should remain faithful to ground-truth. We introduce the reward Rgt measure the similarity between ground-trutt and the output of the tailor. By striking an appropri- ate balance between Rna and R.t,, we can obtain the tailored reference.
To keep the output of the tailor from being lim- ited by the length of ground -ruth, the tailor upsam- ples ground-truie truthh get the input of the tailor, de- noted as y'. During training, CTC loss (Libovický and Helcl, 2018) is used to optimize the tailor. Denoting the output of the tailor as a (a1,..,, (y), the probability distribution modeled by the tailor can be presented as:
Given the output of tailor, we can obtain the normal sentence s by removing the repetitive and blank tokens (Libovický and Helel, 2018). We use BLEU (Papineni et al., 2002) measure the similarity between two sequences. Therefore, Rno and Re for the output of tailor calculated as:
T
Rna (s) (s,
(8)
(a x,y')
I (at x,y'),
(5)
(s) -BEEUss
(9)

---

36
80,E3
26-
24-
Fellocrone
Pull aeliiae
20-
Pudi antiane
Watk Taler
Wezkk Taiee
Welkk Talle
MoG Wonik
Mui:: Wark
Mall Wailk
3
Assage Legging
Average (agp)nn(A))
Averuzt Luggteg
(a) En-VVi
(b) En->o
(c) De-+En
Figure 3: Translation performance of different fixed policies En-+V, En-+Ro and De-+En tasks.
32-
28-
25-
26-
Fillserrece
RME Valer
ITSI
HNT
HMI
24-
20-
RMT
MMA
Adiqtice
MMS
Adppii Watk
25
Avonge Logging
Arauge Legging
Assage Legging (AL)
(a) En
(b) En-Ro
(c) De-EEn
Figure 4: Translation performance of different adaptive policies En En-+Vi, En- and De-+En tasks.
Based on these two rewards, we can obtain the final reward R by balancing Rna and Ryt:
(s)) and update the tailor with estimated gradient V log (s x, ())(s) In the calculation of (s y,y 0). we use dynamic programming accelerate the process. Additionally, adopt the baseline reward strategy to reduce the variance of the estimated gradient (Weaver and Tao, 2001).
R(s) = (nna +
(10)
where a E [0, a hyperparameter. Subse- quently, we use REINFORCE algorithm (Williams, 1992) to optimize the final reward R to obtain the tailored reference:
In this stage, we utilize reinforcement learning to optimize the final reward ((s) and train the SiMT model with tailored reference using Lt_simt- As a result, the SiMT model and the tailor are jointly optimized to enhance performance.
VoJ(A) Vo Σ Pa(s) x,y, (s)
E s~Ps
V Po x, x,y,, ())(
4 Experiments
E Vologp. [ (a) x,y, (T (a)).
4.1
Datasets
arya
(11)
We evaluate our method on three translation tasks. IWSLT152
where represents the collapse function and denotes the parameter of tailor. During training, we sample the sequence from the distribution (a y'y f)) using Monte Carlo method. As the tailor adopts non--utorreresssvve structure where all positions independent of each other, can concurrently sample tokens for all positions from the distribution. We then apply collapse function to sequence a to obtain the normal se- quence s, which is used to compute the reward
(En-+Vi) (Cettolo et al., 2015) We use TED tst2012 the development set and TED tst2013 the test set. In line with Ma al. (2020), replace the tokens occurring less than with (unk). Consequently, the vocabulary sizes of English and Vietnamese are 17K and 7.7K, respectively.
WMT16 English Romanian (En-RRo) We
Phttps://nIp stanford. weterdddddupooojeeeeee pdj/str/jets//mm/ www. www.st.ttttt.mm. tttaatt..rg//mttte.t orgg/wt16//

---

newsdev-2016 the development set and newstest-2016 as the set. The source and get languages employ a shared vocabulary. Other
settings are consistent with Gu et al. (2018).
WMT15' German--ngglish (De+Enn) Fol- lowing Ma et al. (2020), we use newstest2013 as development set and newstest2015 as test set. We apply BPE (Sennrich et al., 2016) with 32K sub- word units and use shared vocabulary between source and larget.
26
22
Tailored Reference
Ground
Top40% Pseudo Refs
*+ 100% Pscudo-ReF
18
4.2 System Settings
Average Lagging (AL)
Our experiments involve the following methods and we briefly introduce them.
Figure Translation performance different training methods on Wait-k policy.
Full-sentence model is the conventional full- sentence machine translation model.
Wait-k policy (Ma et al., 2019) initially reads k tokens and then alternates between writing and reading source token.
HMT Tailor applies our method HMT.
All systems are based on Transformer archi- tecture (Vaswani et al., 2017) and adapted from Fairseq Library (Ott et al., 2019). We apply Transformer-Small (6 layers, 4 heads) for En-Vi task and Transform-Bas (6 layers, 8 heads) for En-RRo and De tasks. Since PED and Adap- tive Wait-k do not report the results the En->Ro task, we do not compare them in the experiment. For our method, we adopt the non-regressive de- coder structure with layers for the tailor. We empirically set the hyperparameter a as 0.2. The non-anticipatory reference used for RL Fine-tuning of SiMT model is generated by Test-time Wait-k method (Ma et al., 2019) with corresponding la- tency. Other system settings are consistent with Ma et al. (2020) and Zhang and Feng (2023b). The detailed settings are shown in Appendix C. We use greedy search during inference and evaluate all methods with translation quality estimated by BLEU (Papineni et al., 2002) and latency measured by Average Lagging (AL) (Ma et al., 2019).
Multi-path (Elbayad et al., 2020) introduces the unidirectional encoder and trains the model by sampling the latency
Adaptive Wait-k (Zheng et al., 2020) employs multiple Wait-k models through heuristic method to achieve adaptive policy.
MMA (Ma et al.. 2020) makes each head de- termine the translation policy by predicting the Bernoulli variable.
MoE Wait-k (Zhang and Feng. 2021), the cur- rent state-of-the-aae fixed policy, treats each head as an expert and integrates the decisions of all experts.
PED (Guo et al., 2022) implements the adaptive policy via integrating post-evaluation into the fixed translation policy.
BS-SIMT (Guo al., 2023) constructs the opti- mal policy online via binary search.
ITST (Zhang and Feng. 2022b) treats the transla- tion as information transport from source to target. HMT (Zhang and Feng, 2023b) models simul- tancous machine translation Hidden Markov Model, and achieves the current state-of-the-arr performance in SiMT.
4.3 Main Results
The performance comparison between our method and other SiMT approaches on three translation tasks illustrated Figure and Figure 4. Our method achieves state-of-the-art translation perfor- mance in both fixed and adaptive policies. When comparing with other training methods in Figure 5, our approach also achieves superior performance.
* ++100% Pseudo-Refs (Chen et al., 2021) trains the Wait-k model with ground-truth and pscudo reference, which is generated by applying Wait-k policy to the Full-sentence model.
*TTop 40% Iseudo-Refi (Chen et al., 2021) fil- pseudo references in the top 40% of quality train the model with ground-truth.
When selecting the most commonly used Wait- policy as the base model, our method outper- forms MoE Wait-k, which the current state-of- the-art fixed policy. Compared to Wait-k policy,
Wait-k + Tailor applies our method on Wait-k.
www. www.st.tttttmmmmm...... www.tt..tgg/tttttt org/wmtt/// /wmm5//////

---

N₂
2
4
Method
AL
BLEU
AL
1.50
1.89
1.84
0.1
1.72
24.82
BLEU
24.43
25.90
25.46
Wait-k + Tailor
0.2
1.89
25.90
0.3
1.95
25.30
Table 2:
Performance of the SiMT model when the tailor has different number layers.
w/o Base Model
0.2
1.77
22.89
w/o Pre-training
0.2
1.80
24.66
w/o RL Fine-tuning
0.2
1.86
24.60
our method brings significant improvement, espe- cially under low latency. Wait-k policy is trained on ground and cannot be adjusted, which may force the model to predict tokens before reading corresponding source information (Ma et al., 2019). In contrast, our method provides tailored refer- ence for the SiMT model, thereby alleviating the issue of forced anticipations. Our method also exceeds Multi-path and MoE Wait-k. These two methods are trained using multiple Wait-k policies (Elbayad et al., 2020) and gain the ability to trans- late under multiple latency (Zhang and Feng, 2021), )), but they still utilize ground-truth at all latency, lead- ing to lower performance.
Table 3:
Ablation study on training method of the tai- lor and ratio between two rewards. "w/o Base Model' removes the training stage of the base model. "w/o Pre- Pre-training' removes the pre-training stage. "w/o RL Fine-tuning" removes the RI fine-tuning stage.
Analysis
To gain comprehensive understanding of method, conducted multiple analyses. All of the following results are reported on De-+En task.
5.1 Ablation Study
Our method can further enhance the SiMT per- formance by selecting adaptive policy as the base model. As the current ofte-oof-the-rt adaptive pol- icy, HMT possesses the ability to dynamically ad- just policy balance latency and translation quality (Zhang and Feng. 2023b). However, still relies on ground-truth for training SiMT models across different latency settings. By providing tailored reference that matches the latency, our method can alleviate the latency burden of the SiMT model, resulting in state-of-the performance.
We conduct ablation studies on the structure and training method of tailor to investigate the influ- ence of different settings. The experiments all use Wait-k model the base model with k set to 3. Ta- ble 2 presents comparison of different structures. The best performance is achieved when the tailor has 2 layers. The performance affected by both excessive layers and insufficient layers. Table 3 illustrates the results of the ablation study on the training method. Each stage of the training method contributes to the performance of the SiMT model and the training stage of the base model has the most significant impact the per- formance. This can be attributed to the fact that well-trained encoder information to the tailor, enabling the generation of appropriate tailored references. Additionally, when a selected as 0.2, our method yiclds the best per- formance, indicating an optimal balance between word order and quality for the tailor.
be negatively
Our method also surpasses other training ap- proaches. Ground-truth is not suitable for incre- mental input due word reorderings, resulting in performance degradation (Zhang and Feng, 2022b). On the contrary, pseudo reference can avoid forced anticipations during training (Chen al., 2021). However, constructed offline by applying the Wait-k policy on the Full-sentence model. It im- poses an upper bound on the performance of the SiMT model. The tailored reference avoids forced anticipations while maintaining high quality, lead- ing the best performance.
provide accurate source
5.2 Analysis of Tailored Reference
Anticipation Rate Furthermore, we conduct an analysis of the tailored reference assess its influ- ence. We first explore the rate of forced anticipa- tions caused by using different references during training. Using the anticipation rate (AR) (Chen et al., 2021) as the metric, the results in Table 4 show that the tailored reference can effectively re-
In addition to enhancing translation performance, method effectively narrows the gap between fixed and adaptive policies. By leveraging our method, the SiMT model can achieve compara- ble performance to Full-sentence translation with lower latency on En Vi and De-+En tasks.

---

k
1
3
5
7
Wait-k
9.0
Wait-k Taibor
Ground-Trus 28.17 7
8.68
3,49
1.12
IIM
Tailored Ref
19.84
8.29
2.98
0.90
0.5
LIM Tailor
0.0
Table 4: The anticipation rate (AR[%]) when applying the Wait-k policy on different references, which are based on De test set.
7.5
7.0
k-
3
5
6.5
Tailored Ref
79.67
86.01
84.83
92.40
6.0
2
Non-Anti Ref
21.87
24.07
26.29
27.24
Average Lagging (AI.)
Figure 6: The hallucination rate (HR) (Chen et al., 2021) of different methods. measures the proportion of to- kens in translation that cannot find corresponding source information.
Table 5: The quality (BLEU) of difference references compared to ground-truth for training of Wait-k: pol- icy. 'Non-Anti Ref" represents the reference generated by applying Wait-k policy on Full-sentence model.
fectively mitigates the hallucinations by alleviating the burden of latency. This signifies that minimiz- ing forced predictions during training can enhance the faithfulness of the translation to the source sen- tence, thereby improving translation quality (Ma et al., 2023).
duce the forced anticipations during the training the SiMT model under all latency. This implies that, compared to ground-truth, the word reorder- ings between the tailored reference and the source sentence can be more effectively handled by the SiMT model at different latency.
Related Work
Quality However, one concern is whether the quality of tailored reference will deteriorate like non-anticipatory reference after adjusting the word order. To assess this, we compare different ref- erences with ground-truth to measure their qual- ity. As shown in Table 5, we observe that the tai- lored reference exhibits significantly higher quality than the non-anticipatory reference. Therefore, our method successfully reduces the rate of forced an- ticipations during training while remaining faithful ground-truth. To provide better understand- ing of the tailored reference, we include several illustrative cases in Appendix
Simultaneous machine translation (SiMT) gener- ates translation while reading the source sentence. It requires a policy to determine the source informa- tion read when translating each target token, thus striking balance between latency and translation quality. Current research on SiMT mainly focuses on two areas: policy improvement and adjustment of the training method.
For policy improvement, it aims to provide suffi- cient source information for translation while avoid- ing unnecessary latency. Ma et al. (2019) propose Wait-k: policy, which initially reads k tokens and alternates between writing and reading one token. Zhang and Feng (2021) enable each head obtain the information with a different fixed latency and integrate the decisions of multiple heads for trans- lation. However, the fixed policy cannot be flexibly adjusted based on context, resulting in suboptimal performance. Ma et al. (2020) allow each head to determine its own policy and make all heads decide the translation. Miao al. (2021) propose gen- erative framework, which uses re-parameterized Poisson prior to regularising the policy. Zhang and Feng (2023a) propose segmentation policy for the source input. Zhang and Feng (2023b) model the simultaneous machine translation as Hidden
5.3 Hallucination in Translation
If the SiMT model is forced to predict target tokens before reading corresponding source information during training, there is a high likelihood of gen- erating hallucinations during inference (Ma et al., 2019). To quantify the presence of hallucinations in the translation, we introduce hallucination rate (HR) (Chen et al., 2021) for evaluation. Figure 6 illustrates that the SiMT model trained with the tailored reference demonstrates reduced probabil- ity of generating hallucinations. Moreover, even though the adaptive policy can adjust its behavior based on the translation status, our approach still ef-

---

Markov Model and achieve state-of-the-art perfor- mance. However, these methods are all trained with ground-truth, leading to forced predictions at low latency.
References
Mauro Ccttolo, Jan Nichues, Sebastian Stüker, Luisa Bentivogli, Roldano Cattoni, and Marcello Federico. 2015. The IWSLT 2015 evaluation campaign. Proceedings of the 12th International Workshop on Spoken Language Translation: Evaluation Cam- puign @/WSLT 2015, Du Nang. Vietnam, December 3-4, 2015.
For the adjustment of the training method, it wants to supplement the missing full-sentence in- formation or cater to the requirements of latency, Zhang et al. (2021) shorten the distance of source hidden states between the SiMT model and the Full-sentence model. This makes the source hid- den states implicitly embed future information, but encourages data-driven prediction. On the other hand, Chen et al. (2021) try to train the model with non-anticipatory reference, which can be ef- fectively handled by the SiMT model at that latency. However, while non-anticipatory reference can al- leviate forced predictions at low latency, it hinders performance improvement at high latency.
Junkun Chen, Renjie Zheng, Atsuhito Kita, Mingbo Ma, and Liang Huang. 2021. Improving simultaneous translation by incorporating pseudo-references with fewer reorderings. In Proceedings of the 2021 Con- ference on Empirical Methods in Natural Language Processing, EMNLP 2021, Virtual Event / Punta Cana, Dominican Republic, 7-11 November, 2021, pages 5857-5864. Association for Computational Linguistics.
Maha Elbayad, Laurent Besacier, and Jakob Verbeek. 2020. Efficient wait-k models for simultaneous ma- chine translation. In Interspeech 2020, 21st Annual Conference of the International Speech Communi- cation Association, Virtual Event, Shanghai, China, 25-29 October 2020, pages 1461-1465. ISCA.
Therefore, we want provide tailored ref- erence for the SiMT models trained at different latency. The tailored reference should avoid forced anticipations and exhibit high quality. In view of the good structure and superior performance of the non-autoregressive model (Gu et al., 201 8; Li- bovický and Helel, 2018), we utilize it to modify the ground-truth to the tailored reference.
Alex Graves, Santiago Fernández, Faustino J. Gomez. and Jürgen Schmidhuber. 2006. Connectionist tem- poral classification: labelling unsegmented sequence data with recurrent neural networks. In Machine Learning, Proceedings of the TwenryThhird Interna- tional Conference (ICML 2006), Pittsburgh, Pennsyl- vania, USA, June 25-29, 2006, volume 148 of ACM International Conference Proceeding Series, pages 369 669-376. ACM.
7 Conclusion
In this paper, we propose a novel method to pro- vide tailored reference for the training of SiMT model. Experiments and extensive analyses demon- strate that our method achieves state-of-the-ee per- formance in both fixed and adaptive policies and effectively reduces the hallucinations in translation.
Jiatao Gu, James Bradbury, Caiming Xiong, Victor O. K. Li, and Richard Socher. 2018. Non-autoregressive neural machine translation. In 6th International Con- ference Learning Representations, ICLR 2018 8, Vancouver, BC, Canada, April 30 May 3, 2018, Conference Track Proceedings. OpenReview.net.
Limitations
Jiatao Gu, Graham Neubig, Kyunghyun Cho, and Vic- tor O.K. Li. 2017. Learning to translate in real-time with neural machine translation. In Proceedings of the I 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 1, Long Papers, pages 1053-1062, Valencia, Spain. Association for Computational Linguistics.
Regarding the system settings, we investigate the impact of the number of layers and training meth- ods performance. We think that further explo- ration of system settings could potentially yield even better results. Additionally, the tailor mod- ule aims to avoid forced anticipations and main- tain faithfulness to ground-truth. language-related features to the SiMT model using a heuristic method, it may produce more suitable references for the SiMT model. We leave it for future work.
Shoutao Guo, Shaolei Zhang, and Yang Feng. 2022. Turning fixed to adaptive: Integrating post-evaluation into simultancous machine translation. In Findings of the Association for Computational Linguistics: EMNLP 2022, Abu Dhabi, United Arah Emirates, December 7-11, 2022, pages 2264-2278. Associa- tion for Computational Linguistics.
we can add
Shoutao Guo, Shaolei Zhang, and Yang Feng. 2023, Learning optimal policy for simultancous machine translation via binary search. In Proceedings of the 61st Annual Meeting of the Association for Compu- tational Linguistics (Volume I: Long Papers), pages
Acknowledgment
We thank all anonymous reviewers for their valu- able suggestions.

---

2318-2333, Toronto, Canada. Association for Com- putational Linguistics.
Kishore Papineni. Salim Roukos, Todd Ward, and Wei- Jing Zhu. 2002. Bleu: a method for automatic evalu- ation of machine translation. In Proceedings of the 40th Annual Meeting of the Association for Compu- national Linguistics, pages 31 5-1-118, Philadelphia. Pennsylvania, USA. Association for Computational Linguisties.
Fei Huang, Hao Zhou, Yang Liu, Hang Li, and Minlie Huang. 2022. Directed acyclic transformer for non- autoregressive machine translation.
Jindfich Libovický and Jindřich Helel. 2018. End-to- end non-autoregreesive neural machine translation with connectionist temporal classification. In Pro- ceedings of the 2018 Conference on Empirical Meth- ods in Natural Language Processing, pages 3016- 3021, Brussels, Belgium. Association for Computa- tional Linguistics.
Lihua Qian, Hao Zhou, Yu Bao, Mingxuan Wang. Lin Qiu, Weinan Zhang, Yong Yu, and Lei Li. 2020. Glancing transformer for non-autoregressive neural machine translation. arXiv preprint arXiv:2008.07905.
Rico Sennrich, Barry Haddow, and Alexandra Birch. 2016. Neural machine translation of rare words with subword units. In Proceedings of the 54th Annual Meeting of the Association for Computational Lin- guistics (Volume 1: Long Papers), pages 1725- Berlin, Germany. Association for Computational Lin- guistics.
Manuel Lopes, Tobias Lang, Mare Toussaint, and Pierre- yves Oudeyer. 2012. Exploration model-based re- inforcement learning by empirically estimating Ieamn- ing progress. In Advances in Neural Information Processing Systems, volume 25. Curran Associates, Inc.
Chenze Shao, Jinchao Zhang. Jie Zhou, and Yang Feng. 2022. Rephrasing the reference for non-auttoregressive machine translation. CoRR, abs/2211.16863.
Mingbo Ma. Liang Huang. Hao Xiong, Renjie Zheng. Kaibo Liu, Baigong Zheng. Chuanqiang Zhang, Zhongjun He. Hairong Liu, Xing Li. Hua Wu, and Haifeng Wang. 2019. STACL: simultaneous trans- lation with implicit anticipation and controllable latency using prefix-to-prefix framework. In Pro- ceedings of the 57th Conference of the Association for Computational Linguistics, ACI. 2019, Florence, Italy, July 28- August 2, 2019, Volume 1: Long Pa- pers, pages 3025-300 -036.. Association for Computa- tional Linguistics.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones. Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Advances in Neural Information Pro- cessing Systems 30: Annual Conference on Neural Information Processing Systems 2017, December 4-9, 2017, Long Beach, CA, USA, pages 5998-6008.
Xutai Ma. Juan Miguel Pino, James Cross, Liezl Puzon, and Jiatao Gu. 2020. Monotonic multihead attention. In 8th International Conference Learning Repre- sentations, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020. OpenReview.net.
Lex Weaver and Nigel Tao. 2001. The optimal reward baseline for gradient- based reinforcement learning. In UAI '01: Pruceedings of the 17th Conference in Uncertainty in Artificial Intelligence, University of Washington, Seattle, Washington, USA, August 2-5, 2001, pages 3388.45. Morgan Kaufmann.
Zhengrui Ma, Shaolei Zhang, Shoutao Guo, Chenze Shao, Min Zhang, and Yang Feng. 2023. Non- autoregressive streaming transformer for simultane- translation. In Proceedings of the 2023 Con- ference on Empirical Methods in Natural Language Processing.
Ronald J. Williams. 1992. Simple statistical gradient- following algorithms for connectionist reinforcement learning. Mach. Learn.. 8:229-2555 226..
Ruiqing Zhang, Chuanqiang Zhang. Zhongjun He, Hua Wu, and Haifeng Wang. 2020. Learning adaptive segmentation policy for simultaneous translation. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (FMNLP), pages 2280-2289, Online, Association for Computa- tional Linguistics.
Yishu Miao, Phil Blunsom, and Lucia Specia. 2021. A generative framework for simultaneous machine translation. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Process- ing, EMNLP 2021, Virtual Event / Punta Cana, Do- minican Republic, 7-11 November, 2021, pages 6697- 6706. Association for Computational Linguisties.
Shaolei Zhang and Yang Feng. 2021. Universal simul- taneous machine translation with mixture-of-experts wait-k policy. In Pruceedings of the 2021 Confer- ence on Empirical Methods in Natural Language Pro- cessing, EMNLP 2021, Virtual Event / Punta Cana, Dominican Republic, 7-11 November, 2021, pages 7306-7317. Association for Computational Linguis- tics.
Myle Ott, Sergey Edunov, Alexei Baevski, Angela Fan, Sam Gross, Nathan Ng. David Grangier, and Michael Auli, 2019. fairseq: A fast, extensible toolkit for sequence modeling, In Proceedings of the 2019 Con- ference of the North American Chapter of the Asso- ciation for Computational Linguistics: Human Lan- guage Technologies, NAACL-HLT 2019, Minneapo- lis, MN, USA, June 2-7, 2019, Demonstrations, pages 48-53. Association for Computational Linguistics.
Shaolei Zhang and Yang Feng. 2022a. Gaussian multi- head attention for simultaneous machine translation.

---

In Findings the Association for Computational Linguistics: ACL 2022, pages 3019-3030, Dublin, Ireland. Association for Computational Linguisties.
A Anticipation & Hallucination Rate
In our analyses, we evaluate the reference and trans- lation using anticipation rate (AR) and hallucina- tion rate (HR), respectively. We then introduce the calculation method of these two evaluation metrics in detail.
Shaolei Zhang and Yang Feng. 2022b. Information- transport-based policy for simultaneous translation. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 992- 1013, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics.
Given a sentence pair (x, y), there exists an alignment set h, which is a set of source-target index pairs (j. i) where yth source token zy aligns to the jth target token y/- If we apply the policy g = (g1,..., 91) on this sentence pair, the target token is forcibly anticipated (A(é, h, g) = 1) if it aligms to least one source token zy where j 9::
Shaolei Zhang and Yang Feng. 2022c. Modeling dual read/write paths for simultancous machine transla- tion, In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Vol- ume 1: Long Papers), pages 2461-24424 2461-2477, Dublin, Ireland. Association for Computational Linguisties.
Shaolei Zhang and Yang Feng. 2023a. End-to-end si- multancous speech translation with differentiable seg- mentation. In Findings of the Association for Com- putational Linguisties: ACL 2023. Association for Computational Linguistics.
A(i, h. g) = i hlj > g + Ø). (12)
Therefore, we can define the anticipation rate (AR) of (x, y, h) under the policy g:
Shaolei Zhang and Yang Feng. 2023b. Hidden markov transformer for simultaneous machine translation. CoRR, abs/2303.000
1 v
AR(x.y, h,
h(i,
(13)
Shaolei Zhang, Yang Feng, and Liangyou Li. 2021. Future- guided incremental transformer for simulta- neous translation. In Thirty-Fijy Fifth AAAI Conference on Artificial Intelligence, AAAI 2021, Thirty-Thing Conference on Innovative Applications of Artificial Intelligence, IAAI 2021, The Eleventh Symposium on Educational Advances Artificial Intelligence, EAAI 2021, Virtual Event, February 2-9, 2021, pages 14428 14436. AAAI Press.
Future-guided
The anticipation rate is a metric used to quantify the degree to which the target token is predicted before reading all relevant source tokens.
We then introduce the hallucination rate (HR). We first define the translation as y. A target token is in translation ý is hallucination (H (i, h)=1) if can not be aligned any source token:
Shaolei Zhang, Shoutao Guo, and Yang Feng. 2022. Wait-info policy: Balancing source and target at in- formation level for simultancous machine translation. In Findings the Association for Computational Linguistics: EMNLP 2022, pages 2249-2263, Abu Dhabi, United Arab Emirates. Association for Com- putational Linguistics.
H 1 E h = 2).
(14)
Therefore, the hallucination rate can be defined as:
| H(i.h).
HR(x,
(15)
Baigong Zheng, Kaibo Liu, Renjie Zheng, Mingho Ma, Hairong Liu, and Liang Huang. 2020. Simultane- ous translation policies: From fixed to adaptive. In Proceedings of the 58th Annual Meeting of the As- sociation for Computational Linguistics, ACL 2020, Online, July 5-10, 2020, pages 2847-2853. Associa- tion for Computational Linguistics.
B Case Study
We also provide two cases in the De En test set to understand our method. The cases are shown Figure 7 and Figure 8. It presents that the tailored reference is more consistent with the word order requirements of the specific latency.
In Figure 7, if we train the SiMT model on Wait- 1 policy with the ground-truth it will be forced to predict 'with him' before reading 'mit ihm' during training. However, training the SiMT model with tailored reference will eliminate forced predictions by adjusting 'like that" to 'such' and positioning forward. Importantly, the tailored reference also maintains the original semantics. This shows that

---

Source
ich hatte nie ein solches Gespräch mit ihm
Sentenee
()) (had) (never) (a) (such) (conversation) (with) (him)
Ground-Truth
never
had conversation with
him like that
Tailored Ref
had
never
had
such
conversation
with him
Figure 7: Case study of #319 in De-TEn test The tokens marked in the color share the semantics.
The tailored reference is more suitable when the model is trained with Wait-1 policy.
und einer Online
@-@
Bestellung sollte prüfen
Source
(and) (before) (an) (online)
(order)
(should) (you) (check)
(so)
Sentence
Gault ob das Buch tatsächlich vorrätig ist
(Gault) (if) (the) (book) (actually)
(in stock) (is)
Ground-Truth before ordering online Gault says be sure to check if the book is actually in stock
and before online order Gault says be sure check if the book is actually in stock.
Tailored Ref
Figure 8: Case study of #1869 in De-EE test set. The tokens marked in the same color share the same semantics. The tailored reference more suitable when the model trained with Wait-1 policy.
the order of the source sentence and the tailored sentence consistent, which makes it suitable for Wait-1 policy.
In Figure 8, using ground-truth as the training target Wait-1 policy also forces the model to pre- dict 'ordering online' before reading 'Online' and 'Bestellung'. In contrast, by replacing 'ordering online' with 'n online order', the word order of tailored reference is the same the source sen- tence, thereby avoiding forced anticipations during the training of Wait-1 policy.
Hyperparameters
The system settings on three translation tasks are shown in Table 6. For more detailed implementa- tion issues, please refer to our publicly available code.
D Numerical Results
addition to the translation performance compari- son in Figure and Figure 4, we also provide cor- responding numerical results for reference. Table 7, 8, respectively report the numerical results IWSLT15 En-+Vi, WMT16 En En and WMT15 De-+En measured by AL (Ma al., 2019) and BLEU (Papineni al., 2002).

---

Hyperparameter encoder layers encoder attention heads encoder embed dim encoder ffn embed dim decoder layers
IWSLT15 En-
WMT16 En On->Ro Ro
WMT15 De
6
6
6
4
8
8
512
512
512
1024
2048
2048
6
6
6
decoder attention heads decoder embed dim decoder ffnn embed dim tailor layers
8
8
512
512
512
1024
2048
2048
2
2
tailor attention heads tailor embed dim tailor ffn embed dim dropout
8
8
8
512
512
512
2048
2048
2048
0.1
0.3
0.3
optimizer adam-33
adam
adam
adam
(0.9, 0.98)
(0.9, 0.98)
(0.9, 0.98)
clip-norm Ir
0
5e-4
5e-4
5e-4
Ir scheduler
inverse sqrt
inverse sqrt
inverse sqrt
warmup-updates
4000
4000
4000
warmup-init-Ir weight decay
le-7
le-7
le-7
0.0001
0.0001
0.0001
label-smoothing
0.1
0.1
0.1
max tokens
16000
8192x4
8192x4
Table 6: Hyperparameters of experiments.

---

IWSLTI5 En-Vi
WMT16 En-RRo
Full-sentence
AL
BLEU
Full-sentence
22.41
28.8
AL
BLEU
n/a
32.74
Wait-k
k
AL
BLEU
Wait-k
3.03
25.28
k
AL
BLEU
3
4.64
27.53
1
2.09
20.67
5
6.46
28.27
3
3.01
25.69
8.11
28.45
5
5.09
29.28
9.80
28.53
7.21
30.50
9
9.03
30.87
MoE Wait-k
AL
BLEU
MoE Wait-k
3.19
26.56
k
AL
BLEU
3
4.70
28.43
1
2.00
23.50
5
6.43
28.73
3
3.30
28.25
7
8.19
28.81
5
5.15
30.94
9.86
28.88
7
7.00
31.19
9
9.00
31.33
Wait-k Tailor
AL
BLEU
Wait-k + Tailor
2.96
27.58
k
AL
BLEU
3
4.49
29.03
1
1.48
23.72
5
6.24
29.25
3
3.07
28.30
7
8.01
29.45
5
4.93
31.04
9
9.70
29.49
7
7.02
32.13
9
8.85
31.97
HMT
L, K
AL
BLEU
HMT
1,2
3.05
28.02
L, K
AL
BLEU
2,2
3.72
28.53
2,4
1.73
24.09
4,2
4.92
28.59
3,6
3.34
28.92
5,4 4
6.34
28.78
5,6
5.33
31.02
6,4 4
8.15
28.86
7.6
7.46
31.97
7,6
9.60
28.88
9.8
9.27
32.18
HMT + Tailor
HMT + Tailor
L, K
AL
BLEU
L K
AL
BLEU
1,2
3.02
28.28
2, 4
1.71
24.80
2,2
3.70
29.02
3,6
3.27
29.11
4,2
5.31
29.24
5,6
5.26
31.42
5,4
6.28
29.39
7,6
7.34
32.16
6,4
8.04
29.34
9.8
9.24
32.13
7,6
9.78
29.39
Table 8: Numerical results of WMT16 En-
Table 7: Numerical results of IWSLT15 En-+Vi.

---

WMT16 En
Full-sentence
AL
BLEU
n/a
31.76
Wait-k
k
AL
BLEU
0.02
17.61
3
1.71
23.75
5
3.85
26,86
5.86
28.20
7.85
29.42
MoE Wait-k
AL
BLEU
-0.18
21.40
3
1.82
25.35
5
3.98
27.50
7
5.97
29.07
7.88
29.52
Wait-k Tailor
AL
BLEU
-0.32
21.12
1.89
25.90
5
3,89
28.44
7
5.91
29.80
9
7.80
30.64
HMT
L, K
AL
BLEU
1,4
0.27
22.52
2,4 4
2.20
27.60
3,6
3.46
29.29
5,6
4.74
30.29
7.6
6.43
30.90
9,8
8.36
31.45
HMT + Tailor
K.
AL
BLEU
- 1,4 4
0.03
23.75
2, 1
2.10
28.12
3,6 6
3.13
29.99
5,6 6
4.60
30.74
7,6
6.31
31.48
9,8
8.18
31.87
Table 9; Numerical results WMT15 De-*En.


Reviews:
[{"review_id": "HTv2JC9zP8", "reviewer": "Reviewer_NbPY", "summary": "", "questions": "1. I am not familiar with reinforcement learning, but what is the action and policy of this tailor model? Also, providing a plot of the reward history curve would be helpful. \n2. Could you explain more on average lagging (x-axis of Figure 3,4,5), non-anticipatory references, and test-time wait-$k$ methods? It would be useful to include more details about them in the paper.\n3. The paper misses some training details (number of epochs, learning rate) in the experimental section. Also, how to input $k$ when training the model? \n4. Providing more examples of tailored references would be more helpful.\n", "limitations": "", "rating": 4, "confidence": 3, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for your detailed and valuable feedback.\n\n&ensp;\n\nQ1: What is the action and policy of the tailor model?\\\nA1: Reinforcement learning guides the tailor in generating references that meet latency requirements and are of high quality. The action is the output of the tailor, denoted as **a** in the paper. The policy for generating actions employs a greedy policy. Following your suggestions, we will add the plot of the reward history curve in the next version.  \n\n&ensp;\n\nQ2: Could you explain more on average lagging, non-anticipatory references, and test-time wait-*k* methods?\\\nA2: **Average Lagging (AL)** [1]  is the most widely used latency metric, which measures the degree to which the evaluated policy lags behind the ideal policy (wait-*0* policy). For example, when the AL is equal to 1, it signifies that the evaluated policy lags behind the ideal policy by one token on average.\\\n&ensp;&ensp;&ensp; In order to guide the tailored references to exclude hallucinations, **non-anticipatory references**, which are generated by the **test-time wait-*k*** method, serves as the optimization direction for tailor. The **test-time wait-*k*** method implements the wait-*k* policy on a conventional full-sentence translation model. Because complete source content is always available during training, the translations generated by the  **test-time wait-*k*** method are unlikely to hallucinate. As a result, they are referred to as **non-anticipatory references** [2].\\\n&ensp;&ensp;&ensp; We will follow your suggestions to include more explanations about AL, tailored references, and test-time wait-*k* method in the next version.\n\n[1] Ma et al., 2019. STACL: Simultaneous Translation with Implicit Anticipation and Controllable Latency using Prefix-to-Prefix Framework. In ACL2019.\\\n[2] Chen et al., 2021. Improving simultaneous translation by incorporating pseudo-references with fewer reorderings. In EMNLP2021.\n\n&ensp;\n\nQ3:  Miss some training details in the experiment section?  \nA3: I apologize for any misunderstanding I may have caused. Due to space limitations, we mention in Section 4.2 that experimental details not listed are kept consistent with those of [3]. The experimental settings are presented in the following table:\n\n| Hyperparameter | WMT15 De->En | WMT16 En->Ro | IWSLT15 En->Vi |\n| :-----------------------| :---------------------------: | :---------------------------: | :---------------------------: |\n| model | Transformer Base | Transformer Base | Transformer Small |\n| dropout | 0.3 | 0.3 | 0.1 |\n| optimizer | adam | adam | adam |\n| adam-$\\beta$ | (0.9, 0.98) | (0.9, 0.98) | (0.9, 0.98) |\n| lr | 5e-4 | 5e-4 | 5e-4 |\n| weight decay | 1e-4 | 1e-4 | 1e-4 |\n| max tokens | 8192$\\times$4 | 8192$\\times$4 | 16000 |\n\n&ensp;&ensp;&ensp; We will supplement the experimental settings in the next version and make our code publicly available to enhance reproducibility.\n\n[3] Ma et al., 2020. Monotonic multihead attention. In ICLR2020.\n\n&ensp;\n\nQ4: How to input k when training the model?\\\nA4: *k* is a hyperparameter of the wait-*k* model, which is manually set based on latency requirements. During training, it enables each target token to attend to different source information by utilizing a cross-attention mask. During inference, it controls the number of source tokens input to the model to achieve simultaneous machine translation.  \n\n&ensp;\n\nQ5: Provide more examples of tailored references?\\\nA5: By analyzing the tailored references for wait-*1* policy, we observe that our method is largely capable of generating references that meet the requirements. Additionally, we choose some cases and analyze their impact on the SiMT model.\n* Case study of #70 in De->En test set. The tailored reference is generated for wait-*1* policy.\n\n|Sentence|Example|\n| :------------------------| :-------------------------------------------------------------------------------------------------------------------------- |\n| Source Sentence | Die Forscher untersuchten die medizinische Versorgung der Frauen über sieben Jahre . |\n| Ground-Truth | The researchers tracked the women &apos;s medical journeys across seven years . |\n| Tailored Ref | the researchers tracked the women &apos;s medical journeys across seven years . |\n| Translation | the researchers studied the effects of medical care for women over seven years . |\n\n&ensp;&ensp;&ensp; In this example, the tailored reference is identical to ground-truth, yet the SiMT model still maintains high translation quality. This is because the word order of the source sentence closely aligns with that of the ground-truth, resulting in rare forced anticipations under the wait-*1* policy. Consequently, the tailored reference remains the same as the ground-truth.\n* Case study of #401 in De->En test set. The tailored reference is generated for wait-*1* policy.\n\n|Sentence|Example|\n| :-------------------------| :------------------------------------------------------------------------------------------------ |\n| Source Sentence | Ich frage ihn, ob er je wieder eine Stand-up-Comedytour machen wird . |\n| Ground-Truth | I ask whether he will ever do another stand @-@ up tour . |\n| Tailored Ref | I ask him whether he will ever do another stand @-@ up tour . |\n| Translation | I wonder him , whether he will ever be able to get a stand @-@ up tour . |\n\n&ensp;&ensp;&ensp; In comparison to the ground-truth, the tailored reference includes the object 'him', aligning more closely with the conventions of English expression. Consequently, the tailored reference enhances both the fluency and accuracy of translation.\\\n&ensp;&ensp;&ensp; According to your suggestions, we will add the examples in the appendix to deepen the understanding of our method.\n\n&ensp;\n\nIf our answers can solve your questions, we would appreciate it if you could increase the score.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. The motivation is clear and the problem is interesting. The paper is well-written mostly.  \n2. An effective method to automatically generate references for training a low-latency translation model while avoiding forced anticipations.\n3. Extensive experimental results and analysis show that their proposed algorithm improves the translation quality, anticipating rate, and hallucination rate for this simultaneous machine translation problem. ", "reasons_to_reject": "1. The RL fine-tuning part of the training method (section 3.2) misses some details and is hard to follow. See Q1.\n2. It is unclear how would the unsuccessful tailored references impact the translation performance.\n", "questions_to_authors": "1. I am not familiar with reinforcement learning, but what is the action and policy of this tailor model? Also, providing a plot of the reward history curve would be helpful. \n2. Could you explain more on average lagging (x-axis of Figure 3,4,5), non-anticipatory references, and test-time wait-$k$ methods? It would be useful to include more details about them in the paper.\n3. The paper misses some training details (number of epochs, learning rate) in the experimental section. Also, how to input $k$ when training the model? \n4. Providing more examples of tailored references would be more helpful.\n", "paper_topic_and_main_contributions": "This paper introduces a method aimed at addressing the issue of training a low-latency translation model while avoiding ground truth anticipations. It proposes to generate a tailored reference to accommodate the expected latency associated with the simultaneous machine translation (SiMT) problem. It introduces a three-stage method: 1) train a transformer-based SiMT model using ground truth 2) train an additional shallow transformer decoder as the tailor module using ground truth 3) use reinforcement learning method to fine-tune the tailor module. The generated tailored references are used for fine-tuning SiMT. The experiments evaluate the training strategy across three translation tasks and can be worked with different policies (Walk-$k$, and HMT).\n\nContributions: an effective training mechanism for training a translation model with low-latency, NLP engineering experiment", "excitement": 4, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "vu2Gvu29HK", "reviewer": "Reviewer_WzVC", "summary": "", "questions": "- Any explanation / hypothesis as to why the approaches using the tailor perform so much better on English to Vietnamese than other directions, even beating full-sentence translation? This seems like a major anomaly.\n", "limitations": "", "rating": 4, "confidence": 2, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for your valuable and insightful comments.\n\n&nbsp;\n\nQ1: Discussion or details on the additional training costs?\\\nA1: Our method aims to provide tailored references to the SiMT model online, effectively avoiding forced predictions while ensuring high quality. The experimental results in Figure 5 demonstrate that **our method significantly outperforms previous offline methods**, achieving better latency-quality trade-offs.\\\n&ensp;&ensp;&ensp; Regarding the training complexity, our method introduces two additional training processes: pre-training of the tailor and RL fine-tuning. However, it is important to note that **the associated training costs for these processes are relatively modest**. During the pre-training stage, the optimization objective of the model is relatively straightforward, allowing convergence within just five epochs. Furthermore, after achieving a favorable initial state, RL fine-tuning requires only seven epochs of joint training with the SiMT model.\\\n&ensp;&ensp;&ensp; Considering the performance enhancements achieved, **the training costs introduced by our method are deemed acceptable**. We acknowledge your concern about training costs and will supplement more details in the next version.\t\n\n&nbsp;\n\nQ2: Why the approaches using the tailor perform better on English to Vietnamese than other directions, even beating full-sentence translation?\\\nA2: **English and Vietnamese belong to distinct language families, making translation between them challenging**. In comparison to other translation directions, translating from English to Vietnamese is inherently more difficult. Our approach employs the tailor to convert ground-truth into tailored references, aligning better with the expression patterns of the source sentence. As a result, our method yields a more significant performance improvement in the En->Vi direction, surpassing translations within the same language family, such as De->En and En-Ro.\\\n&ensp;&ensp;&ensp; Prior methods like HMT have showcased the SiMT model can surpass full-sentence translation performance under high latency. This indicates that **a precise policy can guide the translation model to focus on the relevant source tokens while excluding irrelevant ones**. Through the utilization of tailored references, our approach trains the model to acquire a more accurate policy, diminishing forced predictions, and ultimately achieving full-sentence performance.\n\n&nbsp;\n\nIf our answers can solve your questions, we would appreciate it if you could increase the score.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "- The authors' approach seems to represent a new state of the art for simultaneous translation.\n- The specific approach of using reinforcement to train a \"tailor\" module to dynamically produce translation targets with semantic ordering more similar to the input is quite novel.\n- The analysis seems sound, particularly in demonstrating that the tailor approach does indeed produce better translation targets than the \"non-anticipatory references\"", "reasons_to_reject": "- This model seems to introduce a lot of training complexity (and training computational costs) for improvements which are modest, especially in the adaptive policy setting. No discussion or details on the additional training costs are provided.", "questions_to_authors": "- Any explanation / hypothesis as to why the approaches using the tailor perform so much better on English to Vietnamese than other directions, even beating full-sentence translation? This seems like a major anomaly.\n", "paper_topic_and_main_contributions": "This paper addresses a key difficulty in training simultaneous translation models: the fact that reference translations may have a significantly different word order from the source, forcing the model to guess when translating with low latency. It introduces a new non-autoregressive decoder to generate a custom paraphrase of the target during training to address this. This new module (the \"tailor\") is learned using reinforcement learning which balances between producing something like the original reference, and something more similar to the \"non-anticipatory reference\" generated by the translation model using the same latency policy. The experiments are convincing and show that this approach leads to state-of-the-art results (especially for the fixed latency policy, wait-K).", "excitement": 4, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "fFBbl2HcUC", "reviewer": "Reviewer_AVCF", "summary": "", "questions": "1. Why is 1.89 highlighted in Table 3?\n2. What is Nt in Figure 2?", "limitations": "", "rating": 2, "confidence": 4, "soundness": 2, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your thorough and valuable evaluation.\n\n&ensp;\n\nQ1: Why use non-autoregressive models to get pseudo reference?\\\nA1: There are two main reasons for employing non-autoregressive models to generate tailored references. \n1. Our method aims to provide tailored references for the SiMT model online, ensuring translation quality while minimizing forced predictions. To generate tailored references that meet these requirements, we leverage reinforcement learning to optimize the tailor.  In order to accommodate the exploration-based nature of reinforcement learning, we choose the non-autoregressive structure, **enabling independent sampling of tokens across all positions**. This design facilitates concurrent token sampling, leading to significant reductions in training costs.\n2. Non-autoregressive translation models have demonstrated **comparable performance to autoregressive models**, showcasing their capability to generate high-quality references.\n\n&ensp;&ensp;&ensp; Hence, we select the non-autoregressive model as the mechanism for generating tailored references. We appreciate your reminder and will emphasize this aspect in the paper.\n\n&ensp;\n\nQ2: Why share the encoder?\\\nA2: Our method involves adapting the ground-truth to a tailored reference, aligning it more closely with the source input of the SiMT model. Consequently, the tailor needs to obtain the representation of the source sentence. By sharing the encoder, the tailor can take into account the input attributes of the SiMT model, resulting in the generation of a tailored reference that is well-suited for the SiMT model. \tYour question is appreciated, and we will provide additional explanations on sharing the encoder.\n\n&ensp;\n\nQ3: Why joint training?\\\nA3: The primary distinction of our method from prior approaches lies in the online generation of tailored references. This dynamic reference generation approach, grounded in source representation, is more suitable as the training objective of the SiMT model. **To match the dynamic reference generation method, we employ the shared encoder and joint training**.\\\n&ensp;&ensp;&ensp; In scenarios where the encoder is shared, if the approach of training the tailor first and then the SiMT model is adopted, it does not yield optimal performance. Because **training the SiMT model separately can induce alterations in the parameters of the encoder, rendering the tailor incompatible with the modification of the encoder**. Consequently, the quality of the generated tailored reference could degrade, which is not suitable for the SiMT model.\\\n&ensp;&ensp;&ensp; We appreciate your insights, and we will incorporate an explanation of joint training for further clarity.\n\n&ensp;\n\nQ4: Not comparing with other methods that generate pseudo references?\\\nA4: We apologize for any misunderstanding that I may have caused. We compare our method with another training method [1] on De->En task in section 5.2. This comparison demonstrates our method outperforms previous training methods by avoiding forced anticipations while maintaining high quality.  In line with your suggestion, we will move the experimental results from this section to the main experiment section for better clarity.\n\n[1] Chen et al., 2021. Improving simultaneous translation by incorporating pseudo-references with fewer reorderings. In EMNLP2021.\n\n&ensp;\n\nQ5: What is p_a, p_s? Why is y in the condition of Equation (5-7)?  What is the meaning of Eq. 7? \"Efficient calculated through CTC\" is not clear?\\\nA5: **p_a** refers to the probability distribution of the output sequence **a** generated by the tailor. The length of **a** corresponds to the input length of the tailor and may comprise repetitive and blank tokens. On the other hand, **p_s** represents the probability distribution of the normal sentence **s**. It is calculated by considering all possible sequences **a** that can be reduced to **s**.\\\n&ensp;&ensp;&ensp; The rationale for employing **y** as a condition in Equation (5-7) stems from the fact that the tailor generates the tailored reference based on the ground-truth. As illustrated in Figure 2, the tailor unsamples ground-truth **y** and generates the sequence **a** by referring to the source sentence. Then the sequence **a** can be reduced to the normal sentence **s**, which serves as the tailored reference. Therefore, incorporating **y** within the condition can be interpreted as utilizing the upsampled ground-truth.\\\n&ensp;&ensp;&ensp; Eq (7) serves as the training objective during the pre-training of the tailor. It enables the tailor to initially learn how to construct ground-truth while considering both the upsampled ground-truth and the source sentence, which is a relatively straightforward task. This establishes a favorable initial state for the SiMT model during the RL fine-tuning stage.\\\n&ensp;&ensp;&ensp; We will replace the CTC with dynamic programming and polish the parts involving Equation (5-7) according to your suggestions.\n\n&ensp;\n\nQ6: Lack of details for replication?\\\nA6: We apologize for any confusion. Due to space constraints, we mention in Section 4.2 that unlisted experimental details are maintained consistent with [2]. The experimental settings are outlined in the table below.\n\n| Hyperparameter | WMT15 De->En | WMT16 En->Ro | IWSLT15 En->Vi |\n| :-----| :----: | :----: | :----: |\n| model | Transformer Base | Transformer Base | Transformer Small |\n| dropout | 0.3 | 0.3 | 0.1 |\n| optimizer | adam | adam | adam |\n| adam-$\\beta$ | (0.9, 0.98) | (0.9, 0.98) | (0.9, 0.98) |\n| lr | 5e-4 | 5e-4 | 5e-4 |\n| weight decay | 1e-4 | 1e-4 | 1e-4 |\n| max tokens | 8192$\\times$4 | 8192$\\times$4 | 16000 |\n\n&ensp;&ensp;&ensp; We will supplement the experimental settings in the next version and make our code publicly accessible to facilitate better reproducibility..\n\n[2] Ma et al., 2020. Monotonic multihead attention. In ICLR2020.\n\n&ensp;\n\nQ7: L67-L70 appears to be circular reasoning?\\\nA7: We are sorry for causing confusion to you. In this context, it means that employing a single reference could cause SiMT models trained at different latency to adopt the identical translation policy.  This may lead to forced predictions in SiMT models trained at low latency or introduce additional delays for SiMT models trained at high latency.  We will follow your suggestions to polish this section in the paper.\n\n&ensp;\n\nQ8: Appendix A highly resembles the content of Section 4 of [Chen+ EMNLP21], which should be omitted?\\\nA8: We appreciate your valuable suggestions. [1] make great contributions to SiMT, introducing concepts like HR and AR along with detailed calculation methods. In accordance with your recommendations, we will make modifications and link the relevant content in our paper to [1].\n\n[1] Chen et al., 2021. Improving simultaneous translation by incorporating pseudo-references with fewer reorderings. In EMNLP2021.\n\n&ensp;\n\nQ9: The tables need to be clear. Which datasets are they based on?\\\nA9:  We are sorry for causing you to misunderstand. Due to space limitations, we mention in L448-L450 that the experiments conducted in section 5 are all based on the WMT15 De->En dataset. We will include the dataset used for the experiment in the caption of each table according to your reminder.\n\n&ensp;\n\nQ10: Why is 1.89 highlighted in Table 3?\\\nA10: The value of 1.89 is highlighted in Table 3 to indicate that, with $\\alpha$ set to 0.2, the SiMT model achieves the best trade-offs between latency and translation quality.\n\n&ensp;\n\nQ11: What is Nt in Figure 2?\\\nA11: I apologize for any confusion that may have arisen. In Figure 2, Nt represents the number of layers in the tailor. To provide a more comprehensive understanding, We will clearly explain this issue in the main text.\n\n&ensp;\n\nIf our answers can solve your questions and eliminate your concerns, we would appreciate it if you could increase the score.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. Generating pseudo references for simultaneous translation is a reasonable direction. \n2. Using RL to train the pseudo-reference generator is somewhat novel, but there needs more details for reproducibility. ", "reasons_to_reject": "1. The motivation for the model design is not clear. Specifically, the authors should explain the following:\n      1. Why use non-autoregressive models to get pseudo reference?\n      2. Why share the encoder?\n      3. Why joint training? It does not seem like the SiMT model can help the pseudo-reference generator. The direct approach should be training the reference generator first, then training the SiMT model.\n2. Not comparing with appropriate correct baseline models. Admittedly, the proposed approach is orthogonal to HMM Transformer and wait-K, which means the improvements can stack. However, the proposed method should be compared with similar methods that generate pseudo references, such as [Chen+ EMNLP21] and [Zhang&Feng EMNLP22].\n3. Equation (5-7) is not clear. What is p_a, p_s? Why is y in this condition? Particularly, Eq. 7 computes p(y|x,y), which does not make sense. L273 \"Efficient calculated through CTC\" is not clear, and it should be \"through dynamic programming\".\n4. There is a lack of details for replication, e.g., model size and training/finetuning schedule. \n5.  L67-L70 appears to be circular reasoning.\n6. Appendix A highly resembles the content of Section 4 of [Chen+ EMNLP21], which should be omitted.\n7. The tables need to be clear. Which datasets are they based on?", "questions_to_authors": "1. Why is 1.89 highlighted in Table 3?\n2. What is Nt in Figure 2?", "paper_topic_and_main_contributions": "This paper works on improving the simultaneous translation task, where the authors take the direction of generating pseudo references. Specifically, the authors proposed a CTC-based pseudo-reference generator. The generator aims to generate pseudo references that are close to the groundtruth and of low anticipation rate. The training of the generator is through RL. The proposed method can be combined with existing approaches, and the authors show that it can improve the wait-k method and the HMM Transformer.", "excitement": 2, "reproducibility": 2, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "CTZPHL6Q92", "reviewer": "Reviewer_Boo4", "summary": "", "questions": "a. The motivation in this paper is not very clear. It is suggested that the author give examples to explain what kind of problems in simultaneous machine translation have been solved, and the proportion and importance of these problems in this area.\nb. It is suggested that the author supplement the pipeline details of the three-step method and demonstrate their necessity through experiments.\nc. As we all know, hallucinations are very serious in simultaneous machine translation. Can some practical examples be provided to illustrate how the method proposed in this paper alleviates this problem?", "limitations": "", "rating": 3, "confidence": 4, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for your valuable and insightful comments.\n\n&ensp;\n\nQ1: Give more examples to explain what kind of problems in SiMT have been solved, and the proportion and importance of these problems in this area?\\\nA1: We apologize for causing confusion. The SiMT model performs translation while reading the source sentence. Previous SiMT methods are trained using standard full-sentence corpora, where reordering between source and target sentences occurs. **This leads to forced predictions during training and results in hallucinations during inference, posing a significant challenge for SiMT models**. As illustrated in Figure 1, the wait-*1* model will be forced to predict \"an enjoyable activity\" before reading in the aligned source content if it is trained with ground-truth. The problem of forced prediction is pervasive in the SiMT and progressively worsens as the latency decreases, as evident in Table 1.\\\n&ensp;&ensp;&ensp; Therefore, our method is designed to dynamically provide tailored references to the SiMT model, enhancing its translation performance under all latency while avoiding forced predictions. The experimental results in Figure 5 demonstrate that our method is capable of significantly reducing hallucinations in both fixed and adaptive policies.\\\n&ensp;&ensp;&ensp; We greatly appreciate your inquiry, and we will underscore the specific challenges faced by SiMT models and the problems our method addresses in the next version.\n\n&ensp;\n\nQ2: Supplement the pipeline details of the three-step method and demonstrate their necessity through experiments?\\\nA2: I apologize for any confusion that may have arisen. Due to space limitations, we mention in Section 4.2 that experimental details not listed are kept consistent with those of [1]. The experimental settings are presented in the following table.\n\n| Hyperparameter | WMT15 De->En | WMT16 En->Ro | IWSLT15 En->Vi |\n| :-----| :----: | :----: | :----: |\n| model | Transformer Base | Transformer Base | Transformer Small |\n| dropout | 0.3 | 0.3 | 0.1 |\n| optimizer | adam | adam | adam |\n| adam-$\\beta$ | (0.9, 0.98) | (0.9, 0.98) | (0.9, 0.98) |\n| lr | 5e-4 | 5e-4 | 5e-4 |\n| weight decay | 1e-4 | 1e-4 | 1e-4 |\n| max tokens | 8192$\\times$4 | 8192$\\times$4 | 16000 |\n\n&ensp;&ensp;&ensp; We will include these experimental settings in the next version and make our code publicly available to enhance reproducibility.\\\n&ensp;&ensp;&ensp; We demonstrate the necessity of pre-training of the tailor in Table 3 and supplement additional experiments to illustrate the necessity of the other two training stages, as illustrated in the table below.\n\n| Method | AL | BLEU | \n| :----------------------------------| :-------------: | :----------: | \n| **Wait-k + Tailor** | **1.89** | **25.90** |\n| $~~$ w/o Base Model | 1.77 | 22.89 |\n| $~~$ w/o Pre-training | 1.80 | 24.66 |\n| $~~$ w/o RL Fine-tuning| 1.86 | 24.60 |\n\n&ensp;&ensp;&ensp; From the above results, it is evident that the three-step training approach achieves the best performance, thereby showcasing the importance of each training stage.\\\n&ensp;&ensp;&ensp; In line with your valuable suggestions, we will incorporate the pipeline details and the aforementioned experimental results into the paper.\n\n[1] Ma et al., 2020. Monotonic multihead attention. In ICLR2020.\n\n&ensp;\n\nQ3: Provide some practical examples to illustrate how the method proposed in this paper alleviates hallucinations?\\\nA3: I apologize for any confusion. We provide statistical results in Figure 6 to demonstrate that our method can effectively alleviate hallucinations in both fixed and adaptive policies under different latency. In order to provide a practical illustration of how our proposed method alleviates hallucinations, we present the case of #366 in De->En test set.\n\n| Sentence | Example | \n| :-----------------------------------------| :---------------------------------------------------------------- | \n| Source Sentence | Auf Shakespeare war ich regelrecht allergisch . |\n| Ground-Truth | I had a real allergy to Shakespeare . |\n| Tailored Reference| on Shakespeare , I was quite allergic . |\n| Translation (Wait-1) | on Shakespeare , I was quite right to say that . |\n| Translation (Wait-1 + Tailor) | on Shakespeare , I was quite allergic . |\n\n&ensp;&ensp;&ensp; The example illustrates that our method generates a tailored reference that adheres closely to the word order of the source sentence. When the SiMT model is trained with the tailored reference, it can avoid forced predictions during training, leading to a reduction in hallucinations during inference.\\\n&ensp;&ensp;&ensp; We appreciate your suggestion and will certainly include this example in the paper to provide a clearer understanding of the practical impact of our proposed method.\n\n&ensp;\n\nQ4: About typos grammar style and presentation Improvements?\\\nA4: Thank you for your meticulous review and for providing valuable suggestions for improvement. We will make the necessary revisions and refinements in the next version to enhance the clarity, grammar, style, and overall presentation of the paper. Your feedback is greatly appreciated.\n\n&ensp;\n\nIf our answers solve your problem and eliminate your concerns, we would appreciate it if you could increase the score.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "The strengths of this paper are that it proposes a tailored reference for SiMT models trained at different latencies, which achieves superior performance. It also investigates the impact of the number of layers and training methods on performance and suggests that further exploration of system settings could yield even better results. The paper also incorporates pseudo-references with fewer reorderings to improve simultaneous machine translation, which can inform future research.", "reasons_to_reject": "The definition of motivation and tailor in this paper are not very clear, and the author does not clearly introduce what kind of problems are to be solved in simultaneous machine translation. From the only examples available, it can be seen that the paper designed a tailored reference and adjusted the word order for joint training. However, whether this would affect the fluency of the translation was not discussed, and how much of the ability to adjust the order was lost. This paper proposes a three-stage training method for training the SiMT model with the tailor, but the details of the pipeline were not very clear, and the ablation experiment did not demonstrate the necessity and effectiveness of these three stages.", "questions_to_authors": "a. The motivation in this paper is not very clear. It is suggested that the author give examples to explain what kind of problems in simultaneous machine translation have been solved, and the proportion and importance of these problems in this area.\nb. It is suggested that the author supplement the pipeline details of the three-step method and demonstrate their necessity through experiments.\nc. As we all know, hallucinations are very serious in simultaneous machine translation. Can some practical examples be provided to illustrate how the method proposed in this paper alleviates this problem?", "paper_topic_and_main_contributions": "This paper proposes to provide a tailored reference for the SiMT models trained at different latency varies. The tailored reference aims to avoid forced anticipations and maintain faithfulness to ground-truth. The paper investigates the impact of the number of layers and training methods on performance and suggests that further exploration of system settings could yield even better results. The main contribution of this paper is that it improves the performance of simultaneous machine translation by incorporating pseudo-references with fewer reorderings.", "excitement": 3, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Typos_Grammar_Style_and_Presentation_Improvements": "different latency -> different latencies\nthe Full-sentence -> Full-sentence\nprovides tailored reference -> provides a tailored reference\ntailored reference -> the tailored reference"}, "extra_text": {}}, {"review_id": "KnpwesXtEv", "reviewer": "Reviewer_aHPk", "summary": "", "questions": "1. How do the authors upsample the ground-truth for the NAT module?\n2. How do the authors estimate the anticipation rate?", "limitations": "", "rating": 4, "confidence": 2, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for your valuable and insightful comments.\n\n&ensp;\n\nQ1: How do the authors upsample the ground-truth for the NAT module?\\\nA1: In our experimental setup, we upsample ground-truth by duplicating each target token twice.\n\n&ensp;\n\nQ2: How do the authors estimate the anticipation rate?\\\nA2: The anticipation rate measures the proportion of forced predictions that arise when a policy and parallel sentence are provided.  In practical calculations, we first obtain alignments from the parallel sentence. For each target token, if there are still relevant source tokens that have not been read in while translating that target token, it is considered a forced prediction[1]. A comprehensive calculation method is elaborated in detail in Appendix A for your reference.\n\n[1] Chen et al., 2021. Improving simultaneous translation by incorporating pseudo-references with fewer reorderings. In EMNLP2021.\n\n&ensp;\n\nIf our answers can solve your questions, we would appreciate it if you could increase the score.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. A novel and effective approach.", "reasons_to_reject": "1. Some descriptions are not clear.", "questions_to_authors": "1. How do the authors upsample the ground-truth for the NAT module?\n2. How do the authors estimate the anticipation rate?", "paper_topic_and_main_contributions": "This paper presents a novel approach to synthesizing high-quality pseudo-references tailored to a given latency by employing an additional Non-Autoregressive Translation (NAT) module. The authors use Reinforcement Learning to jointly optimize the translation model and the NAT module, resulting in clear advantages in the BLEU-AL curves across different datasets and policies. Overall, this work shows promise. However, some descriptions in the paper could be clearer, such as the method for upsampling the ground-truth and estimating the anticipation rate.", "excitement": 4, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "vG5HJeeqDp", "reviewer": "Reviewer_XdDD", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 4, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for your valuable and insightful comments.\n\n&ensp;\n\nQ1: Concern that our method is overly complicated?\\\nA1: The biggest difference between our method and previous methods is the online generation of tailored references. This online generation method ensures that the tailored references not only consider latency requirements to avoid forced predictions, but also maintain the high quality of the reference. The experiments in Figure 5 show that **our online method is remarkably more effective than previous offline methods**.\\\n&ensp;&ensp;&ensp; Regarding the training complexity, our method introduces two additional training processes: pre-training of the tailor and RL fine-tuning. However, these two training processes require only five epochs and seven epochs respectively to converge. Therefore, **the associated training costs for the two processes are relatively modest**.\\\n&ensp;&ensp;&ensp; Considering the substantial improvement in performance, the moderate increase in complexity introduced by our method is acceptable.\n\n&ensp;\n\nQ2: Append the results of some additional baselines?\\\nA2: Thank you for suggesting additional baselines for our evaluation. We greatly appreciate your insights.\\\n&ensp;&ensp;&ensp; Regarding directly training the SiMT model using non-anticipatory reference, it has been explored by [1]. We implement this method on the WMT15 De->En dataset and the results are in Figure 5. We also implement the second baseline you propose. The corresponding results are included in the following table for comparison.\n\n| Method | AL | BLEU | \n| :-----| :----: | :----: | \n| **Tailored Reference** | **1.89** | **25.90** |\n| Non-anticipatory Reference | 2.55 | 22.73 |\n| Mixed Loss | 3.39 | 24.23 |\n\n&ensp;&ensp;&ensp; It can be seen that our method significantly outperforms other baselines and achieves better latency-quality trade-offs. We will include the above results in the paper to demonstrate the effectiveness of our proposed method.\n\n[1] Chen et al., 2021. Improving simultaneous translation by incorporating pseudo-references with fewer reorderings. In EMNLP2021.\n\n&ensp;\n\nIf our answers can solve your questions, we would appreciate it if you could increase the score.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "The positive results demonstrate that adapting the ground-truth translation with a third model can indeed create a better reference for training SIMT models.", "reasons_to_reject": "I'm concerned that the proposed method is overly complicated. Since we are creating non-anticipatory references by applying the wait-K policy to the full-sentence model. Looking at Figure 2, I think there are some additional baselines we can construct easily that do not require training a \"tailor\" model:\n\n\n1/ Directly training the SIMT model using non-anticipatory reference\n\n2/ Training the SIMT model with a mixed loss \n\n     alpha * L_ground_truth + (1 - alpha) * L_non_anticipatory\n\nBoth are related to mode distillation.", "questions_to_authors": "", "paper_topic_and_main_contributions": "This paper addresses the problem of finding optimal training references for simultaneous machine translation (SiMT) models under varying latency constraints. The key insight is that using the full ground truth translation as the training target is suboptimal, as it forces premature anticipations at low latency and degrades quality at high latency.\n\n\n\nThe paper proposes to dynamically generate tailored training references by modifying the ground truth. This is achieved via a trainable \"tailor\" model, which is trained based on NAT pretraining and RL training.", "excitement": 4, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "sNjOrUZO0d", "reviewer": "Reviewer_DPEY", "summary": "", "questions": "1. In Figures 3 and 4 for the translation direction of En->Vi, the proposed method even outperforms the full-sentence translation that I consider as the upper bound. Do you have any explanation for this? I think one reason might be the first point in reasons to reject. You use the tailor to implicitly generate some synthetic data.", "limitations": "", "rating": 4, "confidence": 4, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for your valuable and insightful comments.\n\n&ensp;\n\nQ1: Append the results of the additional baseline?\\\nA1: Thank you for your valuable suggestion. We take your feedback into consideration and include an additional baseline. It involves optimizing the SiMT model with ground-truth and non-anticipatory reference concurrently. The comparison of our method with other baselines is shown in the table below.\n\n| Reference | AL | BLEU | \n| :-----| :----: | :----: | \n| **Tailored Reference** | **1.89** | **25.90** |\n| Non-anticipatory Reference | 2.55 | 22.73 |\n| Ground-Trurh, Non-anticipatory Reference | 3.39 | 24.23 |\n\n&ensp;&ensp;&ensp; As evident from the table, our method exhibits better latency-quality trade-offs compared to the other baselines, demonstrating the effectiveness of our method. We sincerely appreciate your suggestion and will ensure that the newly added baseline results are included in our paper.\n\n&ensp;\n\nQ2: Lack of some training details?\\\nA2: We deeply appreciate your attention to detail and your valuable feedback. Due to space limitations, we mention in Section 4.2 that experimental details not listed are kept consistent with those of [1]. The experimental settings are presented in the following table:\n\n| Hyperparameter | WMT15 De->En | WMT16 En->Ro | IWSLT15 En->Vi |\n| :-----| :----: | :----: | :----: |\n| model | Transformer Base | Transformer Base | Transformer Small |\n| dropout | 0.3 | 0.3 | 0.1 |\n| optimizer | adam | adam | adam |\n| adam-$\\beta$ | (0.9, 0.98) | (0.9, 0.98) | (0.9, 0.98) |\n| lr | 5e-4 | 5e-4 | 5e-4 |\n| weight decay | 1e-4 | 1e-4 | 1e-4 |\n| max tokens | 8192$\\times$4 | 8192$\\times$4 | 16000 |\n\n&ensp;&ensp;&ensp; We acknowledge the significance of these details for replicability and will ensure to provide a more comprehensive account of our experimental settings in the next version. Additionally, to enhance the reproducibility of our work, we will make our code publicly available. \n\n[1] Ma et al., 2020. Monotonic multihead attention. In ICLR2020.\n\n&ensp;\n\nQ3: Do you have any explanation for the phenomenon that the proposed method even outperforms the full-sentence translation in the direction of En->Vi?\\\nA3: Previous approaches such as HMT have demonstrated that the SiMT model can outperform the performance of the full-sentence translation under high latency. I think there are mainly two reasons why our method outperforms the full-sentence translation. The first interpretation is indeed what you mentioned: \"**the tailor implicitly generates some synthetic data**,\" which can help the model reduce hallucinations and generate more reliable translations. Another interpretation is that our method empowers the adaptive policy to acquire more accurate policies. This enables the translation model to **disregard irrelevant source tokens and concentrate on the necessary source tokens**, thereby obtaining high-quality translation.\n\n&ensp;\n\nIf our answers can solve your questions, we would appreciate it if you could increase the score.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. The motivation is clear, and the resolved issue is also practical for SiMT;\n2. Extensive experiments are conducted to support the main claims, and the results are convincing. \n3. Thorough analysis and ablation study.", "reasons_to_reject": "1. Missing baseline: the proposed method is very related to the target-side augmentation method. It's better to add some related baselines, like forward-translation, i.e. applying a trained SiMT model to translate the source sentence and combining these synthetic data with the original data together to further trains the SiMT model. \n2. The wring should be improved, especially the sections about the proposed method.\n3. Experimental details are missing, like batch size, learning rater and so on, which makes it difficult to reproduce. ", "questions_to_authors": "1. In Figures 3 and 4 for the translation direction of En->Vi, the proposed method even outperforms the full-sentence translation that I consider as the upper bound. Do you have any explanation for this? I think one reason might be the first point in reasons to reject. You use the tailor to implicitly generate some synthetic data.", "paper_topic_and_main_contributions": "This paper is about avoiding forced anticipations during training a simultaneous machine translation (SiMT) model. Based on the conventional training of SiMT, this paper proposes to add a decoder (tailor) module which is trained on a combined target of the non-anticipatory reference and the ground truth. In this way, the tailor module could act as a regularizer to avoid forced anticipations.\n\nThe main contributions of the paper are:\n1. The motivation is clear, and the resolved issue is also practical for SiMT;\n2. Extensive experiments are conducted to support the main claims, and the results are convincing. \n3. Thorough analysis and ablation study.", "excitement": 4, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper addresses simultaneous machine translation (SiMT), which performs translation based on partial input and is useful for real-time translation. A trained SiMT is prone to suffer from the anticipation problem, i.e., the training loss encourages the SiMT to hallucinate. This paper addresses the problem by training from a wait-k non-autoregressive model, which tends to generate less anticipating samples (better aligned with the source order).

Overall, the idea is analogous to using KD to address the multimodality issue of NAT -- I am not saying the paper is the same as KD-NAT, but they share the same spirit. 

Reviewers generally find the idea makes sense and intuitively it may mitigate the anticipation problem of SiMT.

However, the paper has a few issues. 
1. The paper lacks clarity and some formulations are sloppy. For example, p(y|x,y) is not sound.
2. Some design choices are not clearly explained, such as why choosing CTC-NAT as the tailored reference. In the author response, the author suggest it's for efficiency concerns, but training efficiency is not analyzed in the paper.
3. Experimental details are unclear, especially the RL part, which may hinder the replication of this paper.
Decision: Accept-Findings