Hidden Markov Transformer for Simultaneous Machine Translation
Authors: Shaolei Zhang
Yang Feng
Venue: ICLR 2023 notable top 25% (2023)
Keywords: Simultaneous machine translation
Machine translation
Natural language processing
Transformer

Abstract: Simultaneous machine translation (SiMT) outputs the target sequence while receiving the source sequence, and hence learning when to start translating each target token is the core challenge for SiMT task. However, it is non-trivial to learn the optimal moment among many possible moments of starting translating, as the moments of starting translating always hide inside the model and can only be supervised with the observed target sequence. In this paper, we propose a Hidden Markov Transformer (HMT), which treats the moments of starting translating as hidden events and the target sequence as the corresponding observed events, thereby organizing them as a hidden Markov model. HMT explicitly models multiple moments of starting translating as the candidate hidden events, and then selects one to generate the target token. During training, by maximizing the marginal likelihood of the target sequence over multiple moments of starting translating, HMT learns to start translating at the moments that target tokens can be generated more accurately. Experiments on multiple SiMT benchmarks show that HMT outperforms strong baselines and achieves state-of-the-art performance.

Published as a conference paper at ICLR 2023
HIDDEN MARKOV TRANSFORMER FOR SIMULTANEOUS MACHINE TRANSLATION
Shaolei Zhang Yang Feng 1.2+
Key Laboratory of Intelligent Information Processing
Institute of Computing Technology, Chinese Academy of Sciences (ICT/CAS)
University of Chinese Academy of Sciences, Beijing, China
2hangshaolei20z fengyang @ict. wwl.taacc...... an.cn
ABSTRACT
Simultaneous machine translation (SiMT) outputs the target sequence while re- ceiving the source sequence, and hence learning when to start translating each target token is the core challenge for SiMT task. However, it is non-trivial to learn the optimal moment among many possible moments of starting translating, the moments of starting translating always hide inside the model and can only be supervised with the observed target sequence. In this paper, we propose a Hidden Markov Transformer (HMT), which treats the moments of starting trans- lating as hidden events and the target sequence as the corresponding observed events, thereby organizing them as hidden Markov model. HMT explicitly mod- els multiple moments of starting translating as the candidate hidden events, and then selects one to generate the target token. During training, by maximizing the marginal likelihood of the target sequence over multiple moments of starting trans- lating, HMT learns to start translating at the moments that target tokens can be generated more accurately. Experiments on multiple SiMT benchmarks show that HMT outperforms strong baselines and achieves state-of-the- performaned]
INTRODUCTION
Recently, with the increase of real-time scenarios such as live broadcasting, video subtitles and con- ferences, simultaneous machine translation (SiMT) attracts more attention (Cho & Esipova 2016 Gu et al. 2017 Ma et al. 2019 Arivazhagan et al. 2019), which requires the model to receive source token one by one and simultaneously generates the target tokens. For the purpose of high-quality translation under low latency, SiMT model needs to learn when to start translating each target token (Gu et al. 2017), thereby making a wise decision between waiting for the next source token (i.e.,
READ action) and generating target token (i.e., WRITE action) during the translation process.
However, learning when to start translating target tokens is not trivial for SiMT model, as the mo- ments of starting translating always hide inside the model and we can only supervise the SiMT model with the observed target sequence (Zhang & Feng 2022a Existing SiMT methods are divided into fixed and adaptive in deciding when to start translating. Fixed methods directly decide when to start translating according to pre-defined rules instcad learning them (Dalvi et al., 2018) Ma et al. 2019 Elbayad et al. (020)). Such methods ignore the context and thus sometimes force the model to start translating even if the source contents are insufficient (Zheng et al., 2020a). Adaptive methods dynamically decide READ/WRITE actions, such as predicting a variable indicate READ/WRITE action (Arivazhagan et al., 2019: Ma et al. 2020 Miao et al.) (021). However, due to the lack of clear correspondence between READ/WRI WITE actions and the observed target sequence (Zhang & Feng [022c), it is difficult to learn precise READ/WRITE actions only with the supervision of the observed target sequence (Alinejad al. 2021 Zhang & Feng 2022a Indurthi et al.. 2022).
To seek the optimal moment of starting translating each target token that hides inside the model, an ideal solution is to clearly correspond the moments of starting translating to the observed target
Corresponding author: Yang Feng. Code is available at at/httss: http:://gggtuub.cco///c/////////

---

Published as a conference paper at ICLR 2023
Selected State
READ.
Unselected State
hg
Encoder
Hidden Markov Decoder
4 XLAAA
1B500 Jooty
Reel
very
happy
Figure 1: Illustration of hidden Markov Transformer. HMT explicitly produces K = 3 states for each target token represent starting translating the target token when receiving different numbers of source tokens respectively (where hgn means starting translating when receiving the first n source tokens). Then, HMT judges whether select each state from low latency to high latency (i.e., from left to right). Once a state is selected, HMT will translate the target token based on the selected state.
sequence, and further learn to start translating at those moments that target tokens can be gener- ated more accurately. To this end, we propose Hidden Markov Transformer (HMT) for SiMT, which treats the moments of starting translating as hidden events and treats the translation results as the cor- responding observed events, thereby organizing them in the form of hidden Markov model (Baum & Petrie 1966 RRabiner& & Juang 1986, Wang et al. 2018). As illustrated in Figure ! HMT explicitly produces set of states for each target token, where multiple states represent starting translating the target token at different moments respectively (i.e., start translating after receiving different numbers of source tokens). Then, HMT judges whether select each state from low latency to high latency. Once a state is selected, HMT will generate the target token based on the selected state. For example, HMT produces 3 states when generating the first target token represent starting translating after receiving the first 1, 2 and 3 source tokens respectively (i.e., hgg, hg2 and hça)) Then during the judgment, the first state is not selected, the second state is selected output 'I' and then the third state is not considered anymore, thus HMT starts translating 'I' when receiving the first 2 source tokens. During training, HMT is optimized by maximizing the marginal likelihood of the target sequence (i.e., observed events) over all possible selection results (i.e., hidden events). In this way, those states (moments) which generate the target token more accurately will be selected more likely, thereby HMT effectively learns when to start translating under the supervision of the observed tar- get sequence. Experiments on geglish->Veetamenn and German -nggishh SiMT benchmarks show that HMT outperforms strong baselines under all latency and achieves state-of-the-art performance.
2 RELATED WORK
Learning when to start translating is the key to SiMT. Recent SiMT methods fall into fixed and adaptive. For fixed method, Ma et al. (2019) proposed a wait-k policy, which first READs k source tokens and then READs/WRITE' one token alternately. [Elaaadd et al. (2220) proposed an efficient training for wait-k policy, which randomly samples different k between batches. Zhang & Feng (2021a) proposed a char-level wait-k policy. [hheng et al. (2020a) proposed adaptive wait-k, which integrates multiple wait-k models heuristically during inference. Guo al. (2022) proposed post- evaluation for the wait-k policy. [Zhang et al. (2022) proposed wait-info policy to improve wait-k policy via quantifying the token information. Zhang Feng (2021c) proposed a MoE wait-k to learn multiple wait-k policies via multiple experts. For adaptive method, Gu et al. (2017) trained an READ/WRITE agent via reinforcement learning. Zheng al. (2019b) trained the agent with golden actions generated by rules. Zhang & Fengl (2022c) proposed GMA to decide when to start translating according to the predicted alignments. /Arivazhagan al. (2019) proposed MILk, which uses a variable to indicate READ/WRITE and jointly trains the variable with monotonic attention (Raffel et al. 2017). Ma et al. (2020) proposed MMA to implement MILk on Transformer. [iu proposed CAAT for SiMT to adopt RNN-T to the Transformer architecture. [Miao ct al. 2022 proposed GSiMT to generate READ/WRITE actions, which implicitly considers all combinations during training and takes one combination in inference. Zhhng & Feng (202dd) proposed an cinnuttiontrasspor--baa based policy for SiMT.
et al. (2021 1
READ/WRTT
Compared with the previous methods, HMT explicitly models multiple possible moments of starting translating in both training and inference, and integrates two key issues in SiMT, 'learning when to start translating" and 'learning translation', into a unified framework via the hidden Markov model.
2

---

Published as a conference paper at ICLR 2023
Outputs
Selection
Confidence
6.6
0.4
0.3
FFN
Translation
h
cross attention
self-attention between states
State Production
Target Inputs
Figure 2: The architecture of hidden Markov decoder.
HIDDEN MARKOV TRANSFORMER
To learn the optimal moments of starting translating that hide inside the SiMT model, we propose hidden Markov Transformer (HMT) to organize the 'moments starting translating' and 'observed target sequence" in the form of hidden Markov model. By maximizing the marginal likelihood of the observed target sequence over multiple possible moments of starting translating. HMT learns when to start translating. We will introduce the architecture, training and inference of HMT following.
3.1 ARCHITECTURE
Hidden Markov Transformer (HMT) consists of an encoder and a hidden Markov decoder. Denoting the source sequence tional encoder (Arivazhagan et al. 2019 Ma et al. 2020) Miao et al. (02T)) is used to map x to the source hidden states h = (h1,... hy)) Hidden Markov decoder explicitly produces set of states for Yi corresponding to starting translating at multiple moments, and then judges which state selected output the target token. Specifically, as shown in Figure 2 hidden Markov decoder involves three parts: state production, translation and selection.
(I1, Iz) and the target sequence as y (3/1.-.. ....:"...000" ..r). unidirec-
State Production When translating ys. HMT first produces set of K states s; ( 8i, to represent starting translating ys when receiving different numbers of source tokens respectively. Then, we set the translating moments t, [t,,1, tt 3 for these states, where state si is required to start translating when receiving the first fi source tokens X5t.
To set the suitable for states, we aim pre-prune those un- favorable translating moments, such as translating V1 after re- ceiving v.z or translating yr after receiving æ1 (Zhang & Feng 20226 a). Therefore, as shown in Figure 3 we introduce wait- path as the lower boundary and wait- K -) path as the upper boundary accordingly (Zheng et al. 2019a), and then consider those suitable moments of starting translating within this interval, where L and K hyperparameters. Formally, the translating moment f of the state 8i,k is defined as:
Lower bounday paind gath
Upeer boundary wati-t-- path
READ
state
Figure 3: Setting of translating
moment t (e.g., 1- =). (I)
4. = min L + (i - 1) + (k - |x|3
Translation The representations of K states for each target token initialized via upsampling the target inputs K times, and then calculated through N Transformer decoder layers, each of which consists of three sub-layers: self-attention between states, cross-attention and feed-forward network.
For self-attention between states, state 8,,8 can pay attention to the state "5,K' of all previous target tokens (i.e., j S i), while ensuring that ",,K' starts translating no later than 8i,k (i.e., 13,4' S (1,,k) to avoid future source information leaking 83,6' to 8i,k-
The self-attention from 8i,k 83K is:
SelfAtt (SL.. S5..))
softmax
if S i and ty S tr
otherwise
3

---

HMM Loss To learn when to start translating, we train HMT by maximizing the marginal likeli- hood of the target sequence (i.e., observed events) over all possible selection results (i.e., hidden events), thereby HMT will give higher confidence to selecting those states that can generate the tar- Published as a conference paper at ICLR 2023
where W and w are learnable parameters. Owing to the self-attention between states, HMT can capture more comprehensive state representations by considering different moments of starting translating. For cross-attention since state si,k starts translating when receiving the first & source tokens, 8i,k can only focus on the source hidden state h2 with j S fi,k, calculated as:
(..EWR( (N.W*))
softmax
S t,,k
CrossAtt (SLA, h2)
va
otherwise
Through N decoder layers, we get the final representation of state Si,k- Accordingly, the translation probability of yi from the state Si,k is calculated based on its final representation:
p(y: Xst.a y v+)) softmax (st,WW),
where Wo are learnable parameters and yii are the previous target tokens.
Selection After getting the final representations of states, in order to judge whether to select state Si.k to generate HMT predicts confidence Ci of selecting state Sik- The confidence Ci.k is predicted based on the final state representation of Si,A and the received source contents:
Ci = sigmoid ([ high,. 81,6 W+),
(5)
where h t. ΣØ h is the average pooling result on the hidden states of the received source tokens, [:] is concatenating operation and W are learnable parameters. In inference, HMT judges whether to select the state from 81, 8, If c 0.5 (i.e., achieving enough confidence), HMT selects the state $j,k to generate y:- Otherwise HMT moves to the next state $1,A+11 and repeats the judgment. Note that we Ci,K ensure that HMT starts translating before the last state 8i,K.
3.2 TRAINING
Since when to start translating is hidden inside the model while the target sequence is observable, HMT treats when start translating target tokens (i.e., which states are selected) hidden events and treats target tokens as the observed events. Further, HMT organizes them in the form of hidden Markov model, thereby associating the moment of starting translating with the observed target token. Formally, for hidden events, we denoted which states are selected as (21, z1). where E [1, K] represents selecting state si,z, to generate y:- Then, following the HMM form, we introduce the transition probability between selections and the emission probability from the selection result.
Transition probability expresses the probability of the selection 2, conditioned on the previous se- lection 1i denoted as p(z; z;-1). Since HMT judges whether to select states from 81,1 8;,K (i.e., from low latency to high latency), 81,4+1 can selected only if the previous state 81,6 is not selected. Therefore, the calculation of p(2; z1-1) consists of two part selected, and (2) those states whose translating moment between 44-1, and ti, (i.c., those states that were judged before $t,,,) are not confident to be selected, calculated a
(1) st,,, confident be
II
(1-ca,1
if if 2 t4-1,8111
p(2 21-1))
4--1. SStAStii
(6)
if f1,2 < (4-1,8-11
Emission probability expresses the probability of the observed target token y; from the selected state $t,,,, i.e., the translation probability in Eq. For clarity, we rewrite as p(y, x 2:) to emphasize the probability of HMT generating the target token yi under the selection zi.
get token more accurately. Given the transition probabilities and emission probabilities, the marginal likelihood of observed target sequence y over all possible selection results z is calculated as:
[x] P[( - p(z; 1
ply x) Σ ply x2 x p(z)
Please refer to Appendix A for detailed instructions the transition probability between selections. We add selection 20 with p(20) before 21 to indicate that no source tokens are received at the beginning of translation, i.c., fo. =0. Therefore, p(=1 z0) the initial prohability of the selection in HMT.
4

---

Published as a conference paper at ICLR 2023
Algorithm Inference Policy of Hidden Markov Transformer
Input: Source sequence x; Translating moments t; Number of states K.
Output: Translated sequence y.
Init: Received source sequence x= []. source index j =0; Translated sequence y [(bos]]. target index i= 1.
I: while Uc-17 * (oos)
2: if [X] 4,,1 then
p To reach the low boundary
3: Wait for source tokens until X
f,,15 j 4 f4.15
4: for & <- 1 to K do
5: if [x]] fi,t then continue;
Skip 85,8 if its translating moment fi is less than [X]
6:
Calculate representation of state s and its confidence Gi according to Eq. SS
7:
if 2 0.5 then
Select WRITE
Translate target token S according Eq.
9:
y + ++ + Ú. i +
10:
Break;
11:
else
Unselect = READ
12:
Wait for the
token 1i+1i
+ & + 1541
j + j
14:
return y:
We employ dynamic programming to reduce the computational complexity of marginalizing all possible selection results, and detailed calculations refer to Appendix B Then, HMT is optimized with the negative log- likeliid loss Lhmm
Chmm log ply x).
Latency Loss Besides, we also introduce a latency loss Liatency to trade off between translation quality and latency. Llateney is also calculated by marginalizing all possible selection results z:
Llatency Σx(z) x C(z) =II p(z 22-1) C(z),
where C(x) is function to measure the latency of a given selection results 2, calculated by the average lagging (Ma et al. 2019) relative to the lower boundary: C(z)
(fr... - +.11)
State Loss We additionally introduce a state loss Cetate to encourage HMT to generate the correct target token yi matter which state is selected (i.e., matter when start translating), thereby improving the robustness on the selection. Since К states are fed into the hidden Markov decoder parallel during training, Latate is calculated through cross-entropy loss all states:
1 lyl
Estate
Σ
log p(y; Xgtra 21)
(10)
K
2,=1
Finally, the total loss L of HMT is calculated as:
L Lrmm + Atatency + Aatatef
(11)
where we set Atateney = 1 and Astate = 1 in our experiments.
3.3 INFERENCE
In inference, HMT judges whether select each state from low latency high latency based on the confidence, and once state is selected, HMT generates the target token based on the state representation. Specifically, denoting the current received source tokens x, the inference policy of HMT is shown in Algorithm [ When translating ÿt. HMT judges whether to select the state from 8,,1 to 8,K, , so HMT will wait at least 44,1 source tokens reach the lower boundary (line 2). During judging, if the confidence CL, 0.5, HMT selects state 8i,k to generate ÿ; (i.e., WRITE action) according to Eq. 4.. otherwise HMT waits for the next source token (i.e., READ action) and moves to the next state 51 We ensure that HMT starts translating ÿ; before the last state 8; (i.e., before reaching the upper boundary) via setting C, 1. Note that due to the monotonicity of the moments to start translating in SiMT, state 81 will be skipped and cannot be selected (line 5) if its translating moment ti is less than the number of received source tokens X (i.c., the moment of translating 9i-1). which is in line with the transition between selections in Eq. 6 during training.
5

---

Published as a conference paper at ICLR 2023
4 EXPERIMENTS
4.1 DATASETS
We conduct experiments the following datasets, which are the widely used SiMT benchmarks.
IWSLT1E| English- Vietnamese (En (133K pairs) We use TED tst2012 (1553 pairs) as the validation set and TED tst2013 (1268 pairs) the test set. Following the previous setting (Ma al. 2020 Zhang & Feng 2021c), we replace the token whose frequency is less than 5 by (urak), and the vocabulary sizes English and Vietnamese are 17K and 7.7K. respectively.
WMT1 German -> English (De->En) (4.5M pairs) We use newstest2013 (3000 pairs) as the validation set and newstest2015 (2169 pairs) as the test set. BPE (Sennrich et al. (016) is applied with 32K merge operations and the vocabulary of German and English is shared.
4.2 EXPERIMENTAL. SETTINGS
We conduct experiments on several strong SiMT methods, described as follows.
Full-sentence MT (Vaswani et al. 2017) Transformer model waits for the complete source sequence and then starts translating, and we also apply uni-directional encoder for comparison. Wait-k (Ma et al., 2019) Wait-k policy first waits for source tokens, and then alternately translates one target token and waits for one source token.
Multipath Wait-k (Elbayad et al. 2020) Multipath Wait-k trains a wait-k model via randomly sampling different & between batches, and uses fixed & during inference.
Adaptive Wait-k (Zheng et al. 2020a) Adaptive Wait-k trains a set of wait-k models (e.g.. from wait-1 1 to wait-13) and heuristically composites these models during inference.
MoE Wait- (Zhang & Feng 2021c) Mixture-of-experts (MoE) Wait-k applies multiple experts to learn multiple wait-k policies respectively, and also uses fixed i during inference.
MMA (Ma et al.] 2020) Monotonic multi-head attention (MMA) predicts variable to indicate READ/WRITE action. and trains this variable through monotonic attention (Raffel et al., 2017).
GMA (Zhang & Feng 2022c) Gaussian multi-head attention (GMA) introduces a Gaussian prior learn the alignments in attention, and decides when to start translating based on the aligned positions. GSiMT (Miao et al., 202T) Generative SiMT (GSiMT) generates a variable to indicate READ/WRITE action. GSIMT considers all combinations of READ/WRITE actions during training and only takes one combination READ/WRITE actions inference.
HMT The proposed hidden Markov Transformer, described in Sec
Settings All systems are based on Transformer (Vaswani et al., 2017) from Fairseq Library Ott et al. 2019). Following Ma al. 2020) and Miao et al. (2021). we apply Transformer-Siee Small (4 heads) for En- Transformer- Base (8 heads) and Transformer-Bit (16 heads) for De-FEn. Due to the high training complexity, we only report GSIMT EE with Transformer-Baan
the same
as its original setting (Miao et al. 2021). The hyperparameter settings are reported in Appendix D
Evaluation For SiMT performance, we report BLEU score (Papineni et al., 2002) for translation quality and Average Lagging (AL) (Ma et al. 2019) for latency. AL measures the token number that outputs lag behind the inputs. For comparison, we adjust L and K in HMT get the translation quality under different latency, and the specific setting of L and K are reported in Appendix
4.3 MAIN RESULTS
We compare HMT with the existing SiMT methods in Figure 4 where HMT outperforms the pre- vious methods under all latency. Compared with fixed methods Wait-k, Multipath Wait-k and MoF: Wait-k, HMT has obvious advantages as HMT dynamically judges when to start translating and thereby can handle complex inputs (Arivazhagan et al. 2019). Compared with adaptive methods,
hlp. hbb.sttafford.eeh//rrreeett////a
statmt
Itanouus
trans translation
gi github.com/ictnlppp

---

Published as a conference paper at ICLR 2023
92
28
90
BLEU Score
26
F.81 sncho MY
1001
24
26
Mal Werk
10MA
24
Met
Adppie Wark
Aleeio Wark
Adgine
MaRpur Wabe
Staipea Wabe
Wark
29
Avenge Lagging (AL,, tokson)
Averago Lagging (AL, tokmn)
Avarage Lagging (AL, token)
(a) En-VVi, Transformer-
(b) De Transformer- -Bae
(c) De De->En, Transformer-D Big
Figure 4: Translation quality (BLEU) against latency (AL) of HMT and previous SiMT methods.
HMT outperforms the current state-of-the-art MMA and GSiMT, owing to two main advantages. First, HMT models the moments of starting translating, which has stronger corresponding corre- lations with the observed target sequence compared with READ/WRITE action (Zhang & Feng. 2022c). Second, both MMA and GSiMT can only consider one combination of READ/WRITE ac- tions in inference (Ma et al. 2020, Miao et al. 2021), while HMT can consider multiple moments of starting translating in both training and inference
HMT explicitly produces multiple states for different translating moments. Furthermore, the performance of MMA and GSiMT will drop some latency (Ma et al. 2020), which is mainly because considering too many combinations of READ/WRITE actions in training may cause mutual interference (Elbayad et al. 2020 Zhang & Feng 2021c| Wu et al., (222). HMT pre-prunes those unfavorable moments of starting translating
via the proposed boundaries, thereby achieving more stable and better performance under all latency.
4.4 TRAINING AND INFERENCE SPEEDS
We compare the training and inference speeds of HMT with previous methods on De- with Transformer- Base,, and the results are reported in Table [ All speeds are evaluated on NVIDIA 3090 GPU.
Table 1: Training and inference speeds of HMT.
Training (s/batch)
Inference (s/token)
Systems
#Para.
Full-sentence MT
60.9M
0.204
0.0097
Wait-k
60.9M
0.205
0.0108
Training Speed Compared with the fixed method Wait-k, the training speed of HMT is slightly slower as it upsamples the target sequence by K times. Given the obvious performance improvements. we argue that the slightly slower training speed than the fixed method is completely acceptable.
MMA
62.5M
2.112
0.0647
GSiMT
60.9M
5.090
0.0247
1==2K==
60.9M
0.531
0.0204
HMT
L=
60.9M
0.730
0.0162
(=9.K=0
60.9M
1.051
0.0142
Compared with adaptive methods MMA and GSiMT that compute the representation of the variable to indicate READ/WRITE circularly (Ma et af.) 2020 Miao et al. 2021). the training speed of HMT has obvious advantages owing to computing representations of multiple states in parallel. Besides, compared to GSiMT considering all possible READ/WRITE combinations, the proposed lower and upper boundaries of the translating moments also effectively speed up HMT training.
Inference Speed Compared with MMA which adds more extra parameters to predict READ/WRITE action in each attention head (a et al. 2020). HMT only requires few extra parameters (W' in (5) and thereby achieves faster inference speed. Compared with GSiMT, which needs to cal- culate the target representation and generate READ/WRITE each at each step (Miao et al. 2021 HMT pre-prunes those unfavorable translating moments and only judges among the rest valuable moments, thereby improving the inference speed. Note that as the lower boundary I. increases, HMT can prune more candidate translating moments and thus make the inference much faster.
ANALYSIS
We conducted extensive analyses to study the specific improvements HMT. Unless otherwise specified, all the results reported De- test set with Baansoormr--Bse.
7

---

Published as a conference paper at ICLR 2023
Table 2: Ablation study of HMM
Table Ablation study of the weight Matenay of la- tency loss Liateney-
Table 4: Ablation study of the weight Astate of state loss Statte
loss, including marginalizing all pos- sible selections or only optimizing the most probable selection result.
Untency
AL
BLEU
Astate
AL
BLEU
AL
BLEU
0.0
6.66
30.07
0.0
3.24
27.74
Marginalizing
0.5
4.12
29.80
0.5
3.47
29.10
3.46
29.29
possible selections
1.0
3,46
29.29
1.0
3,46
29.29
Optimizing most probable selection
1.5
2.10
26.65
1.5
3,46
29.16
1.77
24.02
2.0
2.00
25.94
2.0
3.58
29.21
5.1 ABLATION STUDY
We conduct multiple ablation studies, where L=3 and K 6 are applicd all ablation studies.
HMM Loss HMT Iearns when to start translating by marginalizing all possible selection results dur- ing training. To verify its effectiveness, we compare the performance of marginalizing all possible selections and only optimizing the most probable selection in Table where the latter is realized by replacing Σx in Eq.(7) with Max.- Under the same setting of L = 3 and K 6, only optimizing the most probable selection makes the model fall into local optimum (Miao et al.) (021) of always selecting the first state, resulting in latency and translation quality close to wait-3 policy. Marginal- izing all possible selections effectively enables HMT to learn when to start translating from multiple possible moments, achieving a better trade-off between translation quality and latency.
Weight Lateney Loss Table reports the performance of HMT on various Atateney- The setting of Atateney affects the trade-off between latency and translation quality. Too large (i.e., 2.0) or too small (i.e., 0.0) Atateney sometimes makes the model start translating when reaching the upper or lower boundary, while moderate Atatenay achieves the best trade-off. Note that HMT is not sensitive to the setting of Atatency. either Atatency 0.5 0.5 or Atatency = 1.0 can achieve the best trade-off, where the result with Ataency =05 5 is almost on the HMT line in Figure 4(b)
Weight of State Loss Table demonstrates the effectiveness of the introduced state loss Lestate- Cstate encourages each state to generate the correct target token, which can bring about 1.5 BLEU improvements. addition, the translation quality of HMT not sensitive to the weight Astate of state loss, and various Astate can bring similar improvements.
5.2 How MANY STATES ARE BETTER?
HMT produces set of K states for each target token to cap- ture multiple moments of starting translating. To explore how many states are better, we report the HMT performance under various K Table 5 where we adopt different lower bound- aries L (refer to Eq.(1 to get similar latency for comparison.
Table 5: HMT performance with
various states number K.
L
K
AL
BLEU
2.57
25.57
Low latency
3
2
2.15
26.07
The results show that considering multiple states is signifi- cantly better than considering only state (K = 1), demon- strating that HMT finds a better moment to start translating from multiple states. For multiple states, larger state ber K does not necessarily lead to better SiMT performance, and HMT exhibits specific preferences of K at different la- tency levels. Specifically, a smaller K performs better under low latency, and the best K gradually increases as the latency increases. This is because translating moments with large gap may interfere with each other during training (Elbayad et al. 2020: Zhang et al. 2021 Zhang & Feng 2021c). where the gap is more obvious at low la- tency. Taking K = 6 as an example, the gap between READ I tokens/6 tokens (i.e., low latency) is more obvious than READ 10 tokens/15 tokens (i.e.. high latency), as the latter contains more over- laps on the received source tokens. Owing to the proposed boundary for translating moments, HMT can avoid the interference via setting suitable K for different latency levels. Further, compared with GSiMT directly considering arbitrary READ/WRITE combinations during training (Miao et al, 2021), HMT pre-prunes those unfavorable moments and thereby achieves better performance.
2
4
2.20
27.60
1
6
2.28
25,69
5.86
28.20
Middle latency
6
4
4.90
30.14
5
6
4.74
30.29
4
8
4.69
29.35
9.71
30.36
High latency
10
6
9.06
31.32
9
8
8.36
31.45
8 10
8.27
31.36
8

---

Published as a conference paper at ICLR 2023
5.3 SUPERIORITY OF ATTENTION BETWEEN STATES
Self-attention between states (in Eq. enables HMT to consider multiple moments of starting translating and thereby capture comprehensive state representations. To
Table 6: Performance with various
modes of self-attention in HMT.
verify the effectiveness of self-attention between states, we compare the performance of paying attention to mul-
Training Multiple
Inference
AL
BLEU
Multiple
3.46
29.29
tiple states or only one state of each target token. To
Multiple
Max
3.37
28.99
this end, we apply three modes of self-attention, named Multiple, Max and Selected. 'Multiple' is the proposed self self-attention in HMT that the state can attend to multi- ple states of previous target tokens. 'Max' means that the state can only attend to one state with the maximum translating moments of each target token. 'Selected' means that the state can only attend to the selected state used to generate each target token, which is the most common inference way in the existing SiMT methods (i.e., once the SiMT model decides to start translating, subsequent translations will pay attention to the target representation resulting from this decision.) (Ma et al., 2019 2020 Note that all modes of attention need to avoid information leakage between states, i.e., satistying J,K'' I fi,k in (2) The detailed introduction of these attention refer to Appendix C.6
Multiple
Selected
3.42
28.83
Max
Max
3.30
28.66
Max
Selected
3.36
28.46
As shown in Table using 'Multiple' in both training and inference achieves the best performance. Compared with 'Max' which focuses on the state with the maximum translating moment (i.e., con- taining most source information), 'Multiple' brings 0.65 BLEU improvements via considering mul- tiple different translating moments (Zhang & Feng 2021c). In inference, considering multiple states with different translating moments can effectively improve the robustness as SiMT model may find that it made wrong decision at the previous step after receiving some new source tokens (Zhegg et al. 2020b). For 'Selection' if SiMT model makes a wrong decision on when to start translating, subsequent translations will be affected as they only focus on this selected state. Owing the attention between states, HMT allows subsequent translations to focus on those unselected states, thereby mitigating the impact of uncertain decisions and bringing about 0.46 BLEU improvements.
5.4 QUALITY OF SELECTION
HMT judges whether to select state si,k by predicting its confidence G, To verify the quality of the selection based on predicted confi- dence, we calculated the token accuracy under different confidences in Figure 5 There is obvious correlation between confidence and token accuracy, where HMT learns higher confidence for those states that can generate the target tokens more accurately. Espe- cially in inference, since HMT selects state $i,k to generate the tar- get token when its confidence C, 2 0.5, HMT learns to start trans- lating at the moments that can generate target tokens with more than 60% accuracy on average, thereby ensuring the translation quality.
Lev 11
75
Hath Knmes i Kiwdy
(0.025)
(0225.05) (05.8.75) 10.75,1 Cusfidesee GiAk
Figure 5: Token accuracy with predicted confidence.
Besides token accuracy, we further analyse the relationship between translation quality and selection result over the whole sequence. We report the BLEU score under different probabjlities of selection re- sult p(z) in Figure 6 where we apply p(z) to avoid the target length |z| influencing p(z). The selection results with higher proba- bility always achieve higher BLEU scores, showing that training by marginal likelihood encourages HMT to give higher probability the hidden sequence (i.c., selection result over the whole sequence) that can generate the high-quality observed target sequence.
Lay aarvy (1 (1-22.0
BILEU
20
(6.661
10 106.0:11
10 10.20. 00..000 yooR
Figure 6: BLEU score with probability of selection result.
CONCLUSION
In this paper, we propose hidden Markov Transformer (HMT) for SiMT, which integrates learning when to start translating and learning translation into unified framework. Experiments multiple SiMT benchmarks show that HMT outperforms the strong baselines and achieves state-of- of-te----th-an performance. Further, extensive analyses demonstrate the effectiveness and superiority of HMT.

---

Published as a conference paper at ICLR 2023
ACKNOWLEDGMENTS
We thank all the anonymous reviewers for their insightful and valuable comments.
REFERENCES
Ashkan Alinejad, Hassan S. Shavarani, and Anoop Sarkar. Translation- supervision for policy generation simultaneous neural machine translation. Proceedings of the 2021 Conference Empirical Methods in Natural Language Processing, pp. 1734-1744, Online and Punta Cana, November 2021. Association for Computational Linguistics. https:::
Dominican Rcoumbie,,ovemberr
emn semmlp-main... 130
Naveen Arivazhagan, Colin Cherry, Wolffang Macherey. Chung-Chemt Chiu, Semih Yavuz, Ruom- ing Pang, Wei Li, and Colin Raffel. Monotonic infinite lookback attention for simultaneous ma- chine translation. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pp. 1313-1323, Florence, Italy, July 2019, Association for Computational Linguis- tics. doi: 10. 18853/1111199 126. URL htLpttss::////// //ap::://cca http///aaaattttoo antho wtg.y/org/PP1. 1126
Leonard E. Baum and Ted Petrie. Statistical inference for probabilistic functions of finite state markov chains. The Annals of Mathematical Statistics, 3766):555--1663 1966. ISSN 00034851.
URL http::tp///ttt jstor. httg///taaleee2222////
Kyunghyun Cho and Masha Esipova. Can neural machine translation do simultaneous translation? 2016. UtLhttp::/// httl:tttpp//xi......mm wwg. 1606..6660000 02012
Fahim Dalvi, Nadir Durrani, Hassan Sajjad, and Stephan Vogel. Incremental decoding and train- ing methods for simultaneous translation in neural machine translation. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Lin- guistics: Human Language Technologies, Volume 2 (Short Papers), pp. 493 499, New Orleans, Louisiana, June 2018 Association for Computational Linguistics. doi: 10. 10.18653/v1/N18- 1N7722299/
URL https: www. aclweb. wwtp/anthooogy/...aooooo
Maha Elbayad, Laurent Besacier, and Jakob Verbeek. Efficient Wait-k Models for Simultane- ous Machine Translation, 2020. URL htttp::/dx.....oo//..ooo.. doi org/ 21411/11331111 21437//ntersspeee. Interspeech..
2020-1247
Alvin Grissom II, He He, Jordan Boyd-Graber, John Morgan, and Hal Daumé III. Don't 't until the final verb wait: Reinforcement learning for simultaneous machine translation. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp- 1342-1352, Doha, Qatar, October 2014. Association for Computational Linguistics, doi: 10. 1/D14-114 URL
www. aclweb...c.... org/ http//oohhooly//a/e 1140
Jiatao Gu, Graham Neubig, Kyunghyun Cho, and Victor O.K. Li. Learning to translate in real-
time with neural machine translation. In Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 1, Long Papers, pp. 1053 1062, Valencia, Spain, April 2017. Association for Computational Linguistics. URL https:
// //www. ac web. orgo ant www.nhhholyyyly.oo (atgy///171111 ottpl//////1999911
Shoutao Guo, Shaolei Zhang, and Yang Feng. Turning fixed to adaptive: Integrating post-evaluation
into simultaneous machine translation. In Findings of the Association for Computational Lin- guistics: EMNLP 2022, pp. 2264-2278, Abu Dhabi, United Arab Emirates, December 2022. Association for Computational Linguistics. URL Sindings-emnlp...
Sathish Reddy Indurthi, Mohd Abbas Zaidi, Beomseok Lee, Nikhil Kumar Lakumarapu, and Sangha Kim. Language model augmented monotonic attention for simultaneous translation. In Pro- ceedings of the 2022 Conference of the North American Chapter of the Association for Com- putational Linguistics: Human Language Technologies, pp. 38-45. Seattle, United States, July 2022. Association for Computational Linguistics. doi: 10.18653/v maacl-main.3. URL
https: ttops://aci antho www.antwwoo.oogg/.aooe ogy.org/2022. naacl -ma in.3
10

---

Published as a conference paper at ICLR 2023
Dan Liu, Mengge Du, Xiaoxi Li, Ya Li, and Enhong Chen. Cross attention augmented trans- ducer networks for simultaneous translation. In Proceedings of the 2021 Conference on Em- pirical Methods in Natural Language Processing, pp. 39-55, Online and Punta Cana, Do- minican Republic, November 2021. Association for Computational Linguistics. URL https: //acllantholoooy.orrggo
emn emnlp-main..
Mingbo Ma. Liang Huang, Hao Xiong, Renjie Zheng, Kaibo Liu, Baigong Zheng, Chuanqiang Zhang, Zhongjun He, Hairong Liu, Xing Li, Hua Wu, and Haifeng Wang. STACL: Simultaneous translation with implicit anticipation and controllable lateney using prefix-to- toet-oiixxxxxxo framework. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pp. Florence, Italy, July 2019. Association for Computational Linguistics. doi: 10.18653/ v1/P19- 1289. URL https:::///........ www. ac web. o
Xutai Ma, Juan Miguel Pino, James Cross, Liezl Puzon, and Jiatao Gu. Monotonic multihead attention. In International Conference Learning Representations, 2020. URL https:// openreview net fort://frrmriii Rd=HHy
Yishu Miao, Phil Blunsom, and Lucia Specia. generative framework for simultaneous machine translation. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pp. 6697-6706, Online and Punta Cana, Dominican Republic, November 2021. Association for Computational Linguistics. URL httpss//aclanttheloryycooo org.22221110../222111111
emm.ppmminn. 536
Myle Ott, Sergey Edunov, Alexei Baevski, Angela Fan, Sam Gross, Nathan Ng, David Grangier, and Michael Auli. fairseq: A fast, extensible toolkit for sequence modeling. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics (Demonstrations)) pp. 48-53, Minneapolis, Minnesota, June 2019. Association for Computational Linguisties. doi:
URLhtttss www. acl wwwwww.w.bb.....ccc.......w.wwbw wwtp://to//aatloggggoeo
N19-4009
Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. Bleu: a method for automatic
evaluation of machine translation. In Proceedings of the 40th Annual Meeting of the Associa- tion for Computational Linguistics, pp. 311-318, Philadelphia, Pennsylvania, USA, July 2002.
Association for Computational Linguistics. doi: 10.3115/177303.1073133 URL https:
7/ //www. ac web.. org/ httprg///nooooogoooPooooo 1040
I. Rabiner and B. Juang. An introduction to hidden markov models. IEEE ASSP Magazine, 3(1):4- 16, 1986. doi: 10. N0.1109/MASSPPP198..16666 165342.. URL
ieee org/
document/116533
Colin Raffel Minh- Than-Thang Luong. Peter J. Liu, Ron J. Weiss, and Douglas Eck. Online and linear-time attention by enforcing monotonic alignments. In Doina Precup and Yee Whye Teh (eds.), Proceedings of the 34th International Conference on Machine Learning, volume 70 of Proceedings of Machine Learning Research, pp. 2837-2846. PMLR, 06-11 Aug 2017. URL ml www.ppressslooomm www.0//aaaaalaaaaoaaaamaa httl
Rico Sennrich, Barry Haddow, and Alexandra Birch. Neural machine translation of rare words with subword units. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). pp. 1715-1725, Berlin, Germany, August 2016. Association for Computational Linguistics. doi: (8.18653/v1/) 1/P16 1162. htttt:: www aclweb.
org//anthology/Pl1-111
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, ukasz Kaiser, and Illia Polosukhin. Attention is all you need. In I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett (eds.), Advances in Neu- ral Information Processing Systems 30, pp. -008-6008. Curran Associates, Inc., 2017. URL
nips. wttpss/pppppppppeeeeeeeeeeeeee http:pppppee/lllattteeooooo--o--
yoll---o--eeeeee pdf
Weiyue Wang. Derui Zhu, Tamer Alkhouli, Zixuan Gan, and Hermann Ney. Neural hidden Markov model for machine translation. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pp. 377-382, Melbourne, Australia, July 2018. Association for Computational Linguistics. doi: 10.18653//1/PP 2060.. URL https: 7/acl Monclannholocc hww.aaoooooo..oooolll......
11

---

Published as a conference paper at ICLR 2023
Xueqing Wu, Lewen Wang, Yingee Xia, Weiqing Liu, Lijun Wu, Shufang Xie, Tao Qin, and Tie- Yan Liu. Temporally correlated task scheduling for sequence learning. In Marina Meila and Tong Zhang (eds.), Proceedings of the 38th International Conference on Machine Learning, volume 139 of Proceedings of Machine Learning Research, pp. 11274-11284. PMLR, 18-24 Jul 2021. URL httth::://pp/seeeeeee ings..ngg..... ml www...press/vll333
html
Shaolei Zhang and Yang Feng. ICT's s system for AutoSimTrans 2021: Robust char-level simultane- ous translation. In Proceedings of the Second Workshop on Automatic Simultaneous Translation, pp. 1-11, Online, June 2021a. Association for Computational Linguisties. doi: 10. 10.18653//1/2011. autosimtrans- 1.1. URL httpts///aaclaabtoooo.ooo/////// amtoo
Shaolei Zhang and Yang Feng. Modeling concentrated cross-attention for neural machine translation with Gaussian mixture model. In Findings of the Association for Computational Linguistics:
EMNLP 2021, pp. 1401-1411, Punta Cana, Dominican Republic, November 2021b. Association for Computational Linguistics. doi: 10. 1/2021 ceinddins-emmnl...
URL https://
aclanthology org. 2ttgg2221....mm wwtp:/iiiiiiiinn---eeeee
121
Shaolei Zhang and Yang Feng. Universal simultaneous machine translation with mixture-of-expprrts wait-k policy. In Proceedings of the 202/ Conference Empirical Methods in Natural Lan- guage Processing, pp. 7306-7317, Online and Punta Cana, Dominican Republic, November 2021c. Association for Computational Linguistics. doi: 11010////11222111mmpmmm.o emnlp-main, 581
URL
Shaolei Zhang and Yang Feng. Modeling dual read/write paths for simultaneous machine transla- tion. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume Long Papers), pp. 2461 -2477, Dublin, Ireland, May 2022a. Association for Compu-
tational Linguistics. URL https: http://claaaathooocoooooo
org 2022 acl woni-long.... 176
Shaolei Zhang and Yang Feng. Reducing position bias in simultaneous machine translation with length-aware framework. In Proceedings of the 60th Annual Meeting of the Association for Com- putational Linguistics (Volume 1: Long Papers), pp. 6775 Dublin, Ireland, May 2022b. Association for Computational Linguistics. URL /tRphtttp////clcccccoo/y// orgg220020......222..2..
acl www.nn..5... 467
Shaolei Zhang and Yang Feng. Gaussian multi-head attention for simultaneous machine transla- tion. In Findings of the Association for Computational Linguisties: ACL 2022, pp. 3019-3030 Dublin, Ireland, May 2022c. Association for Computational Linguistics. doi: 10. 10.18653/v1/2022. andings-acl.233. URL. URI.https::/aclanthol www.a//nonoooooooo//2222.... findi findings-asn naddings-al.... 238
Shaolei Zhang and Yang Feng. infformation-transportt-tt policy for simultaneous translation. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pp. 992-1013, Abu Dhabi, United Arab Emirates, December 2022d. Association for Computational Linguistics. URL
Logy. org/2022.2222....o emnlp-main. 65
Shaolei Zhang, Yang Feng, and Liangyou Li. Future-guided incremental transformer for si- multaneous translation. Proceedings of the AAAI Conference Artificial Intelligence, 35 (16):1442--14436, May 2021. URL
wwl.oorg/nndde.. wiiiinixxx.. www.de.wppp///////// phpp/AAI/
articl hww.cc/levewweeo. artiicl/vvee/117666
Shaolei Zhang, Shoutao Guo, and Yang Feng. Wait-info policy: Balancing source and target information level for simultancous machine translation. In Findings of the Association for Com- putational Linguistics: EMNLP 2022, pp. 2249-2266 Abu Dhabi, United Arab Emirates, De- cember 2022. Association for Computational Linguistics. URL org/2022. findings emnlp. 166
Baigong Zheng, Renjie Zheng, Mingbo Ma, and Liang Huang. Simultaneous translation with flexi- ble policy via restricted imitation learning. In Proceedings of the 57th Annual Meeting of the As- sociation for Computational Linguistics. pp. 5816-5822, Florence, Italy, July 2019a. Association for Computational Linguistics. doi: 10. 10.18653/v 1/P19- 1582. URL https: :ttppp/// wwww. aclweb. lorg/antho ogy P-tp:/19911588
12

---

Published as a conference paper at ICLR 2023
Baigong Zheng, Renjie Zheng, Mingbo Ma, and Liang Huang. Simpler and faster learning of adaptive policies for simultaneous translation. In Proceedings of the 2019 Conference on Em- pirical Methods in Natural Language Processing and the 9th International Joint Conference Natural Language Processing (EMNLP-IJCNIPP) pp. 1349-1354 Hong Kong, China, Novem- ber 2019b. Association for Computational Linguistics. doi: 10.18655/1//D19-1133. URL
www. wttp::/wwww.......//a//oooooooo.
Baigong Zheng, Kaibo Liu, Renjie Zheng, Mingbo Ma. Hairong Liu, and Liang Huang, Simul-
taneous translation policies: From fixed adaptive. Proceedings of the 58th Annual Meet- ing of the Association for Computational Linguistics, pp. 2847-2853, Online, July 2020a. As-
sociation for Computational Linguistics. doi: 10. 10.18653/v ac0000000m.mm...mmm2222.. //www.aclweb org/ wttgg///aaatoooo//22aaa.....
URL https:
ma ma.in... 254
Renjie Zheng, Mingbo Ma, Baigong Zheng, Kaibo Liu, and Liang Huang. Opportunistic de- coding with timely correction for simultaneous translation. In Proceedings of the 58th An- mual Meeting of the Association for Computational Linguistics, pp. 437-442, Online, July 2020b. Association for Computational Linguistics. doi: 10.18653/v1/201/ acl- 42.....2.. URL httpss//eeeaartloooy..ooo//////. www.aaalmmmmmammmm.m
main.
13

---

Published as a conference paper at ICLR 2023
A CALCULATION OF TRANSITION PROBABILITY
selection
21
22
2:3
24
26
27
result
21
122
0
0
12
Transition from the 3rd state $2.3 when generating to the 4th state #4.4 when generating thi
St
0
0
O
3g
34
0
24 =
pl2s = 3)
C3.3 x
(1 (3.2 X
26
select state #3.4
don't select fa.2 and 83.3
96
26 =
Figure 7: An example to depict the calculation of transition probability p(z; 23-1).
We give the transition probability p(2; (4-1) between selection results in Sec 3.2 and here we describe the calculation of transition probability in more detail.
First of all, specific selection 2; represents translating Y: when receiving first fa,s, source tokens through the state 81 Similarly, selection 2,-1 represents translating yi-1 when receiving first 4,-12; source tokens through the state 8i- 1:,_--;1- Due to the monotonicity of the moment to start translating SiMT, it is not possible to transfer from 2;-1 to 2; 11-1,2;-1 > li,,, Then for f4-1, t2,25 since HMT judges whether to select each state by one (from 81,1 to ki,K) and starts translating if a state selected, the premise of the transition from 2;-1 to 2 is that all states whose translating moment belongs to [ fz..)) and 81,2, is confident to be selected. As the example shown in Figure 7 the transition probability p(23 4 22 = 3) from 22 23 consists of the probability of unselecting $3,2 and 83,3, and the probability of selecting $3,4- Formally, the transition probability p(z; 21-1) is calculated as:
not confident be selected,
Ci.s X
II
(1-c.) if the 2 4--18111
p(2; | 12-1) =
14-1,4-1 Shilsh
(12)
if bf, < 41-1
B DYNAMIC PROGRAMMING IN HMT
HMT treats which states are selected (i.e, when to start translating) as hidden events and the target tokens as observed events, and organizes the generation of the target sequence as a hidden Markov model. During training, HMT is optimized by maximizing the marginal likelihood of the target sequence (i.e., observed events) over all possible selection results (i.e., hidden events):
ply x) = Σ ply x, 2) x p(z)
(13)
Σ E p(y Xst., y cin 2)) p(z: 2--1)
(14)
We apply dynamic programming (a.k.a. forward algorithm in HMM) to calculate the marginal like- lihood efficiently (Baum & Petrie 1966). Formally, we introduce the intermediate variable a; (k) to represent the probability of selecting the htt state ši,k when generating the first target tokens yso defined as:
a, (k) =
l)).
(15)
Initialization The initial a1 (k) is calculated as:
0n (k) Ty pvv1 Xg6. V<1,
(16)
where Fe = =((2 =k) is the initial probability of selecting $1 In the implementation, we add a certain selection 20 with p(2o) before 21 to indicate that no source tokens received at the
14

---

Published as a conference paper at ICLR 2023
beginning of translation, i.e., tu,s0 = 0. Accordingly, the initial probability p(21) is calculated via the transition probability from 2y to 21-
Recursion Following, at (k) is calculated by summing the extensions of all transitions from the previous step's selection to the current selection. Therefore, ag (k) is calculated as the following recursion form:
K ar (k) Σ ai-1 (k)) x =k 2--1 x =1=1
x p(u X = =k).
(17)
Termination Finally, the marginal likelihood of the target sequenee over all possible selection sults is calculated as:
ply x) 0a
al (k).
(18)
C EXPANDED ANALYSES
C.I SPECIFIC IMPROVEMENTS OF POLICY AND TRANSLATION
The proposed HMT integrates the two key issues in SiMT 'learning when to start translating" (i.e., inference policy) and 'learning translation' (i.e., translation capability) into uni- fied framework. Since HMT explicitly models multiple mo- ments of starting translating in both training and inference, HMT architecture can flexibly cooperate with other inference policies, such as wait-k policy (Ma et al. 2019), which allows us to learn the specific improvements brought by HMT archi- tecture and inference policy, Specifically, we report the results of 'HMT architecture + Wait-k inference' in Figure 8 where wait-k inference for HMT architecture is realized by forcing HMT to always select the last state.
BLEU Soore
F.21 oTl
HMT aechsee HIMT adereae)
HMT metue Wark EOENEE
18
Watk
Matic informee
10
12
Average Lagzing (AL, token)
Figure 8: Specific improvements brought by HMT architecture and inference policy.
By comparison, for wait-k inference, HMT architecture has stronger translation capability due to the comprehensive con- sideration of multiple translating moments, thereby bringing about 0.8 BLEU improvements, Fur- ther, compared to wait-k inference, HMT inference learns more accurate moments to start translating and brings about 2.8 BLEU improvements on average. In particular, the improvements brought by the inference policy are more obvious at low latency, as the precise translating moments are more important for SiMT under low latency (Arivazhagan et al. 2019).
C.2 ABLATION STUDY ON THRESHOLD OF CONFIDENCE
In inference, HMT will select state &t, generate y; when its confidence C 2 0.5, where 0.5 can be regarded as the confi- dence threshold that HMT believes that the state can generate the correct target token. Figure 5 also proves that the state with higher confidence can generate the target token more ac- curately. To study the effect of the confidence threshold in
82
30
BLEU Score
8.5)
HMT, we show the HMT performance under different confi-
6-
dence thresholds ă in Figure where HMT will select state 8t,k to generate y/: when its confidence C 2 inference.
0-
10
Average Lagging (AI, taken)
In inference, moderate confidence thresholds, such 0.4 and d = 0.5, achieve similar SiMT performance, indicating that HMT is not sensitive to the setting of the confidence threshold. Furthermore, as the confidence threshold decrease to 0.2, HMT starts translating much earlier, resulting in slight decrease in translation quality. As the threshold increases 0.8, the la- tency HMT increases, but the improvement in translation quality not obvious, which indicates that 0.5 confidence is enough to generate the correct target token for the state in HMT.
Figure 9: HMT performance under various thresholds confidence.
15

---

Published as a conference paper at ICLR 2023
C.3 CASE STUDY
We conduct case studies on two difficult De-+En cases Figure T0 and Figure 11 where the word order difference between German and English is more challenging for SiMT model (i.e., the verb in German always lags behind) (Ma et al.) 2019 Zhang & Feng 2021c). We show the specific inference process of HMT and visualize the outputs of all states (including selected, unselected and not considered.) to illustrate that HMT successfully learns when to start translating.
Souree:
Lisle
stehen
Pro
mis
<
the
stavel
J0y
colelritiey
Reference:
Chere 10] colibbont 120/ ily names the list, Sig
Step
Inputs (reeeived source sequeeee)
State
Confidence)
Outputs
$1,1
auf
Listo
$1.2
0.38
auf
der
Liste
stehen
$155
0.38
Lide stehen
$1.
there
5
auf der
Liste stehen
101
0.98
ure
auf der
Lide stehen
101
$1,2
0.85
101
stchen
94,1
8
auf der
Listc stchen
101
Pro
K4.2
0.02
stehen
Pro2
54.5
celebvin
10
auf
Lisic stehen
101
Pro1 mis
%
0.87
rities
11
auf der
Liste stchen
101
Pro mis
0.26
-i--
Liste stehen
Profe
542
on
13
auf
Lide stehen
101
Prog mis
$7,1
0.99
the
14
auf
List stehen
101
Protedé
mis
1.00
list
Lide
stehen
Pro
9g.1
16
auf
der
iite
stehen
101
Progee
mis
</p>
$1p1
1,00
</>>
(a) Inference process of HMT. 'State' records the currently considered state, and HMT selects the current state and starts translating the target token when its corresponding confidence greater than 0.5. The outputs marked in red strikethrough represent potential outputs for those states that are not selected.
Target
also
Liste
stehen
101
Pro@už
mis
(b) Visualization of all state outputs with their cross-atiention to the received source tokens. For the outputs, the outputs marked in red represent unselected states, the outputs marked in blue represent the selected states, and the outputs marked in gray represent the states that are not considered in inference (i.c., those states after the selected state, or the states whose translating moment is carlier than the moment of translating the previous target token). For the cross atteentttn, the numerical values in the squares report the cross catss-attettion weight, and blank squares indicate that those source tokens have not been received when translating the target token.
Figure 10: Case study of #2124 in De-En test set, where we apply HMT with L 2 and K =4 4.
Case with Word Order Difference As shown in Figure 10 "auf der Liste" in German is at the beginning of the sentence, while the corresponding translation 'on the list" is at the end in English. For this case, HMT decides not to select states $1,1, $1.2 and $1,3 because their confidences are less than 0.5, especially the outputs of these states also prove that these moments are not good to start translating. Then, HMT generates 'There' at state 81,4- Similar situations also occur when generating 'celebrities" and 'on'. Figure TO(b) shows more specific outputs for all states and their cross-attention on the received source tokens, where each state corresponds to a moment of starting translating. By selecting one state from K states, HMT effectively finds the optimal moment to start translating, i.e., the moment that can generate the correct target token with lower latency. In particular, those unselected states (i.e., less confidence) tend to produce incorrect translations, while the selected states can produce correct translations by paying attention to the newly received source contents. About this, already present statistical analysis between confidence and token accuracy
16

---

Published as a conference paper at ICLR 2023
in Figure Further, the selected state always tends to be the earliest state that can generate the correct translation, i.c., the state with lower latency.
Source:
Sie schiggis chen einen A Autgueg ourgd (s) fall zn erwarten und dis passojas siene eintich nicht
Sig> XAp
secallen
dhat Auypooed
slmpll
Ihyy
quiti huppen,
Step
Inputs freceived source sequence))
State
Confidence
Outputs
schnye'e enen
0.51
they
schingg enen
F21
sremed
Sie schigg enen cinen Au
F41
0.87
to
Sie wchive
Au1g000 ounfe 68)
0.09
-be-
& esen Autiga oun(2
Sae & enen cinen Au ounic
[Gll zu
34,3
0.07
Jassp
Sie schiggg enen cinen Autage oun/e
fall zu ereaain
56,4
0.88
expect
Sie schigg & esen
Autade oum/g00
fall
0.91
Autg4a ounvi (
crwante
auto
Sãe schigin esen cinen A ounig.
fall zu crwarion
5r,1
0.67
accident
11
Sie schire' enen einen Autgle ounug (6) fhll erwante
0.90
Sie 2 shiio enen cinen Aurgoe ourced (6)
Autäia cunil fall
und und
0.64
and
HMll 20 cnwarien
$10.1
0.55
Sie schigg enen cinen Autgle ounug (6)
fall zu erwarten
und das pass Gill enwarten
9111
0.24
Is
Sie whin'
Aurgue oun
und das pus skrric
0.54
happened
& esen
A oun(z
crwarten
pas@ ( sione
S12.1
Sae & eaen cinen A ounicd
Gill zu cowarton
und das puse0 siene cinfach
$122
0.56
simphy
18
Sie schígg enen cinen A oun/e
fall zu cnwaaten
und das p sieme einfach
$131
0.03
becouse
2
Au oum(e.))
fill
und pasej
einfach nicht
0.97
& esen
Autzia ounve @ fall crwante
p siene
5p4.
21
Sie schigng esen cinen A ounig fall zu crwarton
und das pas@u siene einfach nicht
$14
0.95
22
Sie schim' enen einen Aurgie ounus
fhll erwarten
und das passi sieme einfach nicht
0.40
schii
Aut
posdelg)
Inference process of HMT.
Target
aeciddennt
Ibyy
schiigo8
Autigoak 0um
Source
ubd
des
slerte
nicht
Simpl Humphy
Sir
1000
cince
Gill
crwaren-
das-
sterie
sinfach-
</p>
(b) Visualization of all state outputs with their gross-attention the received souree tokens.
Figure 11: Case study of #378 in De- test set, where we apply HMT with L-3
3 and K 6.
17

---

Published as a conference paper at ICLR 2023
Case that Verb Lags Behind Figure 11 gives more complex case the verb 'erwarten" and 'passierte' in German lag behind, which requires the SiMT model to accurately judge when to start translating (Grissom II et al. 2014; Ma et al. 2019). In HMT, when the verb has not been received, the states always get lower confidence, so HMT can start translating the corresponding 'expecr' and 'happened' after receiving the source verb. Besides, the cross-attention in Figure [T(b] shows that HMT generates the correct translations at state 54.4 and $11.2 because they pay attention to the received verb 'erwarten' and "passierte" (Zhang & Feng 2021b), proving the effectiveness of predicting confidence according to the target representation and received souree contents.
C.4 EFFECT OF STATE LOSS Latate
Target
(a) HMT without the state loss Lstate- Translation result: 'my first choice was to train the army.". The green box marks the incorrect translation, where the correct translation should be "in" but the model generated 'the'.
Targel
Vill
Source
(b) HMT with the state loss Eatate. Translation result: 'my first choice was to train in the army.'.
Effect of the proposed state loss Estate- We apply HMT with L = and K = 6 and visualize the state outputs of case #912 in De-FE test set (Source: 'meine erste Wahl war eine Ausbildung in der Armee.'; Reference: 'my first choice was to go in the army."), "). The outputs marked in red represent unselected states, the outputs marked in blue represent the selected states, and the outputs marked in gray represent the states that are not considered in inference. The numerical values in the squares report the cross-attention weight, and blank squares indicate that those source tokens have not been received when translating the target token.
Figure 12:
For HMT training (refer Sec 3.2). we propose the state loss Lstate encourage HMT generate the correct target token no matter which state is selected (i.e., no matter when to start translating). and the ablation study in See 5.1 demonstrates the improvements brought by the state loss. To further study the effect of state loss Lstote, we visualize the state outputs of case #912 with and without Catate in Figure
When removing the state loss Cytate in training, those states that are selected with lower probability hardly learn to generate the correct target token, because the emission probability from these states may contribute little to the marginal likelihood of the target sequence. As shown in Figure [2(a] some later states incorrectly generate '!' when generating the first target token, and some states generate "was" when generating the third target token, etc. Although in most cases HMT will not select these states during inference, when the selection is slightly uncertain, the model may output the incorrect target token, such as generating 'the" instead of 'in' (marked with the green box in Figure 12(a).. As shown in Figure 12(b), with the state loss Cstates multiple states in HMT all can learn to generate the correct target token, matter when to start translating the target token. Therefore, HMT can still generate the correct target token even if makes the wrong decision in the selection, thereby improving the robustness on the selection.
18

---

Published as a conference paper at ICLR 2023
C.5 QUALITY OF POLICY
The quality of the policy directly affects the SiMT performance, and good policy should en- sure that the model receives its corresponding source token before translating each target token, thereby achieving high-quality translation. Following Zhang & Feng (2022a), we calculate the pro- portion of the ground-truth aligned source tokens received before translating for the evaluation of the policy quality. We apply RWTH De-=E alignment dataset and perform force-decoding To get the moments of translating each target token. Specifically, we denote the ground-truth aligned source position of y; as d, and use Di to record the translating moments of ys (i.e., the number of received source tokens when translating yi). Given the alignment a, and translating moment Di. the proportion of aligned source tokens received before translating is calculated
1.00
% Alignments are Received
Proportion
(19)
before Translating
Y
0.95
1,
a
S 9i
where 1 14,59
(20)
0, 0 9i
0.90
-
HMT
The evaluation results are shown in Figure 13 Compared with previous policies, HMT receives more aligned source tokens before translating under the same latency, which demonstrates that HMT can make more precise decisions on when to start translating. Owing to the superiority of policy, HMT can receive more aligned source tokens and thereby achieve higher translation quality than previous methods under the same latency.
0.85
Adaptive Watkk
10 12
Average Lagging (AL, token)
Figure 13: Quality of the policy in HMT. We calculate the proportion of aligned source tokens received before translating in various policies.
C.6 WHY SELF-ATTENTION BETWEEN STATES?
selected
Sta 6
Eiu GIEmT
ya
V3
(a) Multiple
(b) Max
(c) Selected
Figure 14: Schematic diagram of self-attention between states in HMT, named 'Multiple.. We also propose two variants, 'Max" and 'Selected', to demonstrate the superiority of 'Multiple' attention mode. The schematic diagram shows an example of HMT with L = and K = 4, where the translating moments of states for ys and y4 to (3, 4, 5, 6) and t. = (4, 5, 6, 7), respectively.
HMT applies self-attention among all states based on Eq. and here we explain why HMT applies this attention pattern in more depth. For comparison, we introduce three modes of self-attention in named Multiple, Max and Selected:
Sec 5.3
Multiple:
The self-attention mode between states in HMT. In 'Multiple', the state can pay attention to multiple states of previous target tokens. Taking Figure [4(a) as an example, when translating y4, state 84,3 can pay attention 83,1, 83,2, 83,3 and 83,4 of ys, meanwhile state $1,3 also pay attention to $4,1, $4,2, $1,3 of y1-
Max: 'Max', the state can pay attention to one state of each target token, which has the example, when translating y4, state 84,3 can only pay attention to 83,4 of ya. as $3,4 = 6 is the state with the maximum
maximum translating moment. Taking Figure 146b
Phttps://wwer i 16. i fonfootaiiii wwwhh aachea .eeecheendeeeeeeeeee /e/goll// 1 dA1 gnmentt/
"Force-decodin:: we force model to generate the ground-truth target token, thereby comparing the translating moments with the ground-truth alignments
For many-to-one alignment, choose the last source position in the alignment.
19

---

Published as a conference paper at ICLR 2023
translating moments that $4.3 (f4.3 6) can pay attention to. Meanwhile, state 84,3 pay attention to $1,1, $1,2, $1,3 of y/ as well.
The most common attention mode in the existing SiMT methods. In 'Selected', the state can pay attention to the selected state used to generate the previous target token. Once the SiMT model determines WRITE (i.c., selects a state), subsequent translations will only depend on this target representation. Taking Figure [4(c) an example, assuming that $3,3 has been selected to generate ys. subsequent 84,2- $4.3 and 84,4 can only focus on the representation of $3,3- Note that 54,1 cannot focus on 53,3 as f3,3 > t4,1-
Selected:
Multiple v.s. Max The results reported in Table 6 show that 'Multiple' brings 0.65 BLEU im- provements compared with 'Max'. The maximum translating moments that 'Multiple' and 'Max' can focus are the same (in both 'Multiple' and 'Max', state 84,3 can pay attention to $4,3 with 13,4 = 6.), but 'Multiple' can also consider those states that start translation earlier, such as 83,1, 83,2 and 83,3- Comprehensively considering the representation of multiple states helps 'Multiple' make more precise judgments and get more accurate state representations (Zhang & Feng 2021c). Fur- thermore, in 'Max', the current state always focuses on one previous state with the largest translating moment, regardless of which state is selected to generate the previous target token. For example, even if 84.2 is selected to generate ys, state 84,3 still only pays attention 84,3 in 'Max', where ignoring the previous selected state $4,2 will disturb the dependency of y4 on Y/ and thereby affect the translation quality. Therefore, owing to more comprehensive attention to multiple states, the proposed 'Multiple' attention mode achieves better performance.
Multiple v.s. Selected 'Selected' is the most commonly used attention mode of the current SiMT method, i.c., once WRITE action is decided, subsequent target tokens can only pay attention to the representation of this state (Ma et al.) 2019 Arivazhagan et al., 2019 Ma et al. 2020). The reason for applying 'Selected' attention mode is that the previous methods translating moment and corresponding representation for each target token in inference, unlike HMT which can explicitly model multiple translating moments for each target token. Keeping the only target representation of the selected translating moment is susceptible to inaccurate decisions (Zheng al. 2020b). Assuming that the model selects the state 83.3 to generate y3, but this decision is not necessarily completely accurate, it may be more reasonable to use $3,2 or $3,1 to generate ys- 'Selected' requires subsequent states only focus on the representation of $3,3, which may cause translation errors. 'Multiple' allows the following states to focus on multiple states, including those not selected, and thereby make comprehensive decisions to achieve better results.
only retain unique
In conclusion, 'Max' ignores the previous selected state, 'Selected' only considers the selected state, while the proposed 'Multiple" attention mode pays attention to all previous states and keeps training and inference matching. Therefore, 'Multiple' performs best among these three attention modes.
D HYPERPARAMETER
Table gives the hyperparameter settings of HMT.
E NUMERICAL RESULTS WITH MORE METRICS
E.1 LATENCY METRICS
Besides Average Lagging (AL) (M et al. 2019). we additionally use Consecutive Wait (CW) GGu et al. 2017), Average Proportion (AP) (Cho & Esipova 2016) and Differenniable Average Lagging (DAL) (Arivazhagan al. 2019) to evaluate the latency of HMT. We denote the number of waited source tokens before translating y/ as gi (i.e., the moment to start translating yı), and the calculations of these latency metrics are as follows.
Consecutive Wait (CW) Gu et al, 2017) evaluates the average number of source tokens waited between two target tokens. Given gr. CW calculated as:
CW = i= (gs 1i-1)
Σ |y|
(21)
Σ
1=1 g gg
20

---

Published as a conference paper at ICLR 2023
Table 7: Hyperparameters of HMT.
En-Vi
De-En Transformer-Base
De-En
Hyperparameters
Transformer-Small
Transformer-Br
encoder-layers
encoder-attention--eaad
4
16
moodder-embet -im
512
512
1024
encoder-fin seneder-ff--ebee--eeeeedd
1024
2048
4096
decoder-layy layers
snndder-ateenion-hhaaa
decoder-embed-ding
512
512
1024
1024
2048
4096
dropout
0.3
0.3
0.3
optimizer
adam
adam
adam
adam-3
(0.9, 0.98)
(0.9, 0.98) 0
(0.9, 0.98)
clip-norm
0
0
Ir
2c-4
5c44
5c-4
Ir-scheduler
Inverse_sqrt
inverse_sqrt
inverse_sqrr _qrt
warmup-updates
4000
4000
4000
warmup-init-Ir
le-7
le-7
le-7
weight-decay
0.0001
0.0001
0.0001
label-smoote smoothig
0.1
0.1
0.1
max- max-tokens
16000
8192x4
8192x4
where 19:-9: 20 counts the number of 9: 9:-1 0.
Average Proportion (AP) (Cho & Esipova 2016) evaluates the proportion between the number of
received source tokens and the total number of source tokens. Given Jt. AP is calculated as:
yyl
AP
9
(22)
X
y
Average Lagging (AL) Ma al. 2019 evaluates the average number of tokens that target outputs lag behind the source inputs. Given Ji. AL is calculated as:
AL
91
where argmin (g. |x[)
(23)
y
Differentiable Average Lagging (DAL) Arivazhagan et al. 2019 is a differentiable version of average lagging. Given Jt. DAL calculated as:
y
9N
i ==
DAL
9
X
where gi
max 9i.
lv|)
i> 1
(24)
E.2 NUMERICAL RESULTS
We adjust L and K in HMT (refer Eq. get the translation quality under different latency, For clarity, we present the numerical results of HMT with the specific setting of hyperparameters L and K in Table Table and Table IO Note that for comparison, we - to get the translation quality of HMT under extremely low latency (AL < 1, i.e., the outputs lagging the inputs less than token on average) on Dn When setting -,=1,, we constrain the translating moment 4, of all states to be at least 1, i..., f.R max {min {L + (i 1) + (A 1). 1). Therefore, all states will start translating after receiving at least source token.
21

---

Published as a conference paper at ICLR 2023
Table 8: Numerical results of HMT on En-+Vi with Transformer-Small.
IWSLTI5 En
+Vi
Transformer-Siee Small
L
K
CW
AP
AL
DAL
BLEU
2
1.15
0.64
3.10
3.72
27.99
2
2
1.22
0.67
3.72
4.38
28.53
4
2
1.24
0.72
4.92
5.63
28.59
4
1.53
0.76
6.34
6.86
28.78
1.96
0.83
8.15
8.71
28.86
7
6
2.24
0.89
9.60
10.12
28.88
Table 9: Numerical results HMT on De-EE with Transformer Base.
WMT15 De
En
Transformer-Ban -ase Base
L
CW
AL
DAL
BLEU
-1
4
1.58
0.52
0.27
2.41
22.52
2
4
1.78
0.59
2.20
4.53
27.60
2.06
0.64
3.46
6.38
29.29
1.85
0.69
4.74
6.95
30.29
7
2.03
0.74
6.43
8.35
30.90
9
2.48
0.79
8.36
10.09
31.45
11
3.02
0.83
10.06
11.57
31.61
3.73
0.86
11.80
13.08
31.71
Table 10: Numerical results of HMT on De-+En with Transformer-Bis
WMT15 De
En
Transformer- Big
CW
DAL
BLEU
-1
1.65
0.52
0.06
2.43
22.70
2
4
1.79
0.60
2.19
4.50
27.97
2,04
0.64
3.46
6.30
29.91
1.88
0.69
4.85
30.85
2,06
0.74
6.56
8.47
31.99
9
2.47
0.79
8.34
10.10
32.28
11
2.98
0.83
10.12
11.58
32.46
13
3.75
0.86
11.78
13.09
32.58
22


Reviews:
[{"review_id": "nrAndG3iRK", "reviewer": "Reviewer_WoTh", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper proposes a new model called Hidden Markov Transformer (HMT) to tackle the problem of simultaneous machine translation (simt), which tries to translate source to target live as the source is being receive. The model has to decide when to read the source buffer and when to generate or write target token that achieves both low latency and high accuracy.\nThe method is based on hidden markov chain, which the moments of starting translating are treated as hidden events and it frames the translation results as the corresponding observed events. The model is trained by maximizing the marginal likelihood of the\nobserved target sequence over multiple possible moments of starting translating\nDuring training, for each target token, is model is trained to output a number of possible states and maximize over which state as the starting state for the current target token.\n\nThe results show that the method outperform previous baselines and achieve better latency and accuracy trade-off curve compared to the baselines.", "summary_of_the_review": "Overall, the method is quite novel and produce good results, even though I suggests to add more language pairs to the experiments.", "strength_and_weaknesses": "Strength:\n\n* the method is quite novel and experimental results seem strong\n\nWeakness:\n\n* The explanation and presentation of the methodology is complicated and can be reduced.\n* More experiments of other language pairs, like french, and low-resource languages (like FLoRes) are needed to confirm the method extendability and generalization.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "hJQZ9weK6Bd", "reviewer": "Reviewer_C1gX", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper focuses on the core issue of simultaneous machine translation: time to start translation. The paper proposes HMT inspired by HMM, where the next token generation is regarded as the hidden variables, and target tokens as the observable variables. At each time step, K indicates available length of source for translation, and each state predicts the next token to be yield. During training, the marginal probability of generating the correct token is maximized. During inference, the first state with over 0.5 confidence is selected as the generated token.\n\nExperiments conducted on two SiMT benchmarks validate the effectiveness of HMT. Extensive experiments are also conducted to promote better understanding of how HMT works.", "summary_of_the_review": "This paper addresses the core problem of simultaneous machine translation: when to start translate. The paper propose HMT as a solution. The experiments demonstrates the effectiveness of the proposed method.", "strength_and_weaknesses": "Reasons to accept:\n- The methods produces latent inputs for the translation to boost latency, which is interesting and promising.\n- The conducted experiments are extensive, providing in-depth analysis on how the proposed method works. \n\n\nQuestions:\n1. Is the encoder in Full-Sentence MT bi-directional or uni-directional? If uni-directional, an alternative paper should be cited instead the original Transformer paper.\n2. Why HMT even outperforms Full-Sentence MT in De->En? Also in Section 5.3, I don't understand why Multiple is better than Max, where the latter always provides more information than the former when generating. Could authors please provide more explanations?\n3. What is the system overhead (e.g., memory) of HMT compared to baselines? Since HMT produces K states for each timestep, and self attentions are conducted on these multiple states, the memory requirements for SA would be K^2 compared to vanilla models. This could be a concern for edge devices. \nMissing reference:\nLiu D, Du M, Li X, et al. Cross attention augmented transducer networks for simultaneous translation[C]//Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing. 2021: 39-55.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "FuSMhK4WaUe", "reviewer": "Reviewer_NSCn", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper used a Hidden Markov Transformer(HMT) to improve the READ/WRITE actions in simultaneous neural machine translation. Specifically, this model treats the moments of starting translating as hidden events and the target sequence as the corresponding observed events. The HMT model judges whether to select each state from low to high latency. The authors hoped that the HMT model effectively learns when to start translating under the supervision of the observed target sequence. The proposed model was evaluated on two language pairs and gained improvement over the baseline systems in terms of BLEU scores and latency.", "summary_of_the_review": "This paper provides a new perspective on simultaneous neural machine translation, and sufficient experiments prove the effectiveness of the method. Compared with several strong baseline systems, the proposed method achieves state-of-the-art performance.", "strength_and_weaknesses": "Strength:\n1. The proposed HMT approach gained improvement over the baseline systems in terms of BLEU scores and latency.\n2. Experiments have proved the effectiveness of the method, and there are very reasonable expanded analyses.\n\nWeaknesses:\nThe average pooling result on the hidden states of the received source is only a summary of the past state. And this model didn't use an explicit alignment method. Therefore, the proposed approach is more like a pruning method to cut low probability branches of the READ/WRITE path.  This doesn't quite match what the paper claims that explicitly models multiple possible moments of starting translating in both training and inference.\n\nQuestions:\n1. How to evaluate the proposed method to make better read/write decisions than the other SNMT?\n2. If there is higher confidence in the later state, will the performance of the model be affected by generating too early? \n3. Can you provide an evaluation of the quality, clarity, and originality of the work?", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works."}, "extra_text": {}}, {"review_id": "7VRplrynPR1", "reviewer": "Reviewer_4GZM", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper proposes a hidden Markov Transformer (HMT) for simultaneous machine translation (SiMT). The proposed HTM is able to simulate when to start translating and to generate the target token. The results of the proposed method are reported on two offline simultaneous translation datasets.\n\n\n", "summary_of_the_review": "This is an extension of the existing wait-k and causes high latency.\n\n\nThe author's response addressed my concerns, and thank you very much. I have raised the score to 8", "strength_and_weaknesses": "Strength:\n\n1. This method explicitly models when to start translating and generating target words;\n2. Ablation experiments verified the key operations of the proposed HMT;\n3. This study reported that the proposed HMT improves the translation performance on two offline SiMT datasets.\n\nWeaknesses：\n1. The training/inference speeds are lower than those of the Wait-k method, which is a disadvantage for SiMT;\n2. The interaction and relationship between the proposed HMT and the read/write policy were not clearly introduced;\n3. Too many hyperparameters (i.e., L、K、Lamda1, and Lamda2) are not conducive to the optimization of this method on other language pairs;\n4. There lack of necessary experiment to verify what the proposed HMT indeed capture compared to the baseline wait-k.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}]

Author rebuttal:

Decision: Accept: notable-top-25%