Non-autoregressive Streaming Transformer for Simultaneous Translation
Authors: Zhengrui Ma
Shaolei Zhang
Shoutao Guo
Chenze Shao
Min Zhang
Yang Feng
Venue: EMNLP 2023 Main (2023)
Keywords: simultaneous translation
non-autoregressive generation
TL;DR: We introduce a non-autoregressive streaming model for simultaneous translation.

Abstract: Simultaneous machine translation (SiMT) models are trained to strike a balance between latency and translation quality. However, training these models to achieve high quality while maintaining low latency often leads to a tendency for aggressive anticipation. We argue that such issue stems from the autoregressive architecture upon which most existing SiMT models are built. To address those issues, we propose non-autoregressive streaming Transformer (NAST) which comprises a unidirectional encoder and a non-autoregressive decoder with intra-chunk parallelism. We enable NAST to generate the blank token or repetitive tokens to adjust its READ/WRITE strategy flexibly, and train it to maximize the non-monotonic latent alignment with an alignment-based latency loss. Experiments on various SiMT benchmarks demonstrate that NAST outperforms previous strong autoregressive SiMT baselines.

Non-autoregresiive Streaming Transformer for Simultaneous Translation
Zhengrui Ma 1.2 Shaolei Zhang"? 12 Shoutao G Chenze Shao!
Min Zhang", Yang Fengl.2.
Key Laboratory of Intelligent Information Processing
Institute of Computing Technology, Chinese Academy of Sciences University of Chinese Academy of Sciences
SSchool of Future Science and Engineering, Soochow University {mazhengrui21b, Fengyang]@ict.. ac. cn zhangminmtt@hotaail com
Abstract
However, achieving the balance between latency and translation quality non-trivial for SiMT mod- cls. Training these models to produce high-quality translations while maintaining low latency often leads to a tendency for aggressive anticipation (Ma et al., 2019), as the models are compelled to output target tokens even before the corresponding source tokens have been observed during the training stage (Zheng al., 2020). We argue that such an is- sue of anticipation stems from the autoregressive (AR) model architecture upon which most existing SiMT models are built. Regardless the specific READ/WRITE strategy utilized, AR SiMT models are typically trained using maximum likelihood es- timation (MLE) via teacher forcing. As depicted in Figure 1, their training procedure can have ad- verse effects on AR SiMT models in two aspects: 1) non-oonotn problem: The reference used in training might be non-monotonically aligned with the source. However, real-time scenarios, SiMT models are expected to generate translations that align monotonically with the source to reduce latency (He et al., 2015; Chen et al., 2021). The inherent verbatim alignment assumption during the MLE training of AR SiMT models restricts their performance; source-info leakage bias: Follow- ing the practice in full- dull-sentence translation systems, AR SiMT models deploy the teacher forcing strat- egy during training. However, may inadvertently result in the leakage of source information. As il- lustrated in Figure 1, even if the available source content does not contain the word "T (hold)", the AR decoder is still fed with the corresponding translation word "held" as the ground truth context in training. This discrepancy between training and inference encourages the AR SiMT model to make excessively optimistic predictions during the real- time inference, leading to the issue of hallucination (Chen et al., 2021).
Simultaneous machine translation (SiMT) mod- els are trained to strike balance between la- tency and translation quality. However, train- ing these models achieve high quality while maintaining low latency often leads to ten- dency for aggressive anticipation. We argue that such issue stems from the autoregressive architecture upon which most existing SiMT models are built. To address those issues, we propose non-autoregressivv streaming Trans- former (NAST) which comprises a unidirec- tional encoder and a non-autoregressive de- coder with intra-chunk parallelism. We enable NAST to generate the blank token or repeti- tive tokens to adjust its READ/WRITE strat- egy flexibly, and train it to maximize the non- monotonic latent alignment with an alignment- based latency loss. Experiments SiMT benchmarks demonstrate that NAST out- performs previous strong autoregressive SiMT baselines. Source code is publicly available at https:///ithhu: com/ com/ictnlp/NASS
various
1 Introduction
Simultameous machine translation (SiMT; Cho and Esipova, 2016; Gu et al., 2017; Ma et al., 2019; Ari- vazhagan et al., 2019; Zhang and Feng, 2023), also known as real-time machine translation, is com- monly used in various practical scenarios such as live broadcasting, video subtitles and international conferences. SiMT models are required to start translation when the source sentence incomplete, ensuring that listeners stay synchronized with the speaker. Nevertheless, translating partial source content poses significant challenges and increases the risk of translation errors. To this end, SiMT models are trained to strike balance between la- tency and translation quality by dynamically de- termining when to generate tokens (i.e., WRITE action) and when wait for additional source in- formation (i.e., READ action).
To address the aforementioned problems in au- toregressive SiMT models, we focus on developing
"Coresppodding author: Yang Feng

---

Refarence
Bush
held
a
talk
with
Sharon
Monotonic
Bush
and
Sharon held
talk
Output
Unidirectional Encoder
Autoregressive SiMT Decoder
A
foft (Bush) 5 (and)
2 (Sitaron)
SIT (bold)
U0 (balk
<BOS>
Bush
held
talk
with
source-info Teakage
Figure 1: Illustration of the non-monee con-moootonicity problem and the source-info leakage bias in the training of autore- gressive SiMT models. In this case, the AR SiMT model learns to predict at the third time step based on the source contexts "ffíll' (Bush)", " (and)". " (Sharon)", and the ground truth contexts "Bush", "held". Although the source token "# FT (hold)" has not been read yet, it is exposed to the AR SiMT model through its corresponding token "held" in the ground truth context.
SiMT models that generate target tokens in autoregressive (NAR) manner (Gu et al., 2018) by removing the target-side token dependency. We ar- gue that an NAR decoder better suited for stream- ing translation tasks. Firstly, the target tokens are modeled independently in NAR models, which fa- cilitates the development of non-monotonic align- ment algorithm between generation and reference, alleviating the non-monotonicity problem. Addi- tionally, the conditional independence assumption of the NAR structure liberates the model from the need for teacher forcing in training, thereby elimi- nating the risk of source-side information leakage. These advantagcous propertics of the NAR struc- ture enable SiMT models to avoid aggressive antic- ipation and encourage the generation of monotonic translations with fewer reorderings that align with the output of professional human interpreters.
marks demonstrate that NAST outperforms previ- ous strong autoregressiive SiMT baselines.
2 Preliminaries
2.1 Simultaneous Translation
Simultancous machine translation models often adopt a prefix-to-preff framework to start gen- erating translation conditioned partial source input. Given source sentence (I1,-.., previous autoregressive SiMT models factorize the probability of target sentence y {y1,,.., Vo as:
lv II
Po
do vec),
where g(t) is monotonic non-decreasing func- tion of i, denoting the number of observed source tokens when generating yt- A function g(t) rep- resents specific READ/WRITE policy of SiMT models.
In this work, we propose non-autoregressive streaming Transformer (NAST). NAST processes streaming input and performs unidirectional en- coding. Translations are generated in chunk-bi bby chunk manner, with tokens within each chunk be- ing generated in parallel. We enable NAST to gen- erate blank token 6 repetitive tokens to build READ/WRITE paths adaptively, and train to maximize non-monotonic latent alignment (Graves et al., 2006; Shao and Feng, 2022) with a further de- veloped alignment-based latency loss. In this way. NAST effectively learns to generate translations that are properly aligned with the source in a mono- tonic manner, achieving high-quality translation while maintaining low latency.
In addition to translation quality, latency is a cru- cial factor in the assessment of SiMT models. The latency of policy g(t) is commonly measured us- ing Average Lagging (AL; Ma et al., 2019), which counts the number of tokens that the output lags behind the input:
B (o(1)
ALlg:x) (Ial)
Σ
t 1-1),
where ((x))) ) is the cut-off function to exclude the counting of problematic tokens at the end:
([=|)) = min(t (()) = [m])
Extensive experiments on WMT15 German + English and WMT16 English - Romanian bench-
and r |y|
[ represents the length ratio between the
target and source sequences.

---

2.2
Non-autoregrresive Generation
2021) and non-autooregressvv decoder with intra- chunk parallelism. The model architecture is de- picted in Figure When a source token æ; read in, NAST passes it to the unidirectional encoder, allowing it to attend to the previous source contexts through causal encoder self-attention:
2.2.1 Parallel Decoding
Non-autoregressive generation (Gu et al., 2018) is originally introduced to reduce decoding latency'. It removes the autoregressive dependency and gen- erates target tokens in a parallel way. Given a source sentence = {II,... ..., Im J. NAR mod- els factorize the probability of target sentence y "U,,,-,)u)]]
SelfAttm(a.,
Concurrently, NAST upsamples ø; À times and feeds them to construct the decoder hidden states as chunk. Within the chunk, NAST handles X states in fully parallel manner. To further clar- ify, we introduce h represent the sequence of decoder states. Thus, the j-th hidden state in the i-th chunk can be denoted as ((-1))A+ subject to S i |x| and 1 < j S X. Those states can attend to information from all currently observed source contexts through cross-attention:
lvl plun m))
p(y/a)
(4)
2.2.2 Connectionist Temporal Classification Unlike autoregressive models that dynamically con- trol the length by generating the <eos> token, NAR models often utilize a length predictor to pre- determine the length of the output sequence before generation. The predicted length may be imprecise and lacks adaptability for adjustment. Connection- ist Temporal Classification (CTC; Graves et al., 2006) addresses this limitation by extending the output space J' with a blank token The genera- tion E ) is referred to as the alignment. CTC defines a mapping function ((y; T) that returns set of all possible alignments of y of length T and a collapsing function 3 (a) that first collapses all consecutive repeated tokens a and then removes all blanks to obtain the target. During training, CTC marginalizes out all alignments:
Cross.Att(hh--ua (- 1))4: #gg),
and to information from all constructed decoder states through self-attention:
SelfAttn(b hga))
Following CTC (Graves et al., 2006), we extend the vocabulary to allow NAST generating the blank token or repeated tokens from decoder states to model an implicit READ action. We refer to the outputs from a states chunk h 11)1+++1: partial alignments (1-1)A+1: where NAST generates them in a non-autoregressive way:
plyl)
Σ p(a w),
where T is a pre-determined length and the align- ment is modeled in a non-autoregressive way:
pooao (10)
II
p(a x)
I p().))).
(6)
To obtain the translation stream, we first apply the collapsing function 3-1 to deal with the partial alignments generated from the i-th chunk:
3 Approach
We provide detailed introduction the non- autoregressive streaming Transformer (NAST) in this section.
chunks
9 (o(
(11)
Then NAST concatenates the outputs from the cur- rent chunk to generated prefix pre according the following rule:
3.1 Architecture Overview
NAST consists of a unidirectional encoder (Ari- vazhagan et al., 2019; Ma et al., 2019; Miao et al.,
pre pre @B V2. chunk: if " chunk: yro are ① y chink, otherwise
(12)
NNte that the concept of latency differs between NAR generation and SiMT. It refers to the delay in generating all target tokens once all source tokens are observed in the first case and to the level of synchronization between target-side generation and source-side observation in the latter case.
where pre denotes the last token in the generated prefix. Consequently, upon receiving a token in the input stream, NAST is capable to generate 0 to À

---

Bush
held
talk
Sharon
CENCLA
Cissmer
Monotonic
Buch
Bush
and
Shamn
held
beld
talk
Output
Unidirectional Encoder
Non-autoregressive Streaming Decoder
fpft (Bush) (and) 2012 (Sharon) T Gold)
DE (alk))
TEIT
5
Glancing
Bush
held
Figure 2: Overview of the proposed non-autoregressivv streaming Transformer (NAST). Upon receiving source token, NAST upsamples À times and feeds them to the decoder as chunk. NAST can generate blank token e or repetitive tokens (both highlighted in gray) to find reasonable READ/WRITE paths adaptively. We train NAST using the non-monotonic latent alignment loss (Shao and Feng, 2022) with the alignment-bast latency loss to achieve translation of high quality while maintaining low latency.
tokens at a time, endowing it with the ability to ad- just its READ/WRITE strategy flexibly. Formally, each full alignment a E [(; A(a:) can be con- sidered as concatenation of all the partial align- ments, and implies specific READ/WRITE pol- icy generate the reference y. Therefore, NAST jointly models the distribution of translation and READ/WRITE policy by marginalizing out latent alignments:
is infeasible to enumerate all possible I to obtain AL(O; x). This limitation motivates us to delve deeper into AT((f: x)) and devise an efficient esti- mation algorithm.
To simplify the estimation process of AT x) while still excluding the lag counting of problem- atic words generated after all source read in, we deploy a new cut-off function that disregards to- kens generated after all source observed, i.e., to- kens from the last chunk:
p(y xc)
p(alæ)
()
Typ ([=)) ( (t) |z)
(15)
(13)
()
ISS5 ISJAA
Then we introduce a moment function m(i) to de- the number of observed source tokens when generating the i-th position the alignment. Given the fixed upsampling strategy of NAST, is clear that:
3.2 Latency Control
While NAST exhibits the ability to adaptively de- termine an appropriate READ/WRITE. policy, we want to impose some specific requirements on the trade-off between latency and translation quality. To accomplish this, introduce alignment- based latency loss and chunk wait-k strategy to effectively control the latency of NAST.
m((i - 1A = I53SA.
(16)
We further define an indicator function I(a;) to denote whether the i-th position in the alignment is reserved after collapsed by 3-1, With its help. it is convenient to express the lagging of alignment a:
3.2.1 Alignment-based Latency Loss
A
Considering NAST models the distribution of READ/WRITE policy by capturing the distribu- tion of latent alignments, it is desirable to measure the averaged latency of all latent alignments and further regularize it. Specifically, we are interested in the expected Average Lagging (AL: Ma al., 2019) of NAST:
((())))
G ((
((x))
g(t)
((xl)
11
m((i1((a)) - Tea (10) (Tx)) ((Trg((ar))) (|æe) 1),
((=1))
2r
i=1
(1
|xc ((x)))
m())
(|=l))
2
AI, (u
(14)
(17)
Equation 17 inspires us to estimate the expected av- erage lagging AL(0; x)) by separately calculating the expected values of the numerator and denomi-
where 9d the policy induced from alignment a. Due to the exponentially large alignment space,

---

NAST is allowed wait for additional source tokens before initializing the generation of the first chunk. The first chunk fed to the decoder at the moment the (A: + 1)-th source token read in. Subsequently, NAST feeds each following chunk as each new source token is received. The partial alignment generated from each chunk is consis- tently lagged by k tokens compared with the corre- sponding source token until the source sentence is complete.
source
source
decoder position
decoder position
Formally, the moment function for the chunk wait-k strategy can be formulated as:
(a) k =
(b) &
Figure Illustration cross-attention with different chunk wait-k strategies.
(22) As depicted in Figure 3, decoder states can fur- ther access information from additional ls observed source tokens through cross-attention
m((i 1)X min (i k |=]]: IsisA.
nator:
AL(0;z)
CrossAtta(hs 1)A4- (1+
(23)
E. (1 1)a (a:))
7 (En To ([x])
-1)
En 7ym [[ar)]
which leads NAST prioritize better translation quality at the expense of longer delay.
(18)
It relieves from the intractable task of enumer- ating gd Instead, we only need handle two expectation terms: Ea Ea [S (1a|-11) m()) and Ba [T ([=:)),, which can be resolved efficiently."
3.3 Non-monotonic Latent Alignments
While CTC loss (Graves et al., 2006) provides the convenience of directly applying the maxi- mum likelihood estimation to train NAST, i.e.. log p(y((()) it only considers the mono- tonic mapping from target positions to alignment positions. However, non-monotonic alignments are crucial in simultaneous translation. SiMT mod- cls are expected to generate translations that are monotonically aligned with the source sentence to achieve low latency. Unfortunately, the train- ing corpus, source and reference pairs are often non-monon non-monottmically aligned due to differences in grammar structures between languages (e.g., SVO vs SOV). Neglecting the non-monotonic mapping during training compels the model to predict to- kens for which the corresponding source has not been read, resulting in vve-aantiiipation. To ad- dress these issues, we apply the bigram-based non- monotonic latent alignment loss (Shao and Feng, 2022) to train our NAST, which maximizes the F1 score of expected bigram matching between target and alignments:
E. Tap [=]] = E p(1(a.)) E. D (1)-- (a)) = - m(1)((((a)) (a:))
C =
(19)
where p( (a:)) represents the probability that the i-th token in the alignment reserved after collaps- ing and can be calculated simply as:
((1()))) = 1- p(a: =)- Σ p(a: = v)poo- eeV/c
(20)
With the assistance of the aforementioned deriva- tion, it efficient estimate the expected average lagging of NAST. By applying along with a tun- able minimum lagging threshold 4mins we can train NAST to meet specific requirements of low latency:
Listency (a8 a)) bmmn).
(21)
3.2.2 Chunk Wait-k Strategy
In addition to the desiring property of shorter lag- ging, there may be practical scenarios where aim mitigate the risk of erroneous translations by increasing the latency. To this end, we propose a chunk wait-k strategy for NAST to satisfy the requirements of better translation quality.
CNMIA (0) 2. 2. min(Co(v). Excoy (C.()) C,())
(24)
where Cy(v) denotes the occurrence count of bi- gram 9 = (01- (9))92) in the target, (t)) represents the expected count of for NAST, and G2 denotes the set of all bigrams in y.
Wo leave the detailed derivation of Equation 19 in Ap- pendix A.

---

3.4 Glancing
as the validation set and newstest-2016 as the test set. For each dataset, we apply BPE (Sennrich et al., 2016) with 32k merge operations to learn a joint subword vocabulary shared across source and target languages.
Due to its inherent conditional independence struc- ture, NAST may encounter challenges related to the multimodality problem² (Gu et al., 2018). To address this issue, we employ the glancing strategy (Qian et al., 2021) during training. This involves randomly replacing tokens in the decoder's input chunk with tokens from the most probable latent alignment. Formally, the glancing alignment the one that maximizes the posterior probability:
Implementation Details We select chunk up- sample ratio of (A =3) and adjust the chunk waiting parameter l and the threshold Imin in alignment-based latency loss to achieve varying quality-latency trade-offs.. 6 For the first stage of training, we set the dropout rate to 0.3, weight decay to 0.01, and apply label smoothing with a value of 0.01. We train NAST for 300k updates on De and 100k updates on En- A batch size of 64k tokens is utilized, and the learning rate warms up 10-4 within 10k steps. The glanc- ing ratio linearly anneals from 0.5 0.3 within 200k steps on Dn and 100k steps on En En-Roo In the second stage, we apply the latency loss only if the chunk wait strategy is disabled (k: = 0). The dropout rate is adjusted to 0.1 for De->En, while no label smoothing is applied to either task. We further train NAST for 10k updates De and 6k up- dates En->Ro A batch size of 256k tokens is uti- lized to stabilize the gradients, and the learning rate warms up to 10-4 within 500 steps. The glanc- ing ratio is fixed at 0.3. During both training stages, all models are optimized using Adam (Kingma and Ba, 2014) with 8 = (0.9, 0.98) and = 10-8 Following the practice in previous research on non- autoregressive generation, we employ sequence- level knowledge distillation (Kim and Rush, 2016) to reduce the target-side dependency in data. We adopt Transformer- (Vaswani al., 2017) as the offline teacher model and train NAST on the distilled data.
arg max p() aed(yrAppe))
(25)
Then we randomly sample some positions in the decoder input and replace tokens the input se- quence with tokens from the glancing alignment sequence at those positions in training.
3.5 Training Strategy
In order to better train the NAST model to adapt to simultancous translation tasks with different la- tency requirements, propose two-stage train- ing strategy. In the first stage, we train NAST using the CTC loss obtain the reference monotonic- aligned translation with adaptive latency:
Latage-1 Lere logl(.)).).
(26)
the second stage, we train NAST using the com-
bination of the non-monotonic latent alignment loss and the alignment-basse based latency loss:
Lxtage-2 LNMIL + Liatency.
(27)
This further enables NAST to generate translations that are aligned with the source in a monotonic manner, meeting specific latency requirements.
4 Experiments
4.1 Experimental Setup
Baselines We compare system with the follow- ing strong autoregressive SiMT baselines:
Datasets We conduct experiments on the following benchmarks that are widely used in previous SiMT studies: 5MT15* German -> English (De->B -En, 4.5M pairs) and WMT16 English + Romanian (En->Ro, 0.6M pairs). For De->En, we use new- stest2013 as the validation set and newstest2015 newsdev-2016
Offline AT Transformer model (Vaswani et al., 2017), which initiates translation after reading all the source tokens. We utilize a unidirectional en- coder and employ greedy search decoding for fair comparison.
the test set. For En Ro,
FFurterr details regarding the settings k and Iain can found in Appendix B.
'The maltimodality problem arises when source sentence has multiple possible translations, which oon-euvtreerressvve system unable capture due to its inability model the target dependency.
"Note that the purpose of offline knowledge distillation is to reduce the dependency between target-side tokens in the data, in order to facilitate the learning of non-autoregressive models. This is diffcrent from the goal of performing mono- tonie knowledge distillation in the field of SiMT, which aims to obtain monotonic aligned data.
"ht //www. statmt. .org org/omtt//t. /w////m15 15/ Phttps: //www//wwww///t/// statnt...tmm hrg/wmt16//

---

32
30
30
26
24
Offiine AT
OcTIbe AT
22
NAST
NAST
20
MrC. Wittk
MOE Wor
MMA
MMA
20
Average Lagging (A1.)
Averuge Lagging (A1.
(a) De->En
(h) En-Ro
Figure Results translation quality (BLEU) against latency (Average Lagging) on De-EE and En-+Ro.
Wait-k Wait-k: policy (Ma et al., 2019), which initially reads & tokens and subsequently alternates between WRITE and READ actions.
k:
3
5
7
NAST w/o LNMLA
30.69
31.58
31.70
31,94
BLEU
28.84
29.72
30.12
30.68
MoE Wait-k Mixture-of-expers wait-k policy (Zhang and Feng, 2021), which involves employing multiple experts to learn multiple wait-k policies during training. MoE Wait-k is the current SOTA fixed policy.
A
1.85
1.86
1.58
1.26
Table 1: Results of BLEU scores on De-En test set with or without LNMLA under different chunk wait-k strategy. Llatency is not applied here.
MMA Monotonic multi-head attention (MMA; Ma et al., 2020) employs Bernoulli variable to predict the READ/WRITE action and is trained using monotonic attention (Raffel et al., 2017).
NAST w/o LNMLA
4.02
5.83
6.85
8.44
AL
3.42
5.11
6.56
8.20
HMT Hidden Markov Transformer (HM'; Zhang and Feng, 2023), which treats the moments of starting translating hidden events and con- siders the target sequence as the observed events. This approach organizes them as hidden Markov model. HMT current SOTA adaptive policy.
Table 2: Results of Average Lagging on De-+En test set with without LNMLA under different chunk wait-k strategy. Llatency is not applied here.
demonstrates superior performance compared the offline AT system even when the AL as low as 6.85, showcasing its competitiveness in scenar- ios where higher translation quality is desired. On En -Ro, NAST also exhibits substantial improve- ment under low latency conditions. On the other hand, NAST achieves comparable performance other models on En-RRo when the latency require- ment is not stringent.
Metrics To compare SiMT models, we evaluate the translation quality using BLEU score (Papineni al., 2002) and measure the latency using Aver- age Lagging (AL: Ma al., 2019). Numerical results with more latency metrics can be found in Appendix B.
4.2
Main Results
Analysis
We compare NAST with the existing AR SiMT methods in Figure 4. On De-+En, NAST outper- forms all AR SiMT models significantly across all latency settings, particularly in scenarios with very low latency. With the latency in the range of [o, 1). where listeners almost synchronized with the speaker, NAST achieves translation quality of 27.73 BLEU, surpassing the current SOTA model HMT by nearly BLEU points. Moreover, NAST
5.1 Importance Non-monotonic Alignments NAST is trained using non-monotonic alignment loss, enabling it to generate source monotonic- aligned translations akin human interpreters. This capability empowers NAST to achieve high- quality streaming translations while maintaining low latency. To validate the effectiveness of non- monotonic alignment, we conduct further experi-

---

NAST --- IMTT
8.5
8.0
7.5
7.0
BLFF
28
6.5
NAST-aay
6.0-
HMT--ay
5.5
NAST mediun
HMT-asiiu
5.0
22
NAST hard
HMThad
4.5
10
10
Average Lagging
Average Lagging
Figure 5: Results hallucination rate against latency (Average Lagging) on De De-+En test set.
Figure 6: Performance on De->En test subsets catego- rized by difficulty.
ments by studying the performance of NAST with- out LNMLA- We compare the translation quality (BLEU) and latency (AL) of models employing different chunk wait-k strategies. The results are reported in Table and Table Note that Llatency is not applied here for clear comparison.
5.3 Performance across Difficulty Levels
To further illustrate NAST's effectiveness in han- dling non-monotonicity problem, we investigate its performance when confronted with samples of varying difficulty levels. It is intuitive to expect that samples with higher number of eross align- ments between the source and reference texts pose greater challenge for real-time translation. There- fore, we evenly partition the De-*En test set into subsets based on the number of crosses in the align- ments, categorizing them as easy, medium, and hard, in accordance with the approach by Zhang and Feng (2021). We compare our NAST with pre- vious HMT model, and the results are presented in Figure 6.
We observe that incorporating CNMLA signifi- cantly enhances translation quality by up to 1.86 BLEU, while maintaining nearly unchanged la- tency. We also notice that the improvement is particularly substantial when the latency is low, which aligns with our motivation. Under low la- tency conditions, SiMT models face more severe non-monotonicity problems. The ideal simulta- neous generation requires more reordering of the reference to achieve source sentence monotonic alignment, which leads to greater improvements of applying non- mon-monoto alignment loss.
Despite the impressive performance of NAST, a closer examination of Figure reveals that the superiority is particular on the challenging subset. Even when real-time requirements are relatively relaxed, the improvement in handling the hard sub- remains noteworthy. We attribute this the stringent demand imposed by the hard subset, re- quiring SiMT models to effectively manage word reorderings to handle the non-mon non-monotonicity. NAST benefits from non-monotonic alignment training
5.2 Analysis on Hallucination Rate
NAST mitigates the risk of source information leak- age during training, thereby minimizing the occur- rence of ballucination during inference. To demon- strate this, we compare the hallucination rate (Chen al., 2021) of hypotheses generated by NAST with that of the current SOTA model, HMT (Zhang and Feng, 2023). A hallucination is defined as a gen- erated token that can not aligned to any source word. The results are plotted in Figure 5.
and excels in addressing these challenges, thus en- hancing its performance in handling those harder samples.
We note that the hallucination rates of both mod- els decrease as the latency increases. However, NAST exhibits significantly lower hallucination compared HMT. We attribute this the fact that NAST avoids the bias caused by source-info leakage and enables a more general generation- reference alignment, thus mitigating compelled pre- dictions during training.
5.4 Concerns on Fluency
While the non-autorcgressive nature endows NAST with the capability tackle the non-monotonieity problem and source-info leakage bias, it also ex- poses NAST the risk of potential fluency degra- dation due to the absence of target-side dependency. To have a better understanding of this problem, we

---

READ/WRITE action. Ma al. (2020) proposed MMA implement MILK on Transformer. Liu et al. (2021) introduced CAAT, which leverages RNN-T and employs a blank token to signify the READ action. Miao et al. (2021) proposed GSiMT to generate the READ/WRITE actions. Chang et al. (2022) proposed to train a casual CTC encoder with Gumbel-Sinkhonn network (Mena et al., 2018) reorder the states. Zhang and Feng (2023) pro- posed HMT to learn when start translating in the form HMM, achieving the current state-of-the- art SiMT performance.
Eetrnna NAST
350
800
Taplasity
150
Average Lagging (NL)
Figure 7: Results of fluency (Perplexity) against latency (Average Lagging) on De test set.
NAR Generation Non- autoregressivve models gen- erate tokens parallel to the sacrifice of target-side dependency (Gu al.. 2018). This property elim- inates the need for teacher forcing, motivating re- searchers to explore flexible training objectives that alleviate striet position-wise alignment imposed by the naive MLE loss. Libovický and Helel (2018) proposed latent alignment model with CTC loss (Graves et al., 2006), and Shao and Feng (2022) further explored non-monotonic latent alignments. Shao et al. (2020, 2021) introduced sequence-level training objectives with reinforcement learning and bag-of-ngrams difference. Ghazvininejad et al. (2020) trained NAT model using the best mono- tonic alignment and Du et al. (2021) further ex- tended it to order-agnostic cross-entropy loss. In addition, some researchers are focusing on strength- ening the expression power to capture the token dependency. Huang al. (2022) proposed directed acyclic graph layer and Gui et al. (2023) introduced probabilistic context-free grammar layer. Build- ing upon that, Shao et al. (2022) proposed Viterbi decoding and Ma et al. (2023) further explored fuzzy alignment training, achieving the current state-of-the -he- art NAR model performance. Apart from text translation, the NAR model also demon- strated impressive performance in diverse areas such as speech-to-text translation (Xu et al., 2023), translation (Fang et al., 2023) and text-to-speech synthesis (Ren al., 2021).
evaluate the fluency of the De->En test set out- put from NAST in comparison to previous HMT. Specifically, we employ the Perplexity value re- ported by an external pre-trained language model transformer_Iis wmt19. 19. en to measure the flu- ency of the generated texts. lower Perplexity value implies more fluent translations. The results are presented in Figure 7.
Though NAST exhibits significantly improved translation quality, we find its non-autoregressive nature does impact fluency to some extent. How- ever, we consider this trade-off acceptable. In prac- tical scenarios like international conferences where SiMT models are employed, the language used by human speakers is often not perfectly fluent. In such contexts, the audience tends to prioritize the overall translation quality under low latency, rather than the fluency of generated sentences.
6 Related Work
SiMT Simultaneous machine translation requires a READ/WRITE policy to balance latency and trans- lation quality, involving fixed and adaptive strate- gies. For the fixed policy, Ma et al. (2019) proposed wait-k, which first reads k source tokens and then alternates between READ/WRITE action. Elbayad et al. (2020) introduced an efficient training method for the wait-k policy, which randomly samples A during training. Zhang and Feng (2021) proposed a mixture--ofeepprrt w learn set of wait-k policies through multiple experts. For the adap- tive policy, Gu et al. (2017) trained an agent to decide READ/WRITE via reinforcement learning. Arivazhagan et al. (2019) introduced MILk, which incorporates Bernoulli variable to indicate the ness.
7 Conclusion
In this paper, we propose non-autoregressive streaming Transformer (NAST) to address the non- monotonicity problem and the source-info leakage bias in existing autoregressive SiMT models. Com- prehensive experiments demonstrate its effective-
Phttps://gitee http:/gghhhb.coeeeaaooooeeeeeeee fairseq/ tree/maain/eeaaple///aagggggg

---

Limitations
Maha Elbayad, Laurent Besacier, and Jakob Verbeek. 2020. Efficient wait-k models for simultaneous ma- chine translation.
We have observed that the performance of NAST is less satisfactory when translating from English to Romanian (En->Ro) compared to translating from German to English (De+EEn). This can be attributed to the fact that Romanian shares the SVO (Subject-Verbhe Verb-Object) grammar with English, while German follows an SOV (Subject-Object- Verb) word order. NAST excels in handling word reordering in translating from SOV to SVO, es- pecially there is a strict requirement for low la- tency. But it is relatively less effective in SVO-to- SVO translation scenarios where there is typically monotonic alignment between the source and ref- erence.
Qingkai Fang, Yan Zhou, and Yang Feng. 2023. Daspeech: Directed acyclic transformer for fast and high-quality speech-to-speed translation. In Ad- vances in Neural Information Processing Systems.
Marjan Ghazvininejad, Vladimir Karpukhin, Luke Zettlemoyer. and Omer Levy. 2020. Aligned cross entropy for non-autoreereesive machine translation. In Proceedings of the 37th International Conference on Machine Learning, volume 119 of Proceedings of Machine Learning Research, pages 3515-3522 33233. PMLR.
excels
Alex Graves, Santiago Fernández, Faustino Gomez, and Jürgen Schmidhuber: 2006. Connectionist temporal classification: labelling unsegmented sequence data with recurrent neural networks. In Proceedings of the 23rd international conference Machine learning. pages 369-376.
Acknowledgements
We thank the anonymous reviewers for their in-
sightful comments.
Jiatao Gu, James Bradbury, Caiming Xiong, Victor O.K. Li, and Richard Socher. 2018. Non-autoregressive neural machine translation. In International Confer- ence on Learning Representations.
References
Naveen Arivazhagan, Colin Cherry, Wolfgang Macherey, Chung-Cheng Chiu, Semih Yavuz, Ruom- ing Pang, Wei Li, and Colin Raffel. 2019. Monotonic infinite lookback attention for simultaneous machine translation. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 1323-1323, Florence, Italy. Association for Computational Linguisties.
Jiatao Gu, Graham Neubig, Kyunghyun Cho, and Vic- tor O.K. Li. 2017. Learning to translate in real-time with neural machine translation. In Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 1. Long Pupers, pages 1053-1062, Valencia, Spain. Association for Computational Linguisties.
Shangtong Gui, Chenze Shao, Zhengrui Ma. Xishan Zhang, Yunji Chen, and Yang Feng. 2023. Non- autoregressive machine translation with probabilistic context-free grammar. In Advances in Neural Infor- mation Processing Systems.
Chih-Chiang Chang, Shun-Po Chuang, and Hung-yi Lec. 2022. Anticipation-free training for simulta- neous machine translation. In Proceedings of the 19th International Conference on Spoken Language Translation (IWSLT 2022), pages 43-61, Dublin, Ire- land (in- nu-persoon and online). Association for Compu- tational Linguistics.
He He, Alvin Grissom II, John Morgan, Jordan Boyd- Graber, and Hal Daumé III. 2015. Syntax-based rewriting for simultancous machine translation. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 55- 64, Lisbon, Portugal. Association for Computational Linguistics.
Junkun Chen, Renjie Zheng, Atsuhito Kita, Mingbo Ma, and Liang Huang. 2021. Improving simultane- ous translation by incorporating pseudo-references with fewer reorderings. In Proceedings of the 2021 Conference on Empirical Methods in Natural Lan- guage Processing, pages 5857-5864, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics.
Fei Huang. Hao Zhou, Yang Liu, Hang Li, and Minlie Huang. 2022. Directed acyclic transformer for non- autoregressive machine translation. In Proceedings of the 39th International Conference on Machine Learning, ICMI 2022.
Kyunghyun Cho and Masha Esipova. 2016. Can neu- ral machine translation do simultancous translation? CoRR, abs/1606.02000
Yoon Kim and Alexander M Rush. 2016. Sequence- level knowledge distillation. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 1317-1327.
Cunxiao Du, Zhaopeng Tu, and Jing Jiang. 2021. Order- agmostic cross entropy for non- autoregressive ma- chine translation. In Proceedings of the 38th Inter- national Conference Machine Learning, volume 139 of Proceedings of Machine Learning Research, pages 2849-2859, PMLR.
Diederik P Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. arXiv preprint 1411v:1412.69880

---

Jindfich Libovický and Jindfich Helel. 2018. End-to- end non- oon-auorregresive neural machine translation with connectionist temporal classification. In Pro- ceedings of the 2018 Conference on Empirical Meth- ods in Natural Language Processing, pages 3016- 3021, Brussels, Belgium. Association for Computa- tional Linguisties.
Colin Raffel, Minh- Thang Luong, Peter J. Liu, Ron Weiss, and Douglas Eck. 2017. Online and linear- time attention by enforcing monotonic alignments. In Proceedings of the 34th International Conference on Machine Learning, volume 70 of Proceedings of Machine Learning Reseurch, pages 2837-2846. PMIR.
Dan Liu, Mengge Du, Xiaoxi Li, Ya Li, and Enhong Chen. 2021. Cross attention augmented transducer networks for simultaneous translation. In Proceed- ings of the 2021 Conference Empirical Methods in Natural Language Processing, pages 19-55, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics.
Yi Ren, Chenxu Hu, Xu Tan, Tao Qin, Sheng Zhao. Zhou Zhao, and Tie Yan Liu. 2021. Fastspeech 2: Fast and high-quality end-to-end text to speech. In International Conference on Learning Representa- tions.
Rico Sennrich, Barry Haddow, and Alexandra Birch. 2016. Neural machine translation of rare words with subword units. In Proceedings of the 54th Annual Meeting of the Association for Computational Lin- guistics (Volume 1: Long Papers), pages 1715-1725. Berlin, Germany. Association for Computational Lin- guistics.
Mingbo Ma. Liang Huang, Hao Xiong, Renjie Zheng. Kaibo Liu, Baigong Zheng, Chuanqiang Zhang, Zhongjun He, Hairong Liu, Xing Li, Hua Wu, and Haifeng Wang. 2019. STACL: Simultancous trans- lation with implicit anticipation and controllable la- tency using prefix-to-preff framework. In Proceed- ings of the 57th Annual Meeting of the Association for Computational Linguisties, pages 3025-3033 Flo- rence, Italy, Association for Computational Linguis- tics.
Chenze Shao and Yang Feng. 2022. Non-monotonic la- tent alignments for etc-based non-autoregressive ma- chine translation. In Advances in Neural Information Processing Systems, volume 35, pages 8159 8173. Curran Associates, Inc.
Xutai Ma, Juan Miguel Pino, James Cross, Liezl Puzon, and Jiatao Gu. 2020. Monotonic multihead attention. In International Conference on Learning Representa- tions.
Chenze Shao, Yang Feng. Jinchao Zhang. Fandong Meng, and Jie Zhou. 2021. Sequence-Leve Training for Non-Autoregressive Neural Machine Translation. Computational Linguisties, 47(4).891-95) -925.
Zhengrui Ma. Chenze Shao, Shangtong Gui, Min Zhang. and Yang Feng. 2023. Fuzzy alignments in directed acyclic graph for con-auttreeressvvee machine trans- lation. In The Eleventh International Conference on Learning Representations.
Chenze Shao, Zhengrui Ma, and Yang Feng. 2022. Viterbi decoding of directed acyclic transformer for non-autoregressive machine translation. In Findings of the Association for Computational Linguisties: EMNLP 2022, pages 4390-4397, Abu Dhabi, United Arab Emirates. Association for Computational Lin- guistics.
Gonzalo Mena, David Belanger, Scott Linderman, and Jasper Snoek. 2018. Learning latent permutations with gumbel-sinkhon networks. In International Conference on Learning Representations.
Chenze Shao, Jinchao Zhang. Yang Feng, Fandong Meng, and Jie Zhou. 2020. Minimizing the bag- of-ngrams difference for non-autoregressive neural machine translation. Proceedings of the AAAI Con- ference on Artificial Intelligence, 34(01):198-20)
Yishu Miao, Phil Blunsom, and Lucia Specia. 2021. A generative framework for simultancous machine translation. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Process- ing, pages 6697-6706, Online and Punta Cana, Do- minican Republic. Association for Computational Linguistics.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. Advances in neural information processing systems, 30.
Kishore Papineni, Salim Roukos, Todd Ward, and Wei- Jing Zhu. 2002. Bleu: method for automatic evalu- ation of machine translation. In Proceedings of the 40th annual meeting of the Association for Computa- tional Linguisties, pages 311-318.
Chen Xu, Xiaoqian Liu, Xiaowen Liu, Qingxuan Sun, Yuhao Zhang, Murun Yang. Qianqian Dong, Tom Ko, Mingxuan Wang. Tong Xiao, Anxiang Ma, and Jingho Zhu. 2023. CTC-based auooerevreeee speech translation. In Proceedings of the 61st An- nual Meeting of the Association for Computational Linguistics (Volume I: Long Pupers), pages 13321- 13339, Toronto, Canada. Association for Computa- tional Linguistics.
Lihua Qian, Hao Zhou, Yu Bao, Mingxuan Wang, Lin Qiu, Weinan Zhang, Yong Yu, and Lei Li. 2021. Glancing transformer for non-autoregressive neural machine translation. In Proceedings of the 59th An- nual Meeting of the Association for Computational Linguistics and the 1 1th International Joint Confer- ence Natural Language Processing (Volume I: Long Papers), pages 1993-2003, Online, Association for Computational Linguistics.
Shaolei Zhang and Yang Feng. 2021. Universal simul- taneous machine translation with mixture

---

wait-k policy. Proceedings the 2021 Confer- ence on Empirical Methods in Natural Language Pro- cessing, pages 7306-7317, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics.
Shaolei Zhang and Yang Feng. 2023. Hidden markov transformer for simultaneous machine translation. In The Eleventh International Conference on Learning Representations.
Baigong Zheng, Kaibo Liu, Renjie Zheng, Mingbo Ma, Hairong Liu, and Liang Huang. 2020. Simultane- ous translation policies: From fixed adaptive. In Proceedings of the 58th Annual Meeting of the Asso- ciation for Computational Linguistics, pages 2847- 2853, Online. Association for Computational Lin- guistics.

---

A Derivation of Equation 19
Table 4: Numerical results of NAST on En-Ro. "-"
indicates that Quteacyy is not applied.
We present the detailed derivation of Equation 19 this section.
WMT16 En Ro
Imin
CW
AP
AL
DAL
BLEU
((al-11A
En To (|2:)] = Σ p(aaa)
Σ (aa))
0
1.41
0.50
0.36
1.77
24.79
0
1,46
0.55
1.58
3.24
26.30
(101-1)
3
1.54
0.65
3.90
5.34
30.01
Σ p(a)))
(28)
1.81
0.72
5.89
7.25
30.98
(1a1-1)A
2.24
0.77
7.85
9.14
31.30
p(1(a:)),
C Case Study
((a)-1)
En
Σ
(a:)
Σ p(aaa)
Σ (())
To gain further insights into NAST's behavior, we examine the generation processes of two different cases within the De->En test set. We visualize the generation by plotting the generated partial align- ments and the collapsed outputs at each step.
(lm)
Σ m(1)
E (aa)
(Im)
1JA
(a.))
die Premlermmister Intless apans Lalen sith Tooloo </5>
(29)
Prime Ministac lodia and Lanan met Toloto
where p( [ (a:)) denotes the probability that the i-th token in the alignment is reserved after collapsing.
Reference
India and Japan prime ministers Tokyu.
Step
Inputs
Allgmments
Outputs
Ute
die
the
the
B Numerical Results with More Metrics
the
Premierminister
the the Indiens prime minlsbbrs ot of India India and
addition to Average Lagging (AL; Ma et al., 2019), we also incorporate Consecutive Wait (CW; Gu et al., 2017), Average Proportion (AP; Cho and Esipova, 2016), and Differentiable Average Lag- ging (DAL; Arivazhagan et al., 2019) as metrics to evaluate the latency of NAST.
prtize
ministers
of India
und
Japans
and
We adjust min Llatency and k in chunk wait-k strategy to achieve varying quality-latenccy trade- offs. For clarity, we present the numerical results
tra@
and
Japan
lpaan
7
fun
Jepan
lapan
NAST using specific hyperparameter settings in Table 3 and Table 4. Note that Llatency is applied to achieve lower latency, while the chunk wait-k strategy is employed to improve translation quality. Therefore, apply Liatency only when k = 0.
8
sich
Jopan
oppan
opan
in
Jopan
Topan
<alank>>>
10
Toke @
z08t
met
Inet
200t
11
to
<blank>
in
Table 3: Numerical results of NAST on De-+En. n."
In
indicates that Listency not applied.
12
in
Tokyo
Tokyo
WMT15 De
>En
Tokyy
13


</S>>
Umin
h
CW
AP
AL
DAL
BLEU
0
0
1,44
0.52
0.65
1.96
27.73
Figure 8: Case study of #0 in De-+En test set, where we configure NAST with Imin and k 0.
0
1.51
0.57
1.87
3.24
29.82
3
0
1.60
0.62
2.97
4.60
30.46
0
1.74
0.66
4.02
5.89
30.69
In Figure 8, we illustrate a in which NAST reorders words at the phrase-level compared the reference. With the streaming input "die Premier- minister Indiens und Japans", NAST produces "the prime ministers of India and Japan" instead of
3
2.03
0.72
5.83
7.64
31.58
5
2.18
0.75
6,85
8.39
31.70
7
2.59
0.79
8.44
9.88
31.94

---

the reference "India and Japan prime ministers". This output represents source-monen monotonic-allgneen phrase, thereby effectively reducing latency.
ausgshon,
Source
seens Like
as no one could assume salee
to
Reference
n0
they
Step
Inpots
Allgmments
Outputs
<Nlane>>
<blank>
<blankes
<blanr>>
2
siehl
<blank>
<blank>>
<blani>>>
3
4
80S
5
6
als
looks
Jooks
7
könne
8
nismand
davon
musgehen
039
one
12
sicher
call
can
13
zu
expect
expeot
ped.
14
sein
to
15
sa2e
saže
16
</3>
Figure 9: Case study of #1083 Dn test set, where we configure NAST with 0 and Liatency not ap- plied.
In Figure 9, depict another generation where NAST manages word reorderings at the sen- tence level in comparison the reference. In order to ensure low latency, NAST adjusts the sentence structure while maintaining meaning consistency with the reference. When NAST processes the source words "es sieht so au", it promptly gener- ates "it looks as if" and continues generating the subsequent words within this grammatical struc- ture. This ensures listeners keep synchronized with the speaker.


Reviews:
[{"review_id": "q2URWtpYNF", "reviewer": "Reviewer_N9PX", "summary": "", "questions": "see \"Reasons To Reject\"", "limitations": "", "rating": 3, "confidence": 2, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your constructive comments. We provide discussions and explanations about your concerns as follows.\n\n> Could the author clarify on the novelty of the proposed method?\n\nIn this paper, we introduce a non-autoregressive streaming model to address the \"non-monotonicity problem\" and \"source-info leakage\" issues in previous AR SiMT models. However, designing a non-autoregressive model in the streaming scenario is not trivial. Our main contributions are concentrated on:\n\n1. We design a non-autoregressive model architecture for streaming generation task.\n\n2. We propose a novel alignment-based latency loss and a chunk wait-k strategy to tailor our non-autoregressive streaming model to meet various requirements of latency and quality effectively. Actually, learning when to start translating tokens to balance latency and quality is the core research problem for SiMT models.\n\n3. We apply the non-monotonic latent alignment loss from the literature to tackle the non-monotonicity problem in SiMT.\n\n> Translation quality, in the current research field, should not be measured by traditional metrics. Maybe the authors could show that NAT does not affect the fluency, coherence of the translated sentences in the SiMT scenario? One tool one could use is GPT-4 which is often used as a scoring function for machine generated contents.\n\nWe appreciate your suggestions regarding the evaluation metrics. We would like to make the discussion about the metrics in the following:\n\n1.We firstly provide an explanation for our choice. In the literature of SiMT, most studies employ the AL-BLEU curve to showcase models' performance under various latency requirements. Hence, in order to ensure a fair and consistent comparison, our research continues with this evaluation metric.\n\n2.To address your concerns about the fluency and coherence of non-autoregressive generation, we assess the fluency of the output on De -> En from our NAST and the previous state-of-the-art AR SiMT model, HMT. Specifically, we use the PPL value reported  by an external pre-trained language model to quantify the fluency of generation. A lower PPL value implies more fluent translations. The results are presented in the table below.\n\n|Reference| |\n|---| ---|\n|PPL | 182.37  |\n\n|NAST | | | | | | | |\n|---| ----------- | ----------- | ----------- | -----------  | ----------- | ----------- |----------- |\n|AL| 0.65\t| 1.87 | 2.97 |\t4.02 |\t5.83 |\t6.85 |\t8.44 |\n|BLEU ↑| 27.73 | \t29.82\t| 30.46 | \t30.69\t| \t31.58 | 31.70 | 31.94\t|\n|PPL ↓| 384.3 |\t361.19 |\t308.88\t |296.06 |\t272.88 |\t272.2 |\t273.15|\n\n|HMT | | | | | | | |\n|---| ----------- | ----------- | ----------- | -----------  | ----------- | ----------- |----------- |\n|AL| 0.27 |\t2.20\t| 3.46\t|4.74|\t6.43|\t8.36|\t10.06|\n|BLEU ↑| 22.52 | 27.60 |29.29 |30.29 |30.90|31.45 | 31.61 |\n|PPL ↓| 276.43 |263.84\t|245.62|\t264.99 |\t272.07 |\t251.23 |\t229.11 |\n\nWe find that while NAST exhibits significantly improved translation quality compared to previous AR SiMT models, its non-autoregressive nature does impact fluency to some extent. However, we consider this trade-off acceptable. In practical scenarios like international conferences where SiMT models are employed, the language used by human speakers is often not perfectly fluent. In such contexts, the audience tends to prioritize the overall translation quality of the SiMT model under low latency, rather than the fluency of the generated sentence.\n\n3.As far as we know, when GPT-4 is used for evaluation, it often assesses the relative quality of two outputs in a comparative manner. It is challenging to provide an absolute value of its quality, and the results are significantly influenced by the prompt used. When evaluating SiMT models, we require a metric with absolute values to assess the quality under different latencies among various SiMT models. Therefore, it becomes very hard to employ large language models to evaluate the performance of SiMT models.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "(1) This paper is well motivated. it starts with the issues in SiMT, and propose its framework to address the issues. \n\n(2) Well organized paper. The method is clearly stated in detail, and the experimental results are presented with visualization, which is easy to follow. \n\n(3) This paper proposes to use NAT decoder to alleviate the source-info leakage bias of SiMT models with autoregressive models. \n\n", "reasons_to_reject": "(1) i am not sure whether the idea of utilizing NAT to SiMT could garantee enough contributions to be accepted. The loss function is from the literature, and the core of this paper is to use chunks to adaptively model the READ/WRITE strategy. Could the author clarify on the novelty of the proposed method?\n\n(2) Translation quality, in the current research field, should not be measured by traditional metrics. Maybe the authors could show that NAT does not affect the fluency, coherence of the translated sentences in the SiMT scenario? One tool one could use is GPT-4 which is often used as a scoring function for machine generated contents. ", "questions_to_authors": "see \"Reasons To Reject\"", "paper_topic_and_main_contributions": "This paper is motivated by the two issues in simultaneous machine translation (SiMT): (1) non-monotonicity problem in the source- reference mis-alignment; (2) source-info leakage bias, which encourage the model to be optimistic, leading to the issue of hallucination. \n\nThus this paper propose the non-autoregressive streaming Transformer (NAST), which utilize a non-autoregressive decoder  with intra-chunk parallelism, and this model is trained with a alignment based latency loss. ", "excitement": 3, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "eEOSVqQ6Ag", "reviewer": "Reviewer_yohv", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 5, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your constructive comments. We provide discussions and explanations about your concerns as follows.\n\n> More analysis on the model behavior, such as case study will help readers better understand the model. \n\nWe appreciate your suggestions regarding the case study. Following your advice, we will incorporate a case study section in the revised version of our paper. Moreover, to provide a more comprehensive analysis of our model, we partition the De -> En test set into subsets based on the number of crosses in the alignments between the source and reference, categorizing them as easy, medium, and hard. Intuitively, a higher number of cross alignments pose greater challenges for a SiMT model. We compare our NAST with previous state-of-the-art AR SiMT model, HMT. The results are outlined below:\n\n|NAST| | | | | | | |\n| --- | --- | --- | --- | --- | ---| ---| ---|\n|AL | 0.65 |\t1.87 |\t2.97 |\t4.02 |\t5.83 |\t6.85 |\t8.44 |\n|Easy |\t27.19 |\t32.11 |\t34.43 |\t34.48 |\t35.21 |\t35.51 |\t35.51|\n|Medium |\t27.54 |\t30.23 |\t30.91 |\t30.90 |\t31.72 |\t31.81 |\t32.09|\n|Hard |\t28.12 |\t28.09 |\t28.65 |\t29.14 |\t30.10 |\t30.18 |\t30.49|\n\n|HMT | | | | | | | |\n| --- | --- | --- | --- | --- | ---| ---| ---|\n|AL |\t0.27 |\t2.20 |\t3.46 |\t4.74 |\t6.43 |\t8.36 |\t10.06 |\n|Easy |\t23.70 |\t31.89 |\t33.59 |\t34.72 |\t34.80 |\t35.44 |\t35.42 |\n|Medium |\t22.22 |\t27.90 |\t29.86 |\t30.83 |\t31.68 |\t31.88 |\t31.70 |\n|Hard |\t20.42 |\t25.50 |\t26.99 |\t28.19 |\t28.96 |\t29.69 |\t29.75 |\n\nCompared to HMT, we observe that NAST excels particularly in handling harder cases, especially when real-time demands are higher. This highlights the superiority of NAST in addressing the non-monotonicity issue, making it more suitable for deployment in low-latency scenarios.\n\n> It can be particularly challenging to adapt the proposed method to other translation tasks.\n\nAs you mentioned, there is still a performance gap between NAR and AR models in these domains. However, we also notice a recent progress for NAR models in speech-related offline tasks. Given the broader application of SiMT models in scenarios involving verbal communication, we plan to further adapt our model in simultaneous speech-to-text and speech-to-speech translation in the future.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. The methods proposed in the paper are of great novelty. This is one of the first attempts on applying NAR to simultaneous translation. Not only the BLEU score, but the NAR model also shows substantial improvement over AR model on hallucination.\n2. The results are strong compared with baseline model", "reasons_to_reject": "1. More analysis on the model behavior, such as case study will help readers better understand the model.\n2. It can be particularly challenging to adapt the proposed method to other translation tasks, such as multilingual translation, low resource translation, speech translation, etc., in which the AR model is significantly better the NAR model", "questions_to_authors": "", "paper_topic_and_main_contributions": "The paper discusses a novel non-autoregressive model for simultaneous translation, which is one of the first attempts for this approach. In NAR architecture proposed in the paper, the mode upsample the hidden states and run CTC training on top of it to learn the monotonic alignment between source and target. During inference time, the model applied the wait-k policy. By introducing latency control, non-monotonic latency alignments, glancing and multi-stage training, the model can further improve the performance. From the numerical results, the model shows significant improvement over baselines.\n\n\n\n", "excitement": 4, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "1DnGQOL04m", "reviewer": "Reviewer_JCCF", "summary": "", "questions": "1. The performance of De -> En and En -> Ro looks pretty different. What's the reason behind it? \n2. I'm also curious about the En -> De performance since De is an SOV language and En is a SVO language. What will this language difference affect the final performance.", "limitations": "", "rating": 4, "confidence": 4, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your constructive comments. We provide discussions and explanations about your concerns as follows.\n\n> The paper lacks some analysis of the results and examples to illustrate the methods.\n\nWe appreciate your suggestions regarding the further anaysis. To provide a more comprehensive analysis of our model, we partition the De -> En test set into subsets based on the number of crosses in the alignments between the source and reference, categorizing them as easy, medium, and hard. Intuitively, a higher number of cross alignments pose greater challenges for a SiMT model. We compare our NAST with previous state-of-the-art AR SiMT model, HMT. The results are outlined below:\n\n|NAST| | | | | | | |\n| --- | --- | --- | --- | --- | ---| ---| ---|\n|AL | 0.65 |\t1.87 |\t2.97 |\t4.02 |\t5.83 |\t6.85 |\t8.44 |\n|Easy |\t27.19 |\t32.11 |\t34.43 |\t34.48 |\t35.21 |\t35.51 |\t35.51|\n|Medium |\t27.54 |\t30.23 |\t30.91 |\t30.90 |\t31.72 |\t31.81 |\t32.09|\n|Hard |\t28.12 |\t28.09 |\t28.65 |\t29.14 |\t30.10 |\t30.18 |\t30.49|\n\n|HMT | | | | | | | |\n| --- | --- | --- | --- | --- | ---| ---| ---|\n|AL |\t0.27 |\t2.20 |\t3.46 |\t4.74 |\t6.43 |\t8.36 |\t10.06 |\n|Easy |\t23.70 |\t31.89 |\t33.59 |\t34.72 |\t34.80 |\t35.44 |\t35.42 |\n|Medium |\t22.22 |\t27.90 |\t29.86 |\t30.83 |\t31.68 |\t31.88 |\t31.70 |\n|Hard |\t20.42 |\t25.50 |\t26.99 |\t28.19 |\t28.96 |\t29.69 |\t29.75 |\n\nCompared to HMT, we observe that NAST excels particularly in handling harder cases, especially when real-time demands are higher. This highlights the superiority of NAST in addressing the non-monotonicity issue, making it more suitable for deployment in low-latency scenarios.\n\n\n> The performance of De -> En and En -> Ro looks pretty different. What's the reason behind it?\n\nThank you for your insightful question regarding the experimental results. We would like to make a brief discussion on the following phenomena: 1. NAST significantly outperforms previous models under low-latency across datasets, while in high-latency scenarios, NAST achieves state-of-the-art results for De -> En and is slightly inferior for En -> Ro. 2. SiMT models are able to approach the performance of full-sentence translation model on De -> En. However, on En -> Ro, there remains a notable gap between all SiMT models and the offline model.\nWe believe this could be attributed to several factors:\n1. The simultaneous translation from English to Romanian involves both SVO language pairs. When latency requirements are lenient, the real-time generation difficulty is reduced. In such cases, the simultaneous translation task between two SVO languages almost degrades into a full-sentence translation task. In those conditions, the slight inferiorty of NAST could potentially reflect the gap between the full-sentence AR and NAR models.\n\n2. The En -> Ro dataset (0.6M) is notably smaller compared to the De -> En dataset (4.5M), which can lead to the overfitting issues. Given that SiMT models rely on a prefix-to-prefix learning approach, the problem of overfitting might be more pronounced. This could possibly explain why, even under larger latencies, all SiMT models on the En -> Ro dataset still exhibit a noticeable gap from the offline model.\n\n> I'm also curious about the En -> De performance since De is an SOV language and En is a SVO language. What will this language difference affect the final performance.\n\nWe would like to express our apologies. Due to time constraints and limited computational resources, we are unable to provide experimental results for the En -> De direction during the rebuttal phase. However, we intend to include these results in a subsequent version of the paper.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "The paper on Non-autoregressive Streaming Transformer for Simultaneous Translation introduces a novel approach to SiMT that combines several new ideas to address the issue of aggressive anticipation in previous autoregressive SiMT models. The proposed approach includes non-autoregressive decoding, which enables intra-chunk parallelism and flexible READ/WRITE strategy. The use of repeated token or blank token to represent delay allows the model to adjust its translation speed and latency dynamically. The alignment-based latency loss encourages the model to generate translations that are non-monotonic but still accurate, which is crucial for SiMT. By combining these ideas, the proposed approach achieves state-of-the-art performance on various SiMT benchmarks, including WMT15 German → English and WMT16 English → Romanian. ", "reasons_to_reject": "The paper lacks some analysis of the results and examples to illustrate the methods", "questions_to_authors": "1. The performance of De -> En and En -> Ro looks pretty different. What's the reason behind it? \n2. I'm also curious about the En -> De performance since De is an SOV language and En is a SVO language. What will this language difference affect the final performance.", "paper_topic_and_main_contributions": "his paper is about a new approach to Non-autoregressive Streaming Transformer (NAST) for Simultaneous Translation (SiMT). The paper addresses the issue of aggressive anticipation in previous autoregressive SiMT models, which results in poor translation quality and high latency. The main contribution of the paper is the development of a new NAST approach that enables flexible READ/WRITE strategy and alignment-based latency loss, which outperforms previous autoregressive SiMT models in terms of translation quality and latency. The paper presents experimental results on SiMT benchmarks, including WMT15 German → English and WMT16 English → Romanian, demonstrating the effectiveness of the proposed NAST approach.", "excitement": 4, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Typos_Grammar_Style_and_Presentation_Improvements": "I suggest the authors to present some generation examples to better illustrate the methods and results."}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper proposes a non-autoregressive streaming transformer for simultaneous translation, which addresses two current problems: "non-monotonicity problem" and "source-info leakage" .  Alignment are specially considered for setting the latency loss and translation loss, which is important in the simultaneous translation setting. The results show substantial improvement over existing research. Reviewers suggest more analysis into the model behaviors.
Decision: Accept-Main