Overcoming Non-monotonicity in Transducer-based Streaming Generation
Authors: Zhengrui Ma
Yang Feng
Min zhang
Venue: ICML 2025 poster (2025)
Keywords: streaming generation
simultaneous translation
Transducer
Primary area: applications->language_speech_and_dialog

Abstract: Streaming generation models are utilized across fields, with the Transducer architecture being popular in industrial applications. However, its input-synchronous decoding mechanism presents challenges in tasks requiring non-monotonic alignments, such as simultaneous translation. In this research, we address this issue by integrating Transducer's decoding with the history of input stream via a learnable monotonic attention. Our approach leverages the forward-backward algorithm to infer the posterior probability of alignments between the predictor states and input timestamps, which is then used to estimate the monotonic context representations, thereby avoiding the need to enumerate the exponentially large alignment space during training. Extensive experiments show that our MonoAttn-Transducer effectively handles non-monotonic alignments in streaming scenarios, offering a robust solution for complex generation tasks. Code is available at https://github.com/ictnlp/MonoAttn-Transducer.

Overcoming Non-monotonicity in Transducer-based Streaming Generation
Zhengrui Ma
Yang Feng
Min Zhang
Abstract
Popular streaming generation methods can be broadly divided into two categories: Attention- based Encoder- Decoder (AED; Bahdanau et al., 2015) and Transducer (Graves, 2012). Streaming AED models adapt the conven- tional aequence-to-sequee to support streaming generation. They often rely on an exter- nal policy module to determine the READ/WRITE actions in inference and to direct the scope of cross-attention in training. Examples include Wait-k policy (Ma al., 2019) and monotonic attention-based methods (Raffel et al., 2017; Arivazhagan et al., 2019; Ma et al., 2020d; 2023a). On the other hand, Transducer models connect the encoder and predictor through a joiner rather than using cross-attention. The joiner is designed to synchronize the encoder and pre- dictor by expanding its output vocabulary to include a blank symbol c, which indicates READ action. Due to the decoupling of the predictor state from the encoder state, READ/WRITE, states in Transducer can be represented by two-dimensional lattice. This allows for the computation of total probabilities using the forward backward algorithm (Graves, 2012), facilitating end-to-end optimization. Bene- fited from joint optimization of all potential policies during training, Transducer often demonstrates better performance compared to AED models (Xue et al., 2022; Wang et al., 2023).
Streaming generation models are utilized across fields, with the Transducer architecture being pop- ular in industrial applications. However, its input- synchronous decoding mechanism presents chal- lenges in tasks requiring non-monotonic align- ments, such as simultaneous translation. In this re- search, we address this issue by integrating Trans- ducer's decoding with the history of input stream via a learnable monotonic attention. Our approach leverages the the posterior probability of alignments between the predictor states and input timestamps, which is then used to estimate the monotonic context rep- resentations, thereby avoiding the need to enumer- ate the exponentially large alignment space during training. Extensive experiments show that our -rannuuuur effectively handles non- monotonic alignments in streaming scenarios, of- fering robust solution for complex generation tasks. Code is available at //thth/:////thb.
framework (Bahdanau, 2014)
algorithm to infer
1. Introduction
During the decoding process of Transducer, each target token is explicitly aligned with a corresponding source to- ken. This input-synchronous decoding property makes the architecture well-suited for tasks like speech recognition, where the input and output align monotonically. However, it poses challenges for non-monotonic alignment tasks such as simultaneous translation (Chuang et al., 2021; Shao & Feng, 2022; Ma et al., 2023b:c). Due to the decoupled de- sign, Transducer models have limited ability to attend to the input stream history during decoding, making it hard to manage reorderings. To address this issue, recent research (Liu al., 2021; Tang al., 2023) has started explore the incorporation of cross-attention mechanism to enhance the capacity for handling complex non-monotonicity. De- spite these efforts, the integration of cross-attention presents significant challenges. By integrating the predictor states with source history through attention, the representation of predictor states becomes relevant not only to the encoder states but also the specific READ/WRITE path history
Streaming generation is a widely studied problem in fields such as speech recognition (Raffel et al., 2017: Zhang et al., 2020; Seide et al., 2024), simultaneous translation (Cho & Esipova, 2016; Gu et al.. 2017; Seamless Communication al., 2023), and speech synthesis (Ma et al., 2020a; Zhang et al., 2024; Wang et al., 2024). Unlike modern based large language models, streaming models need to start gen- crating the output before the input is completely read. This necessitates a careful balance between generation quality and latency.
Institute of Computing Technology, Chinese Academy of Sci- ences University of Chinese Academy of Sciences Schooo of Future Science and Engineering, Soochow University. Correspon- dence Zhengrui Ma <mazhhngrui2llee hnt.eenngnuiiibbii............. Yang Feng (Corresponding Author) <fengyang ict.ac.cn>.
Proceedings of the ndnd International Conference Machine Learning, Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).

---

Overcoming Non-monotonicity in manssducer-basee Streaming Generation
(Tang et al., 2023). This results in an exponentially large state space for Transducer, hindering the application of the forward-backward algorithm for end-to-end training.
are usually modeled using either a recurrent neural network (Graves, 2012) Transformer layers (Zhang et al., 2020). The joiner is typically composed of a feed-forward network.
In this research, we present an efficient training algo- rithm for Transducer models to learn the monotonic cross- attention mechanism. This allows Transducer's predictor to access source history in real-time inference, improving its ability to handle tasks with non-monotonic alignments. We leverage the forward- backward algorithm to infer the posterior probability of alignments between predictor and encoder states in training. This derived posterior alignment enables the estimation of context representation for each predictor state using expected soft attention. In this way, Transducer models adaptively adjust the scope of attention based on their predictions, avoiding the need enumerate the exponentially large alignment space during training.
Since explicit alignment information for parallel pairs is not available during training, it is necessary to solve for the total probabilities of all READ/WRITE paths that can generate the target to perform maximum likelihood esti- mation. Given that the state space of Transducer form a owo-dimensional lattice, the forward-backward algorithm can be utilized to compute the total probability. Define the forward and backward variables as:
all, u) == p(yim/ B(t, u) := p(u++ (eTT)
(1)
The forward and backward variables for all 1 1 t S T and 0 I u S U can be calculated recursively:
We conduct experiments on both ppeech-t-teex/speedd si- multaneous translation to demonstrate the generality of our approach across various modalities. MonoAttn- Transducer shows significant improvements in generation quality with- out a noticeable increase in latency in both ideal and computation-awware settings ($5). Further analysis reveals that MonoAttn- Transducer is particularly effective in han- dling samples with higher levels of non-monotonicity ($6).
att, u)
= alll (((1-1,u)pil 1,u)p(e/--1,) + +all, u ()p(vul,,
B(t, u)
= (t + +1,,)p((ett 2) - B(t, u 1)p((u+1lt..)
(2)
with initial condition (11, 0) = 1 and (T, p(c|T,U) p(ult, u) denotes the probability of generating token from h, and su, v E VU {e}. The total output probability is:
2. Background
ply x) = aTT, (p(eTT U).
(3)
2.1. Streaming Generation
By leveraging the forward-backward algorithm, Transducer models are trained to implicitly acquire the READ/WRITE policy from the data.
Streaming generation models typically process a stream- ing input x {z1,.... [rr and generate target sequence y = (y1, .... } in streaming manner. To measure the amount of source information utilized during generation, monotonic non non-decreasing function g(u) is introduced to represent the number of observed source tokens at the time of generating yu-
3. Method
In this section, we provide detailed introduction to our proposed MonoAttn-Transdueer
2.2. Transducer
3.1. Overview
Transducer model (Graves, 2012) comprises three compo- nents: an encoder, a predictor, and a joiner. The encoder unidirectionally encodes the received input prefix Tia into a context representation ht. The predictor functions sim- ilarly to an autoregressive language model, encoding the dependencies between tokens in the generated prefix Vi.u into s.. The joiner makes predictions based the cur- rent source representation h4 and target representation su- If the model needs to READ more information to update the source representation for continued generation, blank token is generated. Otherwise, a WRITE operation is performed, and the generated token fed back into the pre- dictor to obtain new target representation. Each time h, or su is updated, the joiner performs prediction step until the entire source has been processed. The encoder and predictor
-annAAtt--rrnsducce works similarly to standard Trans- ducer, with the key difference being that its predictor can attend to the encoder history using monotonic attention. During streaming generation, the scope of monotonic atten- tion includes all source context representations that have already appeared. Formally, when the predictor encodes the u-th target state, it depends on the representations of previous target states and the existing source context:
8u = 100000=-1,,,100
(4)
where 1 S u < U and g(u) denotes the number of ob- served source tokens at the time of generating Vu- The edge case so is defined as su = fo()). In both Transducer and MonoAttn- Transducer, token ya is generated based
2

---

Training MonoAttnnTie Transuucer is challenging as it expo- nentially expands Transducer's state space. To address this issue, we firstly leverage the forward-backward algorithm to compute the posterior probability of aligning target represen- tation 8u with source representation ht (i.e., the probability of generating token Yu immediately after reading zz). This posterior alignment is then used to estimate the expected Overcoming Non-monotonicity in mansducer--asse Streaming Generation
on source representation lg(()) and target representation Au-1- Given Su-1 can only attend source contexts up g(u 1) through monotonic attention, related information in 1=(==1)+1:g(() should ideally be encoded within hg(u)-
MonoAttn-Tanssuuce learns monotonic attention mecha- nism while maintaining the same time and space complexity as Transducer.
Formally, given the energy eu, for the pair consisting of en- coder state he and predictor state Su. as well as the posterior alignment probability Tu,t, the expected context representa- tion Cy for predictor state Su can be expressed as:
3.2. Training Algorithm
exp (Cu,t) 2 exp (ew,t")
Cu =
Σ Tu,l
hy.
(8)
This indicates that the expected context representation ea is weighted sum of context representations under various amount of source information, with the weights given by the posterior alignment probability Tu, The nested summation operations in Eq. 8 may lead to an increase in computational complexity. Fortunately, Arivazhagan et al. (2019) suggests that it can be rewritten as:
context vector in the monotonic cross-attention for each pre- dictor state in training. Detailed explanations are provided in the following.
3.2.1. POSTERIOR ALIGNMENT
Suppose we have probability lattice p(ult, u). representing the probability of generating token from h, and 8a, for 1 S S T, < u S U, and V The posterior probability of generating Yu the moment 2; is read can be represented by:
Tu exp (Ew,t)
Pu,t
=1 exp (eu,t")
(9)
C2 =
Dush
Fu,t 1oa 1)p(0un1o (5)
p(u::0 (x1r))
Eq. 9 can then be computed efficiently using cumulative sum operations (Arivazhagan et al., 2019). Then in training,
with the edge case:
Eq. 4 can be estimated
1 t=1 00 4
S, fo($ooa-1,00 C.).
(10)
To,c
(6)
3.2.3. TRAINING WITH PRIOR ALIGNMENT
which implies that the predictor state 80 is generated imme- diately after the first source token arrives. Using the forward and backward variables introduced in Section 2.2, Eq. 5 can be concisely expressed as follows:
The above algorithm facilitates MonoAttn- MonoAAtn-Transsduee Transducer in learning monotonic cross-attentinn with posterior alignment probability. However, this presents chicken-and-egg para- dox: the posterior alignment is derived from an output prob- ability lattice constructed using an estimated context rep- resentation, while the context vector is, in turn, estimated using a posterior alignment. We address this problem by using a prior alignment to construct a prior output proba- bility lattice This lattice is then used to infer the posterior alignment and train MonoAttn- -arnn-tt-aaadurrr" 's monotonic at- tention.
Tut att, u - 1)p(v, L u- u))
(7)
a (T,U))((ET.)
This guarantees that the posterior aligmment probability for all pairs (t, u) can be solved in (TT)) time using the above forward-backwarte algorithm, facilitating the calculation of the expected context representation introduced later.
3.2.2. MONOTONIC ATTENTION
There are several options for the prior alignment Pu,r- The simplest one is the uniform distribution, which assigns an equal probability of being generated at any timestep for all the target tokens:
The incorporation of monotonic attention makes the representation of predictor states relevant specific READ/WRITE history, leading to a prohibitively large state space for enumerating alignments. Therefore, we turn to estimate the context vector in monotonic attention based on the posterior alignment probability during training. This approach enables the model to adaptively adjust the scope of cross-atis antes-attention according to its prediction. Consequently,
uni 1 P T 1SIST,
(1I)
The edge case p is similar to the situation of no,s, where all the probability mass is concentrated at 1.
3

---

Overcoming Non-monotonicity in Transducer- based Streaming Generation
However, it is preferable to select a more reasonable prior. An ideal prior alignment should ensure that the posterior alignment, derived from the lattice constructed using the prior, can accurately estimate the expected context represen- tation. In streaming generation tasks, even though there may be reorderings in the mapping from source to target, a cer- tain level of monotonic alignment generally maintained. Therefore, we propose introducing prior distribution pdia. which assumes that the number of tokens generated for each READ action is uniformly distributed:
Algorithm 1 Training Algorithm of MonoAtti-Trannsdue
Input: Source æ, Target y. Chunk Size
Output: Training Loss L
I: Compute prior alignment Pa. (Eq. 12)
2: Compute chunk-synchronizee prior alignment Pd based on chunk size C' (Eq. 13)
3: Estimate context eprior with Pii (Eq. 9)
4: Forward MonoAttn- Attn-Trannduuer with oprior
5: Infer posterior alignment Tu,t (Eq. 7)
6: Compute chunk sounn---ychronized posterior alignment a... based on chunk size C/ (Eq. 13)
Wu,t = exp (-|u U
T
7: Estimate context Cy with uu,; (Eq. 9)
(12)
Wuat
Forward MonoAttn- -rannsuuurr with Cn.
Put
T Ex=1 Wa,t'
Calculate total output probability C (Eq. 3)
10: return L
for 1 ISIS, The edge case P is handled in the same manner as Po.t- This prior assumes a uniform mapping between the source and target, such that each target token most likely generated at the time its corresponding source token is read. The probability decreases as the time difference from this moment increases.
source be available, limiting to offline generation. For streaming generation, the receptive field of attention must synchronize with the input. This synchronization leads to an exponentially large state space, which significantly com- plicates the training process. To mitigate this issue, Liu et al. (2021) separated the predictor's 's cross- cross-attention attention from its self- attention, ensuring that cross-attention occurs only after self-attention. This approach maintains the independence of predictor states from READ/WRITE path history, allowing for standard training methods. However, this separation limits the richness of the predictor's 's learned representations. Alternatively, Tang et al. (2023) proposed updating the rep- resentation of all predictor states whenever a new source token is received. While this method also preserves the independence of predictor states from READ/WRITE path history, it significantly increases both inference-time compu- tational complexity and training-time memory requirements. It necessitates an additional (T - 1) forward passes of the predictor during decoding, which adversely affects latency- sensitive streaming generation. Furthermore, the GPU mem- ory usage for attention during training increases from O(1) to OTT), leading to prohibitively high training costs and limiting the model's scalability. In contrast to the above, the proposed MonoAAtn- - maintains the same time complexity and memory overhead as Transducer. A detailed comparison between these methods is summarized in Table 1.
3.2.4. CHUNK SYNCHRONIZATION
In speech audio, there often exists strong temporal depen- dencies between adjacent frames. Therefore, a chunk size C' is typically set, and the streaming model makes decisions only after receiving a speech chunk (Ma al., 2020c). In terms of Transducer models, when a READ action is taken, the source representation is updated after new speech chunk is read. The new source representation is then set as the representation of the last frame in the chunk (Liu et al., 2021; Tang et al., 2023). In such a situation, the receptive field of Connottt--Transuuce tor state 8a encompasses all hidden states in the received chunks, i.e., h1:C-3(u). where g(u) denotes the number of received chunks when generating token Yu- To bridge the gap between training and inference, the posterior alignment probability utilized in training process is adjusted by trans- ferring all the probability mass on encoder states within a chunk to the last state in the chunk:
cross cross-attention for predic-
(1= Fup 1=d.c C
Ede
1 d.
(13)
for d= 1,2,,
The prior alignment probability is adjusted in the same manner. We detail the entire training process in Alg. 1.
Our work is also related to researches in designing attention modules for streaming AED models. These works often introduce Bernoulli variables to indicate READ/WRITE ac- tions. The distribution of these variables is used to estimate monotonic alignment and to compute the expected context representation in training (Raffel et al., 2017). Depending on the setting of attention window, these works can be clas- sified into monotonic hard attention (Raffel et al., 2017), monotonic chunkwise attention (MoChA; Chiu & Raffel,
4. Related Work
Our work is closely related to researches in designing cross- attention modules for Transducer models. Prabhavalkar et al. (2017) pioneered the use of attention to link the predic- tor and encoder. However, their design requires the entire
4

---

Overcoming Non-monotonicity in Transducer- based Streaming Generation
Table Comparison of Transducer-based streaming models. Computational Complexity refers to the number of forward passes executed by the predictor in inference. Memory Overhead refers to the memory consumption of the attention module in training.
Method
Merge Module
Computational Complexity
Memory Overhead
Transducer (Graves, 2012)
Joiner
O
N/A
CAAT (Liu et al., 2021)
Joiner
(
O
TAED (Tang al., 2023)
Predictor, Joiner
O TT)
O(T)
MonoAttn- MonoAtu-Transsueer (Ours)
Predictor, Joiner
O(U)
((1)
2018), and monotonic infinite lookback attention (MILk; Arivazhagan et al., 2019). Ma et al. (2020d) subsequently in- troduced the MILK mechanism to Transformer models, and Ma et al. (2023a) further proposed numerically-stable algo- rithm for estimating monotonic alignment. Unlike the afore- mentioned works, our approach learns monotonic attention based on the posterior alignment of Transducer, avoiding the use of unstable Bernoulli variables.
2018) are incorporated into the speech encoder. Sinusoidal positional encoding is applied after the convolution layers. The predictor comprises two autoregressive Transformer layers with post-norm, utilizing only sinusoidal positional encoding. The monotonic attention is similar to standard cross-attentiin but differs in its receptive field. The joiner is implemented as a simple FFN. We incorporate the multi- step decision mechanism (Liu et al., 2021) with a decision step of 4. All Transformer layers described above are con- figured with 512 embedding dimension, 8 attention heads and a 2048 FFN dimension. The total number of parameters for the Transducer baseline and MonoAttn-Transe 65M and 67M, respectively. More implementation details are provided in App. A.
5. Experiments
We validate the performance of our MonoAAtn-Tranndueer on two typical streaming generation tasks: speech-to-text and speech-to-sye soeehhhh simultaneous translation. The differ- ences in grammatical structures between the source and target languages often necessitate word reordering during generating translation. This property makes the simultane- ous translation task well-suited for evaluating the ability in handling non- non-monotonic alignments.
are
Evaluation We use SimulEval toolkit (Ma et al., 2020b) for evaluation. Translation quality is assessed using case- sensitive detokenized BLEU (Papineni et al., 2002; Post, 2018) and neural-based COMET-2 22 score. Latency is mea- sured by word-level Average Lagging (AL; Ma et al., 2019; 2020c) and Length- Average Lagging (LAAL; Papi et al., 2022).! For speech-to-speedh tasks, translation qual- ity is assessed using ASR-BLEU and latency is measured by delay of generated waveform chunks (Ma et al., 2022).
5.1. Experimental Setup
Datasets We conduct experiments on two language pairs of MuST-C speech-to-texx translation datasets: English to Ger- man (En-De) and English to Spanish (En-FEs) (Di Gangi et al., 2019). For speech-to-specch experiments, we evaluate models on CVSS-C French to English (Fr->En) dataset (Jia et al., 2022).
5.2. Main Results
We evaluate the performance of -ransuucc against Transducer bascline across various latency condi- tions obtained by varying the chunk size. In this comparison, we consider two configurations of MonoAttn-Transddee The first, referred to as is trained strictly according to Algorithm 1. The second, termed Mono -att--Trnnduuer-PPPPa Prior, is optimized directly using prior alignment, without inferring the posterior (cal- culate total output probability L using cprior). ). Results are shown in Table 2.
Model Configuration We use the open-source implementa- tion of Peasffrmeer-ransducee (Zhang et al., 2020) from Liu et al. (2021) as baseline and build our MonoAutn- Transducer upon it. The speech encoder consists of two layers of causal 2D-convolution followed by 16 chunk-wise Transformer layers with pre-norm. Each convolution layer has 3x3 kernel with 64 channels and a stride size of 2, resulting in a downsampling ratio of 4. In chunk-wise Trans- former layers. the speech encoder can access states from all previous chunks and one chunk ahead of the current chunk (Wu et al., 2020; Shi et al., 2021). The chunk size is adjusted within the set (320, 640, 960, 1280)ms. ms. Offline results are obtained by setting the chunk size longer than any utterance in the corpus. Both sinusoidal positional encoding ( (Vaswani et al., 2017) and relative positional attention (Shaw et al.,
It can be observed that MonoAttn-Transsuuerr Seersdduucr--osterioo sig- nificantly outperforms the Transducer baseline across var-
Nuuerrcal results with more metrics are provided in App. C. Notably, Table 6 presents comparison of the computation-awore latency metrics for Al. and I.A hetween the Transducer and MonoAun-Transduee Transducer models.
5

---

Overcoming Non-monotonicity in Transducer- based Streaming Generation
Diagonal Prior Alignment
Tambien
queria
estabames
lomando
decisiones
tha
Jnnnrra
que
hileer
más
seguro
<is>
Posterior Alignment
Tambien
queria
enlender
estabomes
Tomando
Jecislones
manera
que
nos
más


also
wanted to
understand
wavere
making
decislans
a:
way
that made:
safer
Figure 1. An example of diagonal prior and posterior alignment from MuST-C Engliss-tooSpannis training corpus. The vertical axis represents the target subword sequence borizontal axis represents the speech waveform. Darker areas indicate higher alignment probabilities. Chunk size in this example set 640ms. More examples are provided App. D.
ious settings of chunk size in both translation directions. Specifically, in En-Es, it shows average improvement of 0.75 BLEU or 0.95 COMET score in generation quality under different latency conditions. In En-De, it achieves an even more significant improvement, with an average in- crease of as much as 2.06 COMET score, while latency remains nearly unchanged. Further analysis reveals that the benefits of learning monotonic attention are more pro- nounced with larger chunk size. Notably, in scenarios where latency exceeds 1.5s and during offline generation, the average improvement reaches 0.88 BLEU or 1.77 COMET score. This can be attributed to MonoAttn- itn-Transducer bene- fiting more from monotonic attention to handle reorderings when it has flexibility to wait for more source information.
alignment. From the chunk synchronization mechanism described in Eq. 13, smaller chunk sizes require finer align- ment granularity between the predictor and encoder states. This increased granularity necessitates more precise align- ment to estimate the expected context representation during training. Figure 1 provides an example diagonal prior and posterior alignment. While the diagonal prior generally captures the trend of the alignment information, it can be skewed by the uneven distribution of speech information and possible local reorderings. In contrast, the inferred posterior offers a more confident and accurate alignment probability. For instance, the diagonal prior assigns high probability to aligning the word "si (if) with the timestep preceding the waveform of "if", while the inferred posterior corrects this misalignment. Therefore, learning monotonic attention with posterior alignment leads to context representation and improved performance." In sub- sequent experiments, represent MonoAttn- -rannsuc-r using the results of MonoAttn Traasducer-Posterre
Moreover, we have observed some notable results of MonoAAtn--ranssuueer--e
accurate estimation of
With larger chunk size, the performance of MonoAtan--Transducer-Piin is compara- ble to that of MonoAttn- Trransducer-Posteeede slightly outperforming the latter in En-De. However, there exists a significant performance drop with a smaller chunk size. Specifically, with a chunk size of 320ms, MonoAutn- Transducer-Prre Prior''s generation quality on average 1.03 BLEU lower than Transducer baseline under similar latency conditions. This phenomenon highlights the importance of learning monotonic attention through inferring posterior
and even
"We present comparison between the prior and posterior under various chunk sizes in App. D. key observation that alignment granularity becomes finer, the differences gradually increases.
6

---

Overcoming Non-monotonicity in Transducer- based Streaming Generation
Table Comparison of MonoAttm- Transducer and Transducer across various chunk size settings MuST-C English to German and
Tonssuun-ransdueer English to Spanish datasets.
En-Es
En-De
Chunk Size (ms)
320
640
960
1280
320
640
960
1280
LAAL (ms)
1168
1466
1847
2220
1258
1563
1942
2312
Transducer
BLEU (1)
24.33
25.82
26.36
26.40
26.75
19.99
22.10 10
22.20
22.96
23.10
COMET (1)
67.94
69.92
70.48
70.65
71.14
62.81
65.01
65.75
66.26
67.03
LAAL (ms)
1230
1475
1837
2204
1317
1582
1957
23305
MonoAttn- MonoAttn-Transduce
BLEU (1)
24.72
26.74
27.05
27.41
27.48
20.22
22.47
22.94
23.74
24.42
(Posterior)
COMET (t)
68.98
70.71
71.21
71.90
72.24
64.24
67.06
68.22
68.54
69.82
LAAL (ms)
1123
1419
1815
2184
1231
1535
1929
2286
MonoAtt--Tanssuue Transducer
BLEU (1)
23.00
26.46
27.07
27.42
27.48
19.26
22.62
23.51
24.01
24.42
(Prior)
COMET (t)
68.24
70.45
71.33
71.99
72.24
63.85
67.63
68.65
69.27
69.82
32
24
22
28
20
24
18
T0ve
18
22:
NAST
Mi-Teada
14
12
16
18
Lagging
Lazzing
(a) En-+De
(b) En-+Es
(c) En En-EEs Subsets
Figure 2. (a). (b): Results of translation quality (BLEJ) against latency (Average Lagging, AL) MuST-C English German and English to Spanish datasets. (c): Performance on MuST-C English to Spanish test subsets categorized by non-monotonicity. In the figures above, MA-T denotes MonoAttn-Transduce
5.3. Comparison with State-of-the-Art
our work. However, the code and distilled data used train TAED in Tang al. (2023) have not been made publicly available. This lack of open access hinders fair compari- son of TAED with our MonoAttn- MoonoAtt--Transuuee Despite this, we attempt to analyze the performance by comparing each with Transducer baseline in their respective experimental settings. The comparison shown Table App. C. have observed that the improvement from TAED more pronounced with smaller chunk sizes, which contrasts with the results of MonoAttn-Transducce We speculate that this is because, in TAED, the representations of all generated predictor states are updated every time the encoder receives new speech chunk. This helps TAED generate more accu- rate representations when the chunk size is small. However, this mechanism in TAED incurs an O(T U) forward propagation cost during simultaneous inference, which can significantly increase latency in practice due to heavy putational overhead when the chunk size is small. In con- trast, MonoAutn-Taansduccm maintains an O(U) complexity as Transducer baseline. As shown in Table 6, this property
We compare MonoAttn-Transduue with state-of-the-art open-source approaches in simultaneous translation, includ- ing Wait-k (Ma et al., 2020c), RealTrans (Zeng et al., 2021), CAAT (Liu et al., 2021), MU-ST (Zhang et al., 2022), EDAut (Papi et al., 2023a), AlignAtt (Papi et al., 2023b), Seg2Seg (Zhang& & Feng. 2023), NAST (Ma et al., 2024) and LLM- based LSG (Guo et al., 2025). Further details about base- lines are available in App. B. Results are plotted in Figure and 2b. We observe that learning monotonic attention signif- icantly enhances the performance of Transducer, achieving leading streaming generation quality. Compared to CAAT, another Transducer-basee model, MonoAttn- Trantttuerraduuuuerr demonstrates superiority in scenarios with less stringent la- tency requirements. This clearly demonstrates the advantage of MonoAAtn- - tightly coupled self attenttonn and tross-attention modules in the predictor, which facilitates the learning of richer representations.
TAED is another Transducer-basee model highly relevant to

---

Overcoming Non-monotonicity in Transducer- based Streaming Generation
highlight the unique capabilities of ConnoAtt--Tannddue in managing non-monotonic alignments. As analyzed in Section 5.2, MonooAtt-Trannsduee benefits more from learn- ing monotonic attention with larger chunk size, and this enhanced ability is evident in subsets with higher levels of non-monotonicity. On the other hand, when the chunk size is extremely small, MoonoAttt-Transduue flexibility to wait for more source information before pro- cessing, thus showing more significant improvement in the easy subset under the condition.
Table Performance on CVSS-C French to English speech-to-
speech translation.
Chunk Size (ms)
320
Offline
ASR-BLEU (1)
17.1
18.0
Transducer
LAAL (ms)
984
StartOffset (17s))
1520
has limited
ASR-BLEU (1)
18.3
19.3
MonoAttn- Transduurr
LAAL (ms)
918
StartOffset (171))
1491
6.2. Training Efficiency
minimizes the gap between ideal and computation-awart -wware latency, offering advantages in real-time applications.
Training Time: We analyze each step in Algorithm 1 to compare the training time differences between MonoAttn- Transducer and baseline. We observe that Lines 1, 2, 6 in- volve naive matrix computation without requiring gradients. The additional time overhead introduced by our method arises from Lines 3, 4, 5. Specifically, this includes an addi- tional forward pass of the predictor and the computation for the posterior alignment. The overhead from the posterior calculation is approximately equivalent to that incurred dur- ing loss calculation, as both rely on the forward-backwe algorithm. Empirically, we found MonoAttn- -aatn---rrrrrrr is 1.33 times slower than Transducer baseline with the same configuration on Nvidia L40 GPU.
5.4. Results of Speech Generation
Speech-to-spest simultaneous translation requires implic- itly performing ASR, MT and TTS simultaneously, and also handling the non-monotonic alignments between languages, making it suitable to evaluate models on streaming speech generation. We adopted textless setup in our experiments, directly modeling the mapping between speech (Zhao al., 2024). Results are provided in Table 3.
The results demonstrate that MonoAttn-Transdus Transducer signifi- cantly reduces generation latency (AL). With a chunk size of 320ms, it achieves Transducer's offline generation quality, but reducing lagging to 1 118ms. For offline settings, our ap- proaches further improves speech generation quality (19.3 vs. 18.0). These results highlight the effectiveness of our approach in achieving a better quality-latency trade-off also for streaming speech generation.
Compared to baseline, the addi- tional memory overhead of -aannttt-aaannerrr comes solely from its monotonic attention module. The extra for- ward pass of the predictor is performed without requiring gradients, so it is excluded from the computation graph. Empirically, we observed that the peak memory usage of Transducer baseline is 28GB, while MonoAttn-Transdueen Transducer exhibits a slightly higher peak usage of 32GB when the total number of source frames is fixed at 40,000 on single Nvidia L40 GPU.
Memory Consumption:
6. Analysis
6.1. Handling Non-monotonicity
To illustrate MonoAttn- ConnoAttn-ranssucerr capability in handling reorderings through learning monotonic attention, we evalu- ate its performance against the Transducer baseline across samples with varying levels of non-monotonicity. Intu- itively, samples with higher number of crosses in the alignments between source transcription and reference text pose greater challenges. We therefore evenly partition the test set based on the number of pross-alignments. labeling them as easy, medium and hard. 3 The results are presented in Figure 2c. We observe that MonoAttn -rannsuuueeee shows more substantial improvement over Transducer in the medium and hard subsets across most chunk size settings. However, with a chunk size of 320ms, the improvement is particularly notable in the easy subset. These findings
7. Conclusion
In this paper, we propose an efficient algorithm to handle the non-monotonicity problem in Transducer- stream- ing generation. Extensive experiments demonstrate that our significantly improves the ability in handling non-monotonic alignments in streaming genera- tion, offering a robust solution for Transducer-base frame- works to tackle more complex streaming generation tasks.
Impact Statement
This paper presents work whose goal is to advance the field of Machine Learning. There many potential societal consequences of our work, none which we feel must be specifically highlighted here.
'The easy subset includes samples with cross count of I or fewer. The medium subset contains samples with a cross count hetween 2 and 6. Samples with cross count greater than 6 classified as hard.
8

---

Overcoming Non-monotonicity in Transducer- based Streaming Generation
References
unsegmented sequence data with recurrent neural net- works. In Proceedings of the 23rd International Con- ference on Machine Learning, ICML '06, pp. 369-376, New York, NY, USA, 2006. Association for Comput- ing Machinery. ISBN 1595933832. doi: 10.1145/
Arivazhagan, N., Cherry, C., Macherey, W., Chiu, C.-C., Yavuz, S., Pang, R., Li, W., and Raffel, C. Monotonic in- finite lookback attention for simultaneous machine trans- lation. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pp. 1313- 1323, Florence, Italy, July 2019. Association for Compu- tational Linguistics. doi: 10. 10.11865/v1//19--111 URL https: / aawaaaaanantoyooaaaaaaaaa wttp//9-1//
1143844.1138991 URL https://dol: org/10. 1145/114384 111111444444.14389.
Gu, J. Neubig, G., Cho, K., and Li, V. O. Learning to translate in real-time with neural machine translation. Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 1, Long Papers, pp. 1053-1062, Valencia, Spain, April 2017. Association for Computational Linggutttionallnginiissssss wrg/e17-109999
Bahdanau, D. Neural machine translation by jointly learning to align and translate. arXiv preprint arXiv: 1409.0473, 2014.
Bahdanau, D., Cho, K., and Bengio, Y. Neural machine translation by jointly learning align and translate. In 3rd International Conference on Learning Repre- sentations, ICLR 2015, San Diego, CA, USA, May 7- 9, 2015, Conference Track Proceedings, 2015. URL http: //tpr//aavvv.//aae/ea/eea.eaaa.a http//aawi....gaassa./. wwtg/ab///4// 0473.
URL http:::tt//....... http:/::/aaaaaathhhaoae
Guo, S., Zhang, S., Ma, Z.., and Feng, Y. Large language models are read/write policy-makers for simultancous generation, 2025. URL https: http:///xxvvvv... abs/ 00868.
Jia, Y. Tadmor Ramanovich, M., Wang, Q. and Zen, H. CVSS corpus and massively multilingual speech- to-speech translation. In Proceedings of Language Re- sources and Evaluation Conference (LREC), pp. 6691 6703, 2022.
Chiu, C.-C. and Raffel, C. Monotonic chunkwise attention. In International Conference on Learning Representations, 2018. URL https:::p.............tt http///ppeeeeeeeeeeeeeeeeeeeeeee n forum? id=Hko85plcW.
Cho, K. and Esipova, M. Can neural machine translation do simultaneous translation? CoRR, 0abs/1606.02012. 2016.
Kim, Y. and Rush, A. M. Sequence-level knowledge distil- lation. In Proceedings of the 2016 Conference on Empiri- cal Methods in Natural Language Processing, pp. 1317- 1327, Austin, Texas, November 2016. Association for Computational Linguistics. doi: 10. 10.11665/11/111--1111 1139. URL https: /t//aclaatttt http//aaaanooooooooooooao org 1139.
URL http: //t:///aaaa /atpxxixivvvv////////a///aaaaa org/ aa/aabsss16600 02012.
Chuang, S.PP. Chuang, Y.-S., Chang, C.-C., and Lee, H.yy. Investigating the reordering capability in CTC- based non-autoregreessive end-to-end speech transla- tion. In Findings of the Association for Compu- tational Linguistics: ACL-IJCNLP 2021, pp. 1068- 1077, Online, August 2021. Association for Computa- tional Linguistics. doi:
Kingma, D. P. and Ba. Adam: A method for stochastic op- timization. In 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings, 2015. URL http:/ /tap:://aaxiv... www.xiii.rrggbs///.eee ab4/1 6980.
92. URL https:/ //aaaaaaantooooyyy..ooo.a org/2011. findings-acl.
Kudo, T. and Richardson. J. SentencePiece: A simple and language independent subword tokenizer and detokenizer for neural text processing. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pp. 66-71, Brus- sels, Belgium, November 2018. Association for Compu- tational Linguistics. doi: 10.18653/v1/D18-2000 URL https: http::://acaattthooo/ hww.hhhwoooooooo..g.eoee Saargg/D8-22011
Di Gangi, M. A., Cattoni, R., Bentivogli, L., Negri, M., and Turchi, M. MuST-C: a Multilingual Speech Transla- tion Corpus. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Com- putational Linguistics: Human Language Technologies, Volume (Long and Short Papers), pp. 2012-2017, Min- neapolis, Minnesota, June 2019. Association for Compu- tational Linguistics. doi: 10.18653/11/N11--10000 URL https: / htt///aaaaaaallceo wwwp:gg..rrrggggeee /ww//11g//// 1202.
Liu, D., Du, M., Li. X., Li, Y., and Chen, E. Cross attention augmented transducer networks for simultaneous transla- tion. In Proceedings of the 2021 Conference Empiri- cal Methods in Natural Language Processing, pp. 39-55, Online and Punta Cana, Dominican Republic, Novem- ber 2021. Association for Computational Linguistics. URL https:
Graves, A. Sequence transduction with recurrent neural networks. CoRR, ab3/1211.371,, 2012. URL http:
www.aii..rg//aaa...a.oa.a.. 1211. 3711.
Graves, A., Fernández, S., Gomez, F,, and Schmidhu- ber, Connectionist temporal classification: Labelling
doi: 10011863//12001.emmpp.... //aclanthology wtwpollloooyooggoggg// emn 1 1--maimm 4.
9

---

Overcoming Non-monotonicity in Transducer-bars based Streaming Generation
Ma, M., Huang, L.,, Xiong, H., Zheng, R., Liu, K., Zheng. B., Zhang, C., He, Z., Liu, H., Li, X., Wu, H., and Wang. H. STACL: Simultaneous translation with implicit an- ticipation and controllable latency using ceeix-to-prefix framework. In Proceedings of the 57th Annual Meet- ing of the Association for Computational Linguistics, pp. 3025-3036, Florence, Italy, July 2019. Association for Computational Linguisties. doi: 10. 10.18653/v1//19- 1289. URL https: htt///aaaaattt //w.aaaaanooooyo...oooooo
Conference on Learning Representations, 2023b. URL https://arsiw..rrg///e/e wttpaaa.....gga//////.aa// 06662.
https:/
Ma, Z.,, Zhang. S., Guo, S., Shao, C., Zhang, M., and Feng, Y. Non-autoregressive streaming transformer for simultaneous translation. In Proceedings of the 2023 Conference on Empirical Methods in Natural Lan- guage Processing, pp. 5177-5190, Singapore, December 2023c. Association for Computational Linguisties. doi: 10. 10.18653ww1/003.emmlpp main.m.............m...... URL https: / aclanthology org 2023 emm1pmmn1pppppp..mmmmmmmmmmm main. 314.
Ma, M. Zheng, B., Liu, K., Zheng, R., Liu, H., Peng, K., Church, K., and Huang, L. Incremen- tal text-to-speech synthesis with prefix-to-prefix frame- work. In Findings of the Association for Computa- tional Linguistics: EMNLP 2020, pp. 3886-3896, On-
Ma, Z., Fang, Q., Zhang, S., Guo, S., Feng, Y., and Zhang, M. A non-autoregressive generation framework for end- to-end simultancous speech-to-an, translation. In Pro- ceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume I: Long Pupers), pp. 1557-1575, Bangkok, Thailand, August 2024. As- sociation for Computational Linguistics. URL https: / //aclanthology or ptgg//2////0000 acl-long 85.
line, November 2020a. Association for Computational Linguistics. doi: 10. 10016/////222....nninnnngs-//
346. URL https: /atpa/caaannoooo..ooooooooo org/2020- Findings-emnip..5 346.
Ma, X., Dousti, M. J., Wang, C., Gu, J., and Pino, SIMULEVAL: An evaluation tookiit for simultaneous translation. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pp. 144-150, Online, October 2020b. Association for Computational Linguistics. doi: 10. 10.118663/11/00000mmmpppp 19 URL https:// ac aclanthology 2tgg//220000 wttp://220.emmmmmmeee www.p-demoss.pee.......
McAuliffe, M., Socolof, M., Mihuc, S., Wagner, M., and Sonderegger. M. Montreal forced aligner: Trainable text- speech alignment using kaldi. In Interspeech 2017, pp. 498-502, 2017, doi: 10. 40224437/neer9pecc221111113
Papi. S., Gaido, M., Negri, M., and Turchi, M. Over- generation cannot be rewarded: Length-adaptive average lagging for simultaneous speech translation. In Proceed- ings of the Third Workshop on Automatic Simultaneous Translation, pp. 12-17, Online, July 2022. Association for Computational Linguistics. doi: 10. 10.18633/1/2022. autosimtrans- URL https: a anthology. org/2022 htt.::22asauttsiitsss-11111
Ma, X., Pino, J., and Koehn, SimulMT to SimulST: Adapting simultaneous text translation to end-to-end si- multaneous speech translation. In Proceedings of the Ist Conference of the Asia-Pacifes Chapter of the Associa- tion for Computational Linguistics and the 10th Interna- tional Joint Conference on Natural Language Processing, pp. 582-587, Suzhou, China, December 2020c. Asso- ciation for Computational Linguistics. URL https: / //aclanthology. org/ 2020 saacc-mmann 58.
Papi, S., Negri, M., and Turchi, M. Attention as guide for simultaneous speech translation. In Proceedings of the 61st Annual Meeting of the Association for Com- putational Linguistics (Volume 1: Long Papers). pp. 13340-13356, Toronto, Canada, July 2023a. Associa- tion for Computational Linguistics. doi: 10.18653/v1/ 2023.acl-long 745. URL tettts::/aalanttiee org 2rg/2023. acl.Bacl--- acl-alngl..naoo...o 745.
Ma, X., Pino, M., Cross, J., Puzon, L., and Gu, Mono- tonic multihead attention. In International Conference on Learning Representations, 2020d. URL https: //openreview tame/fformm iid-idg99gBgKPP
Ma, X., Gong, H., Liu, D., Lee, A., Tang, Y., Chen, P.-.., Hsu, W.-N., Kochn, P., and Pino, J. Direct simultancous speech-to-spern translation with variational monotonic multihead attention, 2022.
Papi, S., Turchi, M., and Negri, M. AlignAtt: Us- ing Attention based Audio Traissatannltionn Alignments as a Guide for Simultancous Speech Translation. In Proc. INTERSPEECH 2023, pp. 3974-3977 2023b. doi: 00224437/neerpeechh2233
Ma, X., Sun, A., Ouyang, S., Inaguma, H., and Tomasello, P. Efficient monotonic multihead attention, 2023a. URL https: arxiv.aiiee org/ abs/ 2212..11222 04515.
Papineni, K., Roukos, S., Ward, T., and Zhu, W.-. Bleu: a method for automatic evaluation of machine transla- tion. In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, pp. 311- 318, Philadelphia, Pennsylvania, USA, July 2002. As- sociation for Computational Linguisties. doi: 10.3115/
Ma, Z., Shao, C., Gui, S., Zhang. M., and Feng. Y. Fuzzy alignments in directed acyclic graph for non- autoregressive machine translation. In International
10

---

Overcoming Non-monotonicity in mansducer-bbsse Streaming Generation
1073083.1073135, URL httpsss//cclaamhhl...o/oooo org/P02-1040.
Shaw, P., Uszkoreit, J., and Vaswani, A. Self-attention with relative position representations. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Lan- guage Technologies, Volume 2 (Short Papers), pp. 464 468, New Orleans, Louisiana, June 2018. Association for Computational Linguistics. doi: 10.18653/ wwtp://-///N//2/// URL https: /ttp:://aclaaa http://aaaaatttthc
Park, D. S., Chan, W., Zhang, Y., Chiu, C.-C., Zoph, B., Cubuk, E. D., and Le, Q. V. Specaugment: A simple data augmentation method for automatic speech recognition. Interspeech 2019, 2019.
Post, M. call for clarity in reporting BLEU scores. In Proceedings of the Third Conference Machine Translation: Research Papers, pp. 186-191, Belgium, Brussels, October 2018. Association for Computational Linguistics. URL https:/ / /t////www/ http:/wwwwwwwwe.eee/////.weeweca wwwwww.www.......o.......... org/ anthology / /W18 10118-6333 6319.
Shi, Y., Wang, Y., Wu, C., Yeh, C.-F., Chan, J., Zhang, F., Le. D., and Seltzer, M. Emformer: Efficient mem- ory transformer based acoustic model for low latency streaming speech recognition. In ICASSP 2021 202 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 6783-6787, 2021. doi: N0.1109/CASSP9977822211110
Prabhavalkar, R., Rao, K., Sainath, T. N., Li, B., Johnson, L., and Jaitly, N. A Comparison of Dequence-to-Sequene Models for Speech Recognition. In Proc. Interspeech 2017, pp. 939-943, 2017. doi: 10.214377Interspeech. 2017-233.
Tang, Y., Sun, A., Inaguma, H., Chen, X., Dong, N., Ma, X., Tomasello, P., and Pino, J. Hybrid transducer and attention based encoder-decoder modeling for speech-to- text tasks. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume Long Papers), pp. 12441-12455, Toronto, Canada, July 2023. Association for Computational Linguistics. doi: 00.18653/v1/20233 acl long.66.......... URL https: aclanthology wwwoorr////.. acl -oog 695.
Raffel, C., Luong, M.-T., Liu, P. J., Weiss, R. J., and Eck, D. Online and linear-time attention by enforcing monotonic alignments. In Proceedings of the 34th International Con-
ference on Machine Learning, volume 70 of Proceedings of Machine Learning Research, pp. 2837-2846. PMLR, 06-11 11 Aug 2017. URL https: /ttp:s//////ccednggs.// m....pres/. https//////aaffafaae/a// html.
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L. u., and Polosukhin, I. At- tention is all you need. In Advances in Neural Informa- tion Processing Systems, volume 30. Curran Associates, Inc., 2017. URL htw.h:tp/s:/eeee /ttss://axxvvoooo haagg//bs/iee 1706. 03762.
Seamless Communication. Barrault, L., Chung, Y.-A., Meglioli, M. C., Dale, D., Dong, N., Duppenthaler, M., Duquenne, P.-A., Ellis, B., Elsahar, H., Haaheim, J., Hoff- man, J., Hwang, M.-J., Inaguma, H., Klaiber, C., Kulikov, I., Li, P., Licht, D., Maillard, J.. Mavlyutov, R., Rako- toarison, A., Sadagopan, K. R., Ramakrishnam, A., Tran, T., Wenzek, G., Yang, Y., Ye, E.,, Evtimov, I., Ferman- dez, P., Gao, C., Hansanti, P., Kalbassi, E., Kallet, A., Kozhevnikov, A., Mejia, G., Roman, R. S., Touret, C., Wong, C., Wood, C., Yu, B., Andrews, P., Balioglu, C., Chen, P.-J., Costa-jussă, M. R., Elbayad, M., Gong, H., Guzmán, F., Heffernan, K., Jain, S., Kao, J., Lee, A., Ma, X., Mourachko, A., Peloquin, B., Pino, J., Popuri, S., Ropers, C., Saleem, S., Schwenk, H., Sun, A., Tomasello, P., Wang. C., Wang, J.. Wang. S., and Williamson. M. Seamless: Multilingual expressive and streaming speech translation. In ArXiv, 2023.
Wang. P., Sun, B., Xue, J., Wu, Y., Zhou, L., Gaur, Y., Liu, S., and Li, J. LAMASSU: A Streaming Language- Agnostic Multilingual Speech Recognition and Trans- lation Model Using Neural Transducers. In Proc. IN- TERSPEECH 2023, pp. 57-61, 2023. doi: 10.21437/ 2iersppech..22332000
Wang, 2., Chen, Y., Wang. X., Xie, L., and Wang. Y. Stream Voice: Streamable context-aware language mod- cling for real-time zero-shot voice conversion. In Pro- ceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume I: Long Papers), pp. 7328-7338 Bangkok, Thailand, August 2024. As- sociation for Computational Linguistics. URL https: //aclanthology org / 2024. acl.long. 396.
Seide, F., Doulaty, M., Shi, Y., Gaur, Y., Jia, J., and Wu, C. Speech reallm - real-time streaming speech recognition with multimodal Ilms by teaching the flow of time, 2024. URL httpt:ttt//// / rx aaaaaivivi..e. orgg abs / 2406.046000 09569.
Shao, C. and Feng, Y. Non-monotonic latent alignments for etc-based non-autoregressive machine translation. In Advances in Neural Information Processing Systems, vol- ume 35, pp. 8159-811 8173. Curran Associates, Inc., 2022. URL https: / rxxiv. org/ atw.///1.... 03953.
Wu, C., Wang, Y., Shi, Y., Yeh, C.-F. and Zhang, F. Streaming Transformer Bassed Acoustic Models Using Self-Attention with Augmented Memory. In Proc. In- terspeech 2020, pp. 2132-2136, 2020. doi: 10.21437/ Interspeech.2020) 2020-2079

---

Overcoming Non-monotonicity in Transducer- Streaming Generation
Xue, J., Wang, P., Li, J., Post, M., and Gaur, Y. Large-Scale Streaming End-to-End Speech Translation with Neural Transducers. In Proc. Interspeech 2022, pp. 3263-3267, 2022. doi: 0022445/mmmrspech2222--00955
Zeng, X., Li, L., and Liu, Q. RealTranS: End-to-end simul- taneous speech translation with convolutional weighted- shrinking transformer. In Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, pp. 2461- 2474, Online, August 2021. Association for Computa- tional Linguistics. doi: 10.18653/w1/2222.mnmnii 218. URL https: http///aannaoool..orgo////////// =indings-acl. 2188.
Zhang, Q., Lu, H., Sak, H., Tripathi, A., McDermott, E., Koo, S., and Kumar, S. Transformer transducer: A streamable speech recognition model with transformer encoders and mn-tt loss. In ICASSP 2020 2020 IEEE International Conference on Acoustics, Speech and Sig- nal Processing (ICASSP), pp. 7829-7833 2020. doi: 00.100//CASSPP476620000000030000
Zhang, R., He, Z., Wu, H., and Wang, Learning adap- tive segmentation policy for end-to-end simultaneous translation. In Proceedings of the 60th Annual Meet- ing of the Association for Computational Linguistics (Volume I: Long Papers), pp. 7862-7874, Dublin, Ire- land, May 2022. Association for Computational Linguis- tics. doi: 10. 10.18653//122222.acllooggg.. //aclanthology. wwwp://..r../..o..eo.o.. acl- aww.://22...lllonggg... 542.
URL https:
Zhang, S. and Feng. Y. Unified pegment-to-segment frame- work for simultaneous sequence generation. In Advances in Neural Information Processing Systems, volume 36, pp. 45235 45258. Curran Associates, Inc., 2023. URL atsp/bbs/2233 17940.
https: //trx/aaavvvv//a//////eee/eawav.
Zhang, S., Fang, Q., Guo, S., Ma, Z., Zhang, M., and Feng, Y. Streamspeech: Simultaneous speech-to-spect tp-ece transla- tion with multi-task learning. Proceedings of the 62th Annual Meeting of the Association for Computational Lin- guistics (Long Papers). Association for Computational Linguistics, 2024. URL https:/ httt:p::/aacaaathoto /aap//holaanttooy.... org/2024 acl wecccllloe acllllng.....a..... 485/.
Zhao, J.. Moritz, N., Lakomkin, E., Xie, R., Xiu, Z., Zmo- likova, K., Ahmed, Z., Gaur, Y., Le, D., and Fuegen, C. Textless streaming specch-to-speech translation using mantic speech tokens. arXiv preprint arXiv:2410.0722 2024.
12

---

Overcoming Non-monotonicity in Transducer- based Streaming Generation
A. Implementation Details
Pre-processing The imput speech is represented as 80-dimensional log ael-filterbank coefficients computed every 10ms with a 25ms window. Global channel mean and variance normalization is applied to the input speech. During training, SpecAugment (Park et al., 2019) data augmentation with the LB policy is additionally employed. We use SentencePiece (Kudo& & Richardson, 2018) to generate unigram vocabulary of size 10000 for the source and target text jointly. Sequence- level knowledge distillation (Kim & Rush, 2016) is applied for fair comparison (Liu et al., 2021). For experiments, we resample the source audio to 16kHz and apply identical preprocessing steps as those used in speech-to-text experiments. For the target speech, we also downsample the audio and extract discrete units utilizing the publicly available pre-trained mHuBERT model and K-means quantizer: No training data manipulation applied speech-to-speech experiments.
Training Details Considering that training MonoAttn-Trannna Transducer involves two critical processes: inferring the posterior alignment and estimating the context vector, instability in either step can lead to training failure. Therefore, we introduce a curriculum learning strategy for MonoAttn-TTannee Transduerr. We first pretrain the model in an offline setting. In pretraining, all predictor states can attend to the complete source input, and the model is trained as an offline Transducer. This pretraining phase allows the monotonic attention module to warm up by learning full-sentence attention, thereby enhancing its stability during subsequent adaptation to a streaming scenario. In finetuning, we apply Algorithm with various chunk size configurations. During both training phases, set the dropout rate to 0.1, weight decay to 0.01, and clip gradient norms exceeding 5.0. The dropout rates for actívation and attention are both set to 0.1. The pretraining spans 50k updates with a batch size of 160k tokens. The learning rate gradually warms up to 5e-4 within 4k steps. Finetuning involves training for 20k updates and other hyper-parameters remain consistent. Throughout the training, we optimize models using the Adam optimizer (Kingma Ba, 2015). Automatic mixed precision training is applied. It takes approximately one day to pretrain an offline setting and another day for streaming adaptation on server with 4 Nvidia L40 GPUs.
adjust MonoAttn-Transduer
B. Baselines
We compare our proposed MonoAttn-Trannsuuer with the following state-of-the-art open-source approaches (without using pretrained encoder or any data augmentation method for fair comparison).
AED-based Models
Wait-k (Ma et al., 2020c): It executes wait-k policy (Ma et al., 2019) by setting the pre-decision window size to 280 ms.
RealTrans (Zeng et al., 2021): detects word number in the streaming speech by counting blanks in CTC transcription and applies wait-k-stride-n strategy accordingly.
MU-ST (Zhang et al., 2022): It trains an external segmentation model, which is then utilized to detect meaningful units for guiding generation.
Seg2Seg (Zhang & Feng, 2023): It alternates between waiting for a source segment and generating a target segment in an autoregressive manner.
EDAtt (Papi et al., 2023a): It calculates the attention scores towards the latest received frames of speech, serving as guidance for an offline-trained translation model during simultaneous inference.
AlignAtt (Papi al., 2023b): It exploits the attention information to generate source-target alignments that guide the model during inference. AlignAtt demonstrates superior performance for delays exceeding 2 seconds, whereas our study primarily focuses scenarios involving delays shorter than 2 seconds.
'https:/ / http:///ccoooteeeeeeeee//////aaa textless_s2st. te__reeealdddaamm_
haapeecchtt__seech//ooott speech/docs/
to
13

---

Overcoming Non-monotonicity in Transducer- based Streaming Generation
CTC-hased Models
NAST (Ma al., 2024): introduces a streaming generation model with fast computation speed by leveraging non- autoregressive transformer and CTC decoding (Graves et al., 2006).
Transducer-based Models
Transducer: It adopts the standard Transducer framework (Graves, 2012) and utilizes Transformer its backend network (Zhang al., 2020).
CAAT (Liu et al., 2021): It incorporates cross-attention module within Transducer's joiner to alleviate its strong monotonic constraint.
LLM-based Models
LSG (Guo et al., 2025): It enables the LLM to devise streaming generation policy that balances latency and generation quality by referring a baseline policy.
C. Numerical Results
In addition Average Lagging (AL; Ma al., 2020c), we also incorporate Average Proportion (AP; Cho & Esipova, 2016). Differentiable Average Lagging (DAL; Arivazhagan al., 2019) and Length- Average Lagging (LAAL: Papi et al., 2022) metrics evaluate the latency. AL, DAL and LAAL all reported with milliseconds. The trade-off between latency and translation quality is attained by adjusting the chunk size C. The offinn results are obtained by setting the chunk size to be longer than any utterance in the dataset (C = oc). We use SvaulEval v1 11 4 for evaluation in all the experiments. The numerical results of MonoAttn- Transducer are presented in Table and 5. comparison of the computation-awaat latency metrics for AL and LAAL between the Transducer and MonoAttn- MonoAttn-Transduec Transducer models is presented in Table 6.
Table 4. Numerical results of Tranducn-r on MuST-C English to German dataset.
MonoAttn- MonoAttn-Transducer on En >De
C(ms)
AP
AL
DAL
LAAL
BLEU
320
0.67
1215
1497
1317
20.22
640
0.77
1470
1872
1582
22.47
960
0.83
1860
2309
1957
22.94
1280
0.86
2215
2719
2305
23.74
00
24.42
Table Numerical results MonoAttn Transducer on MuST-C English to Spanish dataset.
MonoAttn-Transducer En =Es
C(ms)
AP
AL
DAL
LAAL
BLEU
320
0.74
997
1534
1230
24.72
640
0.81
1239
1854
1475
26.74
960
0.88
1606
2304
1837
27.05
1280
0.93
1991
2725
2204
27.41
8
27.48
14

---

Overcoming Non-monotonicity in Transducer- based Streaming Generation
Table Comparison of MonoAttn-Tramene and Transducer various chunk size settings MuST-C English to German and
English to Spanish datasets.
En-Es
En-De
Chunk Size (ms)
320
640
960
1280
320
640
960
1280
AL (ms;)) 4)
886
1193
1591
1997
1126
1434
1830
2215
AL_CA (ms,44 4)
1121
1330
1699
2085
1323
1551
1920
2296
Transducer
LAAL (ms,))
1168
1466
1847
2220
1258
1563
1942
2312
LAAL-CA (ms,)) 4)
1381
1589
1944
2300
1444
1673
2028
2389
AL (ms, 4)
997
1239
1606
1991
1215
1470
1860
2215
ALCCA ()
1239
1385
1724
2089
1407
1596
1964
2301
MonoAttn- TranssducTTe
LAAL (m8, +)
1230
1475
1837
2204
1317
1582
1957
2305
LAAL CA (ms,4)
1453
1607
1945
2295
1501
1702
2056
2387
Table Comparison results reported Tang et al. (2023) and our work on MuST-C English to German dataset.
Chunk Size (m.s)
160
320
480
640
Transducer (Tang et al., 2023)
BLEU (t)
20.76
21.80
22.52
23.32
AL (ms, 4)
1282
1252
1306
1498
TAED (Tang et al., 2023)
BLEU (1)
21.57
22.63
23.48
23.47
AL (ms,
1263
1354
1369
1903
Chunk Size (ms)
320
640
960
1280
Transducer (Our implementation)
BLEU (1)
19.99
20.10
22.20
22.96
AL (ms, 4)
1126
1434
1830
2215
BLEU (1)
20.22
22.47
22.94
23.74
Trannuuure
AL (ms, 4)
1215
1470
1860
2215
D. Visualization
In this section, we present more examples of diagonal prior and its posterior from training corpus. We have observed that, with significant differences in the prior distribution, the posterior remains fairly robust when the chunk size is constant. The vertical axis represents the target subword sequence and the horizontal axis represents the speech waveform. Darker areas indicate higher alignment probabilities. We use Montreal Forced Alignment tools (McAuliffe et al., 2017) to obtain speech-transere trassrptranssiipttion alignments for illustration.
15

---

Overcoming Non-monotonicity in Transducer- based Streaming Generation
Diagonal Prior Allgnment
Tambión
entander
Tomando
decisiones
manera
que
hos
suguro
Posterior Alignment
<5>
eslabamos
Jecisiones
de
Uina
hos
saguro
wanted
understand
decisions
that
made:
Safer
Figure Chunk size in this example to 320ms. (Diagonal Prior)
Diagonal Prior Alignment
Taabbinn
queria
emtander
Tomando
Jackslanes
mänera
que
NClers
guru
Posterior Alignment
<t>
Tambien
eetender
Si
estibamos
Jecisianes
que
hos
hilees
Augoro
also
wanted
understand
If
making
decisions
that made:
safer
Figure 4. Chunk size this example set to 640ms. (Diagonal Prior)
16

---

Overcoming Non-monotonicity in Transducer- based Streaming Generation
Diagonal Prior Allgnment
Tambión
enhander
Tomando
decisiones
manera
que
hos
seguuo
Posterior Alignment
<t>
También
estabamos
Jecisiones
de
Uina
hos
saguro
wanted
understand
decisions
that
made:
Safer
Figure 5. Chunk size in this example to 960ms. (Diagonal Prior)
Diagonal Prior Alignment
Tambbinn
queria
emtander
Tomando
Jackslanes
mänera
que
NClers
guguro
Posterior Alignment
<t>
Tambien
eetender
Si
estibamos
Jecisianes
que
hos
hilees
Augoro
also
wanted
understand
If
making
decisions
that made:
safer
Figure Chunk size this example set 1280ms. (Diagonal Prior)
17


Reviews:
[{"review_id": "6e9Z4z04S4", "reviewer": "Reviewer_zJy7", "summary": "This paper proposes Mono-Attn-Transducer, a streaming sequence model that combines Transducer and monotonic attention. A novel training procedure that utilizes approximate alignment posteriors and alignment priors made training possible without expensive enumeration over an exponential search space or the use of a latency loss. Experimental results demonstrated strong performance on standard speech-to-text and speech-to-speech translation tasks.\n\n## Update after rebuttal\n\nApologies for the delayed response. I have been sick the past 2 weeks.\n\nThanks for clarifying on the CAAT results! That addresses one of my main concerns with this paper. However, I am afraid the other concern (unclear presentation of the model architecture) is still valid. Without major revisions to the current draft, it would be very difficult for readers to grasp even just the high level architecture without looking at the code. For that reason, I could only raise my score to 2.", "questions": "", "limitations": "", "rating": 2, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**Thank you for your efforts in reviewing! We will make every effort to respond to your concerns.**\n\n>***1.However I would need some more information from the authors before I could make a good assessment.***\n\n**We will address your questions about the method point by point.**\n\n1. >*It appears that the overall architecture of Mono-Attn-Transducer is very similar to TAED except that previous decoder states are not updated when new source features become available. This however is only my best guess.*\n    \n    Yes, in the inference of Mono-Attn-Transducer, the generated predictor states are not updated when receiving new source features. When the predictor encodes the $u$-th target state, it depends on previous predictor states and the currently available source:\n        \n    $s_{u} = f_{\\theta}(s_{0:u-1},h_{1:g(u)},y_{u-1})$\n\n2. >*$c_u$ is the approximate average cross attention output based on my interpretation of Equation (8). However $c_u$ depends on $s_u$, the predictor state which in turn depends on the actual alignment path $g(\\cdot)$. It is not clear to me which alignment path is used to produce $s_u$.*\n   \n   In fact, the core idea of the approach lies in the uncertainty of the alignment path used to produce $s_u$ in training. Therefore, a posterior approximation is required to guide the learning process. To analyze this issue, we can further examine Eq. 4 in detail. In fact, the predictor producing $s_u$ primarily relies on two steps: \n\n    A. **Self-attention** with earlier predictor states ($s_{0:u-1}$): **This is deterministic.**\n\n    B. **Cross-attention** with encoder states ($h_{1:g(u)}$): Since the specific value of $g(u)$ is unknown in training, **we instead estimate the posterior alignment (Eq. 7) and use the resulting alignment probability to approximate the expected context representation $c_u$ in the cross-attention (Eq. 8).**\n\n3. >*Further, it is not clear to me how $c_u$ is used. My guess is that it's passed to the joiner to join with each $h_t$ to produce the Transducer sum of alignment path probabilities.*\n   \n   As mentioned earlier, $c_u$ represents the expected context representation (**in other words, $c_u$ is the output of the cross-attention module**). This $c_u$ is then passed to the next module in the predictor, typically an FFN. **We refer to the final output of the predictor as $s_u$** (Eq. 4), which is subsequently passed to the joiner.\n   \n>***2.Key details are missing that prevent the readers from fully understanding or reproducing Mono-Attn-Transducer.***\n\n**We hope the previous clarifications help better understanding. We also provide the source code in this [anonymous URL](https://github.com/RandomUsername192/AnnoymousMaterials) to ensure reproducibility.**\n\n   \n>***3.Concerns on Baseline Results***\n\n1. >*Clarification of CAAT results.*\n\n   We used the code provided by CAAT for replication but did not obtain the results reported in their paper. Since CAAT did not provide the distillation data used to train their model, we trained with the same data we used for training the Transducer, which could be the source of the discrepancy. The official CAAT config (audio_cat) does use fewer params; however, we argue that this is a compromise made due to its $O(T)$ memory usage in training.\n\n2. >*Clarification of S2S results.*\n   \n    In fact, Zhao et. al specifically augmented the speech synthesis module, while the Transducer remains unchanged. They use an acoustic LM to refine semantic tokens generated by Transducer and then generate the waveform. **However, their design is not directly related to the Transducer itself, but rather to improving waveform generation.** Our focus is on the Transducer, and we opt for a standard approach [1]: directly converting generated semantic tokens to waveforms using a vocoder.\n    \n    [1] Speech Resynthesis from Discrete Disentangled Self-Supervised Representations\n\n\n >***4.Comments on definition of $\\alpha(t,u)$ and $\\beta(t,u)$.***\n   \n   We noticed you argued that $\\alpha(t,u)$ is the sum of probabilities of partial alignments where $y_u$ being produced at $x_t$ is the last alignment label.\n\n**However, this is not correct. $\\alpha(t,u)$ does incorporate the probabilities where $y_u$ is followed by zero or more blanks. $\\alpha(t,u)$ is precisely defined as the probability of generating $y_{1:u}$ from $x_{1:t}$ in the original Transducer paper (see Section 2.4 of the paper). Similarly, $\\beta(t,u)$ is the probability of outputting $y_{u+1,U}$ from $x_{t:T}$. There is no need for $y_u$ being produced at $x_t$**.\n\n>***5.Suggestions on Equation (4) and Section 3.2.1.***\n   \nThank you! We will fix it.\n\n>***6.Comments on Equivalence between monotonic attention and Transducer.***\n \nWe quite agree with this statement. Our approach actually leverages the DP algorithm of the Transducer to assist in the computation of monotonic attention. We will include Variani et al. (2020) in the discussion of the paper. Thank you for the reminder.", "claims_and_evidence": "The main claim of this paper is the strong performance of the proposed method for streaming speech-to-text and speech-to-speech translation. To support this claim, this paper presents\n-   Results on streaming speech-to-text translation in Table 2 and Figure 2\n-   Results on streaming speech-to-speech translation in Table 3\n\nThe results were on standard datasets on these tasks, enabling comparison across a wide array of existing results.\n\nHowever, I have a few concerns:\n-   The results of CAAT in Figure 2 are quite a bit worse than what's reported in [Liu et al, 2021]. There, Tables 5 & 6 report the following BLEU/AL for En-De and En-Es for CAAT:\n\n|En-De|AL(ms)|BLEU|\n|-|-|-|\n||508.1|20.5|\n||813.8|21.4|\n||1114.9|21.8|\n||1443.4|22.2|\n||1800.6|22.4|\n||2137.8|22.6|\n||Offline|23.2|\n\n|En-Es|AL(ms)|BLEU|\n|-|-|-|\n||355.9|24.0|\n||623.2|25.8|\n||955.9|26.3|\n||1275.9|26.4|\n||1647.7|26.6|\n||1977.3|27.1|\n||Offline|27.5|\n\nIt's unclear whether the CAAT results in Figure 2 of this paper were reproduced by the authors themselves. What were reported in [Liu et al, 2021] are significantly better and much closer to results of Mono-Attn-Transducer in Table 2. If I am not mistaken, [Liu et al, 2021] used both smaller models and a smaller right context in their experiments. Thus a clarification on the discrepancy between the reported results would be extremely important for more reliably assessment on the actual contribution of Mono-Attn-Transducer.\n\n-   The speech-to-speech translation results were only compared against the authors' own offline model, not even against [Zhao et al, 2024] which was cited in Section 5.4 (which reported much higher offline BLEUs). Admittedly this paper reported BLEUs under much lower latency than [Zhao et al, 2024], it would useful if more comparable results can be reported.\n\n[Liu et al, 2021]: https://aclanthology.org/2021.emnlp-main.4.pdf\n[Zhao et al, 2024]: https://arxiv.org/pdf/2410.03298", "methods_and_evaluation_criteria": "The proposed Mono-Attn-Transducer appears to be a sensible solution to streaming sequence prediction problems. However I would need some more information from the authors before I could make a good assessment, because several key details of the proposed method appear to be have been left out:\n-   There is not a clear description of the overall architecture of Mono-Attn-Transducer. From the context, it appears that the overall architecture of Mono-Attn-Transducer is very similar to TAED in [Tang et al, 2023] except that previous decoder states are not updated when new source features become available. This however is only my best guess.\n-   According to Algorithm 1, the training loss is the negative log-likelihood (from Equation (3)) based on $c_u$. $c_u$ is the approximate average cross attention output based on my interpretation of Equation (8). However $c_u$ depends on $s_u$, the predictor state which in turn depends on the actual alignment path $g(\\cdot)$. It is not clear to me which alignment path is used to produce $s_u$.\n-   Further, it is not clear to me how $c_u$ is used. My guess is that it's passed to the joiner to join with each $h_t$ to produce the Transducer sum of alignmenth path probabilities.\n\nIn summary, I am not confident that anyone with the information currently available in this paper can reproduce Mono-Attn-Transducer.\n\n[Tang et al, 2023]: https://aclanthology.org/2023.acl-long.695.pdf", "theoretical_claims": "There are no theoretical claims in this paper.", "experimental_designs_or_analyses": "The experimental designs are sound and widely accepted practice.", "relation_to_broader_scientific_literature": "This paper is a continuation in the exploration of enhancing a Transducer model with cross attention in streaming sequence prediction. CAAT ([Liu et al, 2021]) and TAED ([Tang et al, 2023]) are the two existing papers most closely related to this paper. The key technique in this paper draws inspiration from the line of work on monotonic attention (such as [Raffel et al, 2017] and [Arivazhagan et al, 2019])\n-   CAAT separates self attention and cross attention layers in the Transformer decoder, using the stack of self attention layers as the Transducer predictor to encode the output history without any alignment information, and the stack of cross attention layers as the joiner. As a result, the standard dynamic programming algorithm for computing the sum of alignment path probabilities in a Transducer lattice is still applicable. Because waiting until the end of source gives the joiner the most complete source information, a latency loss is necessary in CAAT training to prevent degenerate model behavior.\n-   TAED took a different path and kept the Transformer decoder intact as the Transducer predictor. At each time step $t$, the Transformer decoder states are completely recomputed with all the source features available so far ($h_{1:t}$). As a result, the standard dynamic programming algorithm for computing the sum of alignment path probabilities in a Transducer lattice is also applicable at the cost additional Transformer decoder computation. An AED loss term is introduced to, ostensibly, prevent degenerate model behavior similar to CAAT.\n-   Mono-Attn-Transducer took the average cross attention feature technique commonly employed in monotonic attention ([Raffel et al, 2017] and [Arivazhagan et al, 2019]) to approximate the sum of alignment path probabilities, with a novel alignment prior mechanism and a two-stage estimation. The strong alignment priors ensure that training would not lead to degenerate model behavior.\n\n[Raffel et al, 2017]: https://proceedings.mlr.press/v70/raffel17a/raffel17a.pdf\n[Arivazhagan et al, 2019]: https://arxiv.org/pdf/1906.05218\n[Tang et al, 2023]: https://aclanthology.org/2023.acl-long.695.pdf\n[Liu et al, 2021]: https://aclanthology.org/2021.emnlp-main.4.pdf", "essential_references_not_discussed": "None.", "other_strengths_and_weaknesses": "Strengths\n-   The novel training method is effective and elegant in not needing a latency loss.\n-   The reported speech-to-text translation results are strong even though the comparison with CAAT/TAED is inconclusive.\n\nWeaknesses\n-   Key details are missing that prevent the readers from fully understanding or reproducing Mono-Attn-Transducer.", "other_comments_or_suggestions": "-   Euqation (1) is inaccurate and misleading for readers not familiar with Transducers. $\\alpha(t, u)$ and $\\beta(t, u)$ are marginal probabilities over alignment paths, not probabilities over the output prefix/suffix without blanks,\n    -   $\\alpha(t, u)$ is the sum of probabilities of partial alignment paths where $y_u$ being produced at $x_t$ is the last alignment label. $p(y_{1:u} | x_{1:t})$ is a confusing notation that strictly speaking includes probabilities of alignment paths where $y_u$ is produced anywhere between $x_1$ to $x_t$, followed by zero or more blanks.\n    -   $\\beta(t, u)$ is the sum of probabilities of partial alignment paths starting right after $y_u$ being produced at $x_t$. $p(y_{u+1:U} | x_{t:T})$ thus has the same problem as $p(y_{1:u} | x_{1:t})$. Additionally, $x_i$ are the source input features, instead of the encoder features $h_t$, so $\\beta(t, u)$ depends on $x_{1:T}$ regardless of $t$ because $h_t$ in most unidirecitonal encoders can depend on the entire $x_{1:t}$, thus $\\beta(t, u)$ depends on the entire $x_{1:T}$.\n-   Equation (4) should probably include $y_{u-1}$ on the right hand side similar to Equation (5) of [Tang et al, 2023].\n-   Section 3.2.1 should emphasize that the alignment posterior is approximate.\n-   (230-235, left column): Bernoulli variables in monotonic attention and blanks in Transducer are mathematically equivalent. Standard Transducer simply folds the Bernoulli variable into a multi-class prediction. A variant of Transducer [Variani et al, 2020] actually uses a separate Bernoulli variable. Conversely, Transducer's dynamic programming algorithm can also be used to compute the marginals in monotonic attention (and is often one or two orders of magnitude faster than the so-called \"efficient\" algorithm in [Ma et al, 2023b]).\n\n[Tang et al, 2023]: https://aclanthology.org/2023.acl-long.695.pdf\n[Variani et al, 2020]: https://arxiv.org/pdf/2003.07705\n[Ma et al, 2023b]: https://arxiv.org/abs/2312.04515", "questions_for_authors": "As outlined above, my main concerns are the comparison with CAAT, and the lack of key architecture details.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "cyuHw6bcKZ", "reviewer": "Reviewer_ntmp", "summary": "This paper introduces MonoAttn-Transducer to tackle streaming generation tasks. MonoAttn-Transducer adds monotonic attention mechanism on top of Transducer and uses forward-backward algorithm to infer the alignment, which is then used to compute expected context representations used in monotonic attention. Experiments on simultaneous speech-to-text/speech translation (MuST-C and CVSS-C datasets) show that MonoAttn-Transducer performs better than vanilla Transducer when the chunk size of speech encoder is no less than 640ms.\n\n## update after rebuttal\n\nThe authors have addressed my concerns regarding LAAL and the prior distribution. However, while they claim to have tested AlignAtt, no results were provided in the rebuttal. Additionally, I share Reviewer zJy7’s concern about the quality of the writing. Therefore, I will maintain my current score.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**Thank you for your thoughtful review! We will make every effort to respond to your concerns.**\n\n>***1. Concerns regarding latency metric that are biased toward over-generation:*** *The AL in Table 3 is abnormal, both 118 and 153 ms are abnormally low, which I find hard to believe. One possible cause is over-generation. The average lagging (AL) evaluation metric is problematic, since AL is not reliable if the over-generation happens. It is better to use length adaptive average lagging (LAAL).*\n\nThank you for your suggestion. Actually, we present the S2T results using LAAL in Table 6, App. C. Apologies for the confusion. For clarity, we have included the results from Table 6 here and added the S2S results using LAAL and Start Offset.\n\n**L: EN-ES R: EN-DE**\n| | **Chunk Size (ms)** | 320  | 640  | 960  | 1280 | 320  | 640  | 960  | 1280 |\n|-|-|-|-|-|-|-|-|-|-|\n| **Transducer** | **LAAL (ms)**|1168|1466|1847|2220|1258|1563|1942|2312|\n| |**LAAL_CA (ms)**|1381|1589|1944|2300|1444|1673|2028|2389|\n|**MonoAttn-Transducer**|**LAAL (ms)**|1230|1475|1837|2204|1317| 1582|1957|2305|\n| |**LAAL_CA (ms)**|1453|1607|1945|2295|1501|1702|2056|2387|\n\n**FR-EN S2S**\n||**Chunk Size (ms)**|320|\n|-|-|-|\n| **Transducer**| **AL (ms)**|153|\n||**LAAL (ms)**|984|\n||**Start Offset (ms)**|1520|\n| **MonoAttn-Transducer**|**AL (ms)**|118|\n||**LAAL (ms)**|918|\n||**Start Offset (ms)**|1491|\n\n**We have realized that LAAL is a more robust metric for assessing latency (especially in S2S), and we will replace the metrics in the main body of the paper with LAAL for clearer presentation.**\n\n\n>***2.One concern is equation (11). It is possible to obtain a better prior by either leveraging the confidence of pretrained MT models or word alignment methods. Another concern is that the expected contextual representations still leads to training-inference mismatch.***\n\nLeveraging the confidence of pretrained MT models or word alignment methods for prior design is a possible approach. However, it will further complicate the training pipeline. In practice, we find that the posterior alignment is relatively robust to the choice of prior. Specifically, we examine the impact of different prior choices: diagonal prior and uniform prior.\n\n\n|                           | **Chunk Size ($ms$)** | 320   | 640   | 960   | 1280  |\n|---------------------------|-----------------------|-------|-------|-------|-------|\n| **$p^{\\mathrm{dia}}$**    | **BLEU**  | 24.72 | 26.74 | 27.05 | 27.41 |\n|                           | **AL ($ms$)**| 997   | 1239  | 1606  | 1991  |\n| **$p^{\\mathrm{uni}}$**    | **BLEU**  | 24.89 | 26.68 | 27.26 | 27.11 |\n|                           | **AL ($ms$)**| 993   | 1249  | 1601  | 1983  |\n\n\nAs shown, MonoAttn-Transducer's performance demonstrates robustness to the choice of prior alignment. **We visualize the posterior alignment when using different priors in this [annoymous URL](https://github.com/RandomUsername192/AnnoymousMaterials/blob/main/Comparison_cut.pdf). We have observed that, even with significant differences in the prior distribution, the posterior remains fairly robust.** This nice property relieves concerns about the imperfect design of the prior.\n\n\n>***3.The author misses one important baseline AlignAtt [1], which is far better than EDAtt compared in Figure 2.***\n\nThank you for the reminder. I have checked AlignAtt's paper and found that it performs particularly well at medium-to-high level latency. We will include it in Fig. 2 to facilitate a comparison between different streaming generation methods.\n\n>***4.Can you elaborate on why MonoAttn-Transducer does not improve over Transducer at chunk size of 320ms?***\n\nOn one hand, when the chunk size is 320ms, the Transducer's prediction window is very small, which limits its flexibility in managing reordering. On the other hand, we found that the small improvement observed with a 320ms chunk size is mainly reflected when using BLEU as a metric. However, results on COMET show that MonoAttn-transducer still demonstrates significant improvement.\n\n>***5.Could you please also report speech offset for simultaneous speech-to-speech translation?***\n\nThank you for your suggestion. We have incorporated the results into the table above.", "claims_and_evidence": "> Claim 1: vanilla Transducer have limited ability to attend to the input stream history during decoding, making it hard to manage reorderings. MonoAttn-Transducer manages re-ordering better with explicit monotonic attention.\n\nThis claim is true when the chunk size is large or the there is no more than 1 word reorder as shown in Figure 2. It is not true when the chunk size is small and there are more than 1 word reorders. \n\n> Claim 2: an efficient training algorithm is proposed to avoid direct handling exponentially large number of translation trajectories. \n\nThis claim is true as shown in Table 2 and Section 6.2.", "methods_and_evaluation_criteria": "The method itself looks complicated at first glance, but the intuition behind is clear. One concern is equation (11). It is possible to obtain a better prior by either leveraging the confidence of pretrained MT models or word alignment methods as in [1]. Another concern is that the expected contextual representations still leads to training-inference mismatch. \n\nThe average lagging (AL) evaluation metric is problematic, since AL is not reliable if the over-generation happens. It is better to use length adaptive average lagging (LAAL) [2] as in recent IWSLT workshops [3]. \n\n[1] Wang, M., Vu, T. T., Wang, Y., Shareghi, E., & Haffari, G. (2024). Conversational simulmt: Efficient simultaneous translation with large language models. arXiv preprint arXiv:2402.10552.\n\n[2] Sara Papi, Marco Gaido, Matteo Negri, and Marco Turchi. 2022. Over-Generation Cannot Be Rewarded: Length-Adaptive Average Lagging for Simultaneous Speech Translation. In Proceedings of the Third Workshop on Automatic Simultaneous Translation, pages 12–17, Online. Association for Computational Linguistics.\n\n[3] Ibrahim Said Ahmad, Antonios Anastasopoulos, Ondřej Bojar, Claudia Borg, Marine Carpuat, Roldano Cattoni, Mauro Cettolo, William Chen, Qianqian Dong, Marcello Federico, Barry Haddow, Dávid Javorský, Mateusz Krubiński, Tsz Kin Lam, Xutai Ma, Prashant Mathur, Evgeny Matusov, Chandresh Maurya, John McCrae, Kenton Murray, Satoshi Nakamura, Matteo Negri, Jan Niehues, Xing Niu, Atul Kr. Ojha, John Ortega, Sara Papi, Peter Polák, Adam Pospíšil, Pavel Pecina, Elizabeth Salesky, Nivedita Sethiya, Balaram Sarkar, Jiatong Shi, Claytone Sikasote, Matthias Sperber, Sebastian Stüker, Katsuhito Sudoh, Brian Thompson, Alex Waibel, Shinji Watanabe, Patrick Wilken, Petr Zemánek, and Rodolfo Zevallos. 2024. FINDINGS OF THE IWSLT 2024 EVALUATION CAMPAIGN. In Proceedings of the 21st International Conference on Spoken Language Translation (IWSLT 2024), pages 1–11, Bangkok, Thailand (in-person and online). Association for Computational Linguistics.", "theoretical_claims": "No theoretical claims.", "experimental_designs_or_analyses": "Issues:\n1. The author misses one important baseline AlignAtt [1], which is far better than EDAtt compared in Figure 2. \n2. The AL in Table 3 is abnormal, both 118 and 153 ms are abnormally low, which I find hard to believe. One possible cause is over-generation. \n\n[1] Papi, S., Turchi, M., Negri, M. (2023) AlignAtt: Using Attention-based Audio-Translation Alignments as a Guide for Simultaneous Speech Translation. Proc. Interspeech 2023, 3974-3978, doi: 10.21437/Interspeech.2023-170", "relation_to_broader_scientific_literature": "The primary contribution of this work is the integration of monotonic attention [1] into the Transducer framework, which overcomes the limitations of the standard Transducer in handling the word reordering challenges inherent in simultaneous translation.\n\n[1] Naveen Arivazhagan, Colin Cherry, Wolfgang Macherey, Chung-Cheng Chiu, Semih Yavuz, Ruoming Pang, Wei Li, and Colin Raffel. 2019. Monotonic Infinite Lookback Attention for Simultaneous Machine Translation. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 1313–1323, Florence, Italy. Association for Computational Linguistics.", "essential_references_not_discussed": "The paper claims to compare with previous state-of-the-art methods, but missing AlignAtt [1].\n\n[1] Papi, S., Turchi, M., Negri, M. (2023) AlignAtt: Using Attention-based Audio-Translation Alignments as a Guide for Simultaneous Speech Translation. Proc. Interspeech 2023, 3974-3978, doi: 10.21437/Interspeech.2023-170", "other_strengths_and_weaknesses": "No.", "other_comments_or_suggestions": "When you mention the energy $e_{u,t}$, it is better to describe the meaning of it. Otherwise, it will be hard to understand for audience who did not read the monotonic attention paper.", "questions_for_authors": "1. Could you please report the results with LAAL and include AlignAtt as the baseline? It will be interesting to see how MonoAttn-Transducer performs compared with a strong baseline under a more robust latency metric. \n2. Could you please also report speech offset for simultaneous speech-to-speech translation? \n3. Can you elaborate on why MonoAttn-Transducer does not improve over Transducer at chunk size of 320ms?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "prg4i27bW5", "reviewer": "Reviewer_MRT8", "summary": "This paper investigates an interesting problem of non-monotonic alignment between input/output in streaming generation settings (e.g., simultaneous translation). The solution is to use the forward-backward algorithm to estimate alignment. Results show superior performance on speech-to-text and speech-to-speech simultaneous translation tasks compared with baseline techniques.\n\n## Update after rebuttal\n\nWhile I appreciate the contributions in this work, I share ntmp zJy7's concerns over the clarity of the methodology description. My question and the author's response made me realize that important pieces of technical details are missing. I would still like to keep my score, but would not argue for a accept for this reason.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**Thank you for your acknowledgement of this work. We will make every effort to address your remaining concerns.**\n\n>***1. For Table 2., why would MonoAttn still out-perform Transducer with an infinite chunk size? I would anticipate that in such settings, alignment is not as important, as the context includes everything that may be relevant for genderating output?***\n\nYes, when the chunk size is infinite, the posterior alignment would be that all predictor states align with the last encoder state with a probability of 1. This means that the context in the cross-attention would encompass the entire source. However, it is important to note that the naive Transducer does not include such an attention mechanism. Instead, its encoder and predictor are loosely coupled through a joiner (typically a simple MLP), which may limit its modeling capacity compared to our approach.\n\n\n\n>***2.This paper investigates a less-studied problem of non-monotonic alignment, which is a true problem in real-world applications. For the same reason, the paper is limited in that its impact is limited to these specific areas of real world problems.***\n\nYes, the scope of this paper is limited to streaming sequence generation scenarios, specifically where the target sequence consists of discrete tokens. Given that an increasing number of studies have pointed out that data from audio, images and video modalities can be tokenized into discrete sequences in a manner similar to text [1,2], we believe this research can be widely applied to various real-time streaming tasks.\n\n[1] SpeechTokenizer: Unified Speech Tokenizer for Speech Large Language Models  \n[2] Finite Scalar Quantization: VQ-VAE Made Simple\n\n>***3.Would recommend adding a reference for forward/backward algorithm.***\n\nThank you for your suggestion! We will fix it.", "claims_and_evidence": "Yes, the claims are reasonably supported by experimental evidence.", "methods_and_evaluation_criteria": "Yes. The evaluation focuses on average latency, and standard machine translation metrics (BLEU, COMET), which seems befitting to the problem of speech to speech/text translation.", "theoretical_claims": "N/A", "experimental_designs_or_analyses": "Yes, the experimental design is reasonable. There is a small concern that this technique uses more compute than baseline (30% more), but I think this is largely acceptable given the notable improvements from baseline.\n\nResults analysis also seems reasonable.", "relation_to_broader_scientific_literature": "This paper makes several connections to the brader literature.\n- Extensive disucssion of transducer and its related work in Section 4.\n- Extensive discussion of related prior arts in  Section 5.3, including Wait-k, RealTrans, etc...", "essential_references_not_discussed": "Would recommend adding a reference for forward/backward algorithm.", "other_strengths_and_weaknesses": "This paper investigates a less-studied problem of non-monotonic alignment, which is a true problem in real-world applications. For the same reason, the paper is limited in that its impact is limited to these specific areas of real world problems.", "other_comments_or_suggestions": "N/A", "questions_for_authors": "For Table 2., why would MonoAttn still out-perform Transducer with an infinite chunk size? I would anticipate that in such settings, alignment is not as important, as the context includes everything that may be relevant for genderating output?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {"ethical_review_concerns": "N/A"}}, {"review_id": "rsGD4jevU7", "reviewer": "Reviewer_6uKj", "summary": "* The paper introduces MonoAttn-Transducer, which enhances the handling of non-monotonic alignments in streaming generation by incorporating monotonic attention.\n* The approach leverages the forward-backward algorithm to infer posterior alignment probabilities, enabling efficient training without enumerating exponentially large alignment spaces.\n* Extensive experiments demonstrate that the proposed method significantly improves generation quality while maintaining comparable latency to baseline Transducer models.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**Thank you for your thoughtful review! We will make every effort to respond to your concerns.**\n\n>***1. However, the paper could be improved by providing more detailed theoretical analysis of why the proposed method works well, especially in comparison to other methods. Additionally, while the experimental results are convincing, more comprehensive ablation studies could strengthen the paper's claims.***\n\nThank you for your suggestion. In the rebuttal period, we have designed an additional analysis to validate the rationale of our approach.\n\nThe core technical aspect of this paper is dynamically computing the expected contextualized representation of monotonic attention through inference of the posterior alignment during training. This posterior alignment requires a prior alignment for calculation. **Naturally, there are two potential questions here: 1) Is posterior alignment absolutely necessary, or can the expected contextualized representation be computed directly using a prior alignment? 2) How does the choice of prior alignment affect the result of the posterior alignment, and is it robust?**\n\nFor the first question, we attempted to directly calculate the expected contextualized representation using the prior probability, without inferring the posterior alignment. In Table 2, we present the results obtained by using a diagonal prior for this direct calculation. We found that this approach leads to a significant performance drop, particularly when the chunk size is small, where finer alignment is required. This addresses the first question. **Here, we will focus on the second question: the robustness of the posterior alignment.** We examine the impact of different prior choices: diagonal prior and uniform prior.\n\n|                           | **Chunk Size ($ms$)** | 320   | 640   | 960   | 1280  |\n|---------------------------|-----------------------|-------|-------|-------|-------|\n| **$p^{\\mathrm{dia}}$**    | **BLEU**  | 24.72 | 26.74 | 27.05 | 27.41 |\n|                           | **AL ($ms$)**| 997   | 1239  | 1606  | 1991  |\n| **$p^{\\mathrm{uni}}$**    | **BLEU**  | 24.89 | 26.68 | 27.26 | 27.11 |\n|                           | **AL ($ms$)**| 993   | 1249  | 1601  | 1983  |\n\n\nAs shown, MonoAttn-Transducer's performance demonstrates robustness to the choice of prior alignment. **We visualize the posterior alignment when using different priors in this [annoymous URL](https://github.com/RandomUsername192/AnnoymousMaterials/blob/main/Comparison_cut.pdf). We have observed that, even with significant differences in the prior distribution, the posterior remains fairly robust.** This nice property reinforces the robustness of using the inferred posterior to train monotonic attention.\n\n>***2.Could the proposed method be applied to other types of streaming generation tasks beyond speech translation, such as real-time text summarization or dialogue generation?***\n\nYes, the proposed method is a general framework that can be applied to many streaming generation task, as long as the target sequence consists of discrete tokens. We note a recent trend in research indicating that audio, images, and even video data can be tokenized into discrete sequences with minimal information loss [1,2]. Therefore, we believe this research has broad applicability to various real-time streaming tasks.\n\n[1] SpeechTokenizer: Unified Speech Tokenizer for Speech Large Language Models  \n[2] Finite Scalar Quantization: VQ-VAE Made Simple\n\n\n>***3.How does the performance of MonoAttn-Transducer scale with larger datasets and more complex language pairs?***\n\nThank you for your suggestion. We are currently working on scaling our approach with a larger dataset (GigaST) to further evaluate its performance. However, due to time constraints during the rebuttal period, the experiment has not been completed. We plan to include the results in the revised version.\n\n\n>***4.What are the limitations of the current implementation, and what are the plans for future work to address these limitations?***\n\nCurrently, we have implemented the forward-backward algorithm and posterior inference using ***Numba***. However, we have observed that GPU usage during training occasionally is around 60%. We suspect that some inefficient operators in our implementation may be contributing to this. We believe that optimizing the implementation may benefit better scaling this approach.", "claims_and_evidence": "* The effectiveness of MonoAttn-Transducer in handling non-monotonic alignments is demonstrated through experiments on speech-to-text and speech-to-speech simultaneous translation tasks.\n* The paper provides detailed experimental results, including BLEU scores, COMET scores, and latency metrics (Average Lagging), showing consistent improvements over baseline Transducer models across various chunk size settings.\n* Ablation studies and comparisons with prior alignment methods highlight the importance of learning monotonic attention through posterior alignment.", "methods_and_evaluation_criteria": "* The integration of monotonic attention into the Transducer architecture addresses the specific challenge of input-synchronous decoding limitations.\n* The use of the forward-backward algorithm for posterior alignment inference maintains computational efficiency while enabling effective training.\n* Evaluation on standard benchmark datasets (MuST-C, CVSS-C) and standard metrics (BLEU, COMET, Average Lagging) ensures that the results are comparable and relevant to the streaming generation research community.", "theoretical_claims": "The paper does not include formal proofs for theoretical claims. However, the theoretical foundations of the proposed method are based on established concepts:\n\n* The use of the forward-backward algorithm for computing posterior probabilities is a well-known technique in hidden Markov models and sequence transduction.\n* The formulation of monotonic attention is consistent with previous work in neural machine translation and speech recognition.", "experimental_designs_or_analyses": "* The experiments include comprehensive comparisons with baseline models and state-of-the-art approaches.\n* Results across different latency conditions (chunk sizes) provide insight into the flexibility and robustness of MonoAttn-Transducer.\n* The evaluation on multiple datasets (MuST-C, CVSS-C) and different language pairs (En→De, En→Es) demonstrates the generalizability of the proposed approach.", "relation_to_broader_scientific_literature": "* The paper builds upon previous work in Transducer models and monotonic attention, addressing their limitations in handling non-monotonic alignments.\n* It contributes to the ongoing effort to improve the efficiency and effectiveness of streaming generation models, which is crucial for real-time applications such as simultaneous translation.\n* The proposed method complements other approaches, such as attention-based encoder-decoder models and non-autoregressive models, offering a robust solution for complex streaming generation tasks.", "essential_references_not_discussed": "Since I am not familiar with the relevant literature, I cannot be sure.", "other_strengths_and_weaknesses": "The paper's main strength is its innovative approach to overcoming non-monotonicity in Transducer-based streaming generation by tightly integrating the predictor states with the source history through monotonic attention. This is a significant advancement as it improves the model's ability to handle complex tasks requiring non-monotonic alignments without a notable increase in latency. Furthermore, the method maintains the same time and space complexity as the baseline Transducer model, making it efficient and scalable. \n\nHowever, the paper could be improved by providing more detailed theoretical analysis of why the proposed method works well, especially in comparison to other methods. Additionally, while the experimental results are convincing, more comprehensive ablation studies could strengthen the paper's claims.", "other_comments_or_suggestions": "* In some places, the text is a bit dense and could benefit from additional figures or diagrams to illustrate complex concepts.\n* The discussion of related work could be more detailed, especially regarding how the proposed method differs from and improves upon existing approaches.", "questions_for_authors": "* How does the performance of MonoAttn-Transducer scale with larger datasets and more complex language pairs? (If the authors can provide additional experimental results or analysis on this, it would strengthen the paper's claims about the method's significance and applicability.)\n* Could the proposed method be applied to other types of streaming generation tasks beyond speech translation, such as real-time text summarization or dialogue generation? \n* What are the limitations of the current implementation, and what are the plans for future work to address these limitations?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: This paper proposes Mono-Attn-Transducer, a streaming sequence model to address tasks requiring non-monotonic alignments, such as simultaneous translation. Three reviewers give positive ratings while one reviewer gives negative ratings with some questions, such as more information about the method is needed before making a good assessment, and key details are missing that prevent the readers from fully understanding or reproducing Mono-Attn-Transducer. While the authors have provided adequate information to answer the author's questions, the reviewer did not respond further. Overall, I think the response provided by the author is informative to address the concerns to some extent. Therefore, I recommend an acceptance to this paper.
Decision: Accept (poster)