AEQA-NAT : Adaptive End-to-end Quantization Alignment Training Framework for Non-autoregressive Machine Translation
Authors: Xiangyu Qu
guojing liu
Liang Li
Venue: ICML 2025 poster (2025)
Keywords: Machine Translation
Parallel decoding
Vector Quantization
Primary area: deep_learning->sequential_models_time_series

Abstract: Non-autoregressive Transformers (NATs) have garnered significant attention due to their efficient decoding compared to autoregressive methods. However, existing conditional dependency modeling schemes based on masked language modeling introduce a *training-inference gap* in NATs. For instance, while NATs sample target words during training to enhance input, this condition cannot be met during inference, and simply annealing the sampling rate to zero during training leads to model performance degradation. We demonstrate that this *training-inference gap* prevents NATs from fully realizing their potential. To address this, we propose an adaptive end-to-end quantization alignment training framework, which introduces a semantic consistency space to adaptively align NAT training, eliminating the need for target information and thereby bridging the *training-inference gap*.Experimental results demonstrate that our method outperforms most existing fully NAT models, delivering performance on par with Autoregressive Transformer (AT) while being 17.0 times more efficient in inference.

AEQA-NAT: Adaptive End-to-end Quantization Alignment Training Framework for Non-autoregresive Machine Translation
Xiangyu Qu
Guojing Liu
Liang Li 24
Abstract
37.5
CMLM
Non-autoregressive Transformers (NATs) have garnered significant attention due to their efficient decoding compared to autoregressive methods. However, existing conditional dependency mod- cling schemes based on masked language model- ing introduce a todining-inferene gap NATs. For instance, while NATs sample target words during training to enhance input, this condition cannot be met during inference, and simply an- nealing the sampling rate to zero during train- ing leads to model performance degradation. We demonstrate that this vents NATs from fully realizing their potential. To address this, we propose an adaptive end-to- end quantization alignment training framework, which introduces semantic consistency space to adaptively align NAT training, eliminating the need for target information and thereby bridg- ing the training- infference gap. Experimental re- sults demonstrate that our method outperforms most existing fully NAT models, delivering per- formance on par with Autoregressive Transformer (AT) while being 17.0 times more efficient in in- ference.
GLAT
35.0
DAT
32.5
25.0
22.5
29
30
31
32
33 34
Sampling Ratio
Figure I. An example illustrating the challenge of the training- inference gap faced by former (NAT) models. In this context, the x-axis represents the sampling rate of unmasked target tokens during the inference phase of the NAT model, which is consistent with the training phase. The y-axis denotes the translation quality. The dashed line indicates baseline performance the NAT model under original in- ference conditions.
gap pre-
Non-autoregressive Trans-
interdependicies among target tokens (Guo al., 2019; Xiao et al., 2023). Current research in NAT primarily explores two paradigms: iterative decoding. which refines transla- tions through multiple iterations to balance quality and effi- ciency (Ghazvininejad et al., 2020b; Saharia et al., 2020), and non-iterative decoding, which generates translations in single forward pass to maximize speed advantages (Gu Kong, 2021; An al., 2023).
1. Introduction
Non-auvoregresssvv Transformer (NAT, Gu et al. 2018) has emerged as a promising approach mitigate the high la- tency inherent Autoregressive Transformer (AT, Vaswani al. 2017), which stems from their sequential token-by- token decoding mechanism. While NAT achieves significant speedup by parallel decoding, it often suffers from trans- lation quality degradation due to insufficient modeling of
Conditional masked language modeling (CMLM) stands as one of the most effective approaches for enhancing target to- ken dependencies in NAT (Ghazvininejad et al., 2019; Shao & Feng, 2022; Huang et al., 2022d). This method explic- itly guides the model to learn the mapping from the source sentence X and observable target tokens Yohs to the masked target tokens Y Ymask by incorporating additional target words as part the input. Qian al. (2021) further advanced this explicit dependency modeling capability by introducing the Glancing Transformer (GLAT),, which adaptively adjusts the number of sampled target tokens, a technique that has been widely adopted (Bao et al., 2022; Huang et al., 2022c; Guo et al., 2023). However, this MLM-based training paradigm (X + Yous -> Y Ymask) creates a training-inferene gap, as
'Scoool of Cyber Science and Technology, Shandong Univer- sity *State Key Laboratory of Cryptography and Digital Economy Security, Shandong University 'Occan University China 'QCL. Correspondence to: Liang Li <li.liang@ sdu.cdu.en..
Proceedings of the ndnd International Conference Machine Learning, Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).

---

Adaptive End-to-end Quantization Alignment Training Framework for NAT
NAT inference requires predicting the complete translation (X > Y) rather than reconstructing partial tokens. While annealing the target token sampling rate to zero during train- ing might theoretically address this gap, GLAT demonstrates that doing so leads to significant performance degradation.
2.1. Preliminary
Non-autoregressiv Machine Translation The machine translation task can be formally defined as sequence- to-sequence generation problem, where the neural net- work generates a target sentence Y = {v1.82. V2."" .NN under the condition of a given source sentence X T2,, :. . :. Autoregressive Transformer (AT, Vaswani et al. 2017) factorize the translation probability as follows and maximize it with the cross-entropy loss
The state-of-the-art NAT model, Directed Acyclic Trans- former (DAT, Huang al. 2022c), optimizes translation paths by constructing directed acyclic graph, achieving performance comparable AT models without knowledge distillation. Nevertheless, DAT is not without its limita- tions: 1) its performance gains come at the expense of decoding speed, with speedups ranging from 14.0x to 7.0x (Huang et al., 2022c); and 2) despite employing GLAT training methods, DAT fails to fully completely bridge the gap between training and inference. Furthermore, em- pirical investigationns into the unified training and infer- target information is input during
N
LAT =
Σ
log log
where yi is predicted based the prefix y<i-
The Vanilla NAT (Gu et al., 2018) makes a conditional independent assumption where each token is independent of each other when X is given. Formally, we have
ence of NAT inference to prompt translation-reven significant improve- ments in tramslation quality for MLM-based NAT models, as shown in Fig 1. This indicates that eliminating the training- inference gap can significantly enhance model performance.
N NATT = Σ
log (og
Our main contributions are follows:
Our research has revealed a significant discrepancy between the training and inference of contemporary advanced NAT networks. Specifically, we've ve identi- fied that NAT fails to reach its full potential during inference.
All target tokens are generated in parallel through the con- ditional probability of X. During decoding, NAT lacks appropriate methods to restore correct inter-word dependen- cies.
The Conditional Masked Language Model Enhancing de- pendency modeling is based on the idea of masked language model (MLM), c.g., CMLM (Ghazvininejad et al., 2019), GLAT (Qian et al., 2021) and DAT (Huang et al., 2022c), which allows the model explicitly learn mapping from the source sequence X and the observable variables Yous (unmasked ground truth tokens) to the target sequence y
We propose an Adaptive End-to-End Quantization Alignment (AEQA) framework for NAT, which intro- duces a semantic consistency space to jointly optimize all model components without requiring additional tar- get information. This approach effectively eliminates the training- inference gap inherent in NAT systems.
-einingginferenne
Lmask
Σ yEYmerk
DogpolylYaasn X)
Experimental results demonstrate that our method achieves the fastest decoding speed among all NAT networks while attaining state-of-the-art performance across multiple translation directions. Additionally, our method reduces the performance gap between training on raw data and distilled data to 0.29 BLEU points, demonstrating its superiority in handling multimodal distributions.
Lmask denotes the reconstruction loss. MLM-based con- struction loss plays a critical role in enhancing dependency modeling of target tokens, which significantly improves the translation quality of NAT. However, the training paradigm (X + Yots Ymask) makes NAT learn revised distribu- tion (Huang et al., 2022b), resulting in a gap between NAT training and inference (i.e., NAT inputs X and predicts the full Y in inference).
2. Methodology
'In CMLM, the observable variables are derived from ran- dom sampling RM(Y) of the ground truth Y. which is formu- lated as [MLM Σ EveRwoy log (y)IMMM(() (Y), X). GLAT, the observable variables are obtained through the glancing sam- pling strategy GS(Y, Ý) = Random(Y, DVY (D(,,))))))))))))")))")() Y)). expressed as LGLM Eyecsor log Po(y))8(Y,
In this section, we first systematically introduce the widely used techniques of existing advanced NAT models and re- veal their limitations. We then present our AEQA-NAT framework in detail, including its training and inference process with some effective schemes.
2

---

Adaptive End-to-end Quantization Alignment Training Framework for NAT
Pre-alignment
Training
(H+)
LEN
NAT
Zg
(H2g
(H2)
R
Lax
Pxy
Lay
Encoder
Decoder
x/y-
mBART Encoder
La
Softcopy
Aligned
Semantic
Reordering
Quantization Space
Semantic Quantization Space
AB (Hm))
H
Hy
2q (Hy)
Figure 2. Overview of the AEQA training framework. the pre-alignment phase, we utilize exponential moving average updates on the mBART model refine the Semantic Quantization Space. In the training stage, we optimize the whole model through LNAT, LSQA, and LLEN.
2.2. Adaptive End-to-End Quantization Alignment
Pre-aligned Semantic Quantization Space We leverage the pre-trained multilingual model mBART (Liu et al., 2020) to achieve the alignment between the source language and the target language within the SQS, as depicted in the Fig 2, mBART is pre-trained cross-lingual model based on 25 languages, and by relying its shared encoder, it can effectively embed the source sequence and the target se- quence into the SQS. Specifically, the SQS can be viewed as a K x D-dimensional vocabulary $ = (ei, e2..... ex), where
In contrast to Autoregressive Transformers, which employ consistent left-to-right decoding strategy during both train- ing and inference, NATs face incompatibility issues due to the additional information required during training that is unavailable at inference. Inspired by Vector Quantized Vari-
ational Autoencoder (VQ-VAE, Van Den Oord et al. 2017), we construct an external semantic quantization space (SQS) to serve as a bridge and constraint for the NAT system during training. This SQS is co-optimized with the model to en- sure that the NAT system captures the semantic consistency between the source and target languages. By doing so, we achieve a unified framework for NAT systems across both training and inference phases. Existing approaches such latent-GL.A (Bao et al., 2022) enhance NAT through la- tent variable modeling. Our method fundamentally diverges from such VAE-based NAT frameworks, as elaborated in Appendix A.I.
the number of word embeddings and D is the dimension of the word embedding ej. Intuitively, given the hidden states H (hz,1 2 .... 12,N, J, mapping each dimension of H, to of the K embeddings contained in the SQS. Formally, we have
(h(,),)) arg min||ha ex e
(4)
Zg (hy.)) arg minl|/aan - exrll2
(5)
where denotes the index of SQS vector ek that minimizes the distance || ex l|,, effectively quantizing hz into the category represented by e
2.3. Architecture of AEQA Training Framework
As illustrated in Fig 2, the adaptive end-to-end quantization aligmment training framework mainly consists of three mod- ules: NAT encoder Senc: a shared semantic quantization space SQS and NAT decoder faec- Their functions can be formalized
Following (Van Den Oord al., 2017), constrain the out- puts of the encoder and decoder to the vectors in SQS with commitment loss (i.e., Lx). Concretely, we use exponential moving average (EMA) to update SQS
(h,,,, n,,,) (eno(11, (2,,, L'),
Wkk I((hz,i) kk)hz,i
(hi, hhm) softcopy (accoppyhhelll (hx,1 hx.2 .. h),n)), (Eg (h1); 2g (h2). ... ,k()))))) + SQS(h,, h2, .aa....hoo Am). pY (X(Y))2(X))X)) + (z (1)mmm)) fex 1:n,)))))
Nr
n4k ym + (1 - [1)) Σ I(hha,) =k)
(6)
1 (Yex + (1 7)wk)
We use an extra module fien to predict the target length m and initialize the decoder inputs Hm, = {h,, , h2, ... hmm) with the Softcopy (Li et al., 2018; Wei et al., 2019) mecha- nism, see Appendix C.2 for more details.
Ck
7ik
http://iiicccooleeeeeeeeeeteieer les/mbart
3

---

Figure 3. Aligned Reordering (AR) process applied to sequence of semantic representations. The input sequence zy (H.) fhi, ha, Am.} is aligned with the ground truth sequence Y = (u U2,..., yn The alignment probability distribution matrix A is computed to determine the most probable order the input words. The AR process outputs the reordered sequence A., (Hm)) {/i, hts hs, h4, h2), aligning the input sequence with the correct syntactic structure as indicated by the matrix A. The checked (8) positions in the matrix represent the optimal alignment of each Adaptive End-to-end Quantization Alignment Training Framework for NAT
where I(.) is the indicator function, wy is computed as the sum of the encoder's hidden states h and 7 is decay factor, eg is updated by averaging the previous embedding and the newly computed wi. normalized by ng. For more details on constructing the SQS, refer to Appendix A.2.
Zz (Hm)
Aligned Reordering
AB(A) [,,3,5,42
h
hz
3/2
fgg
2.4. Training
hq
1/2
A
ftg
AEQA-NAT optimizes two training objectives: semantic quantization alignment loss LSQA and translation maximum likelihood loss LNAT.
hg
%/4
Rg
hg
1/5
h12
12
12
hg
R4
hg
Semantic Quantization Alignment Loss LSQA We lever- age the pre-aligned SQS to facilitate the model's semantic mapping from the source sequence the target sequence, thereby ensuring the consistency of the encoder and de- coder outputs within the SQS during the training process. Specifically, we have
CSQA= = Lax + Lxy + Lay
(7)
Lax = H, - sg((2 (H 1)
Lxy =|| 2|g (H₂) - sg( 2z 211
(9) word in the sequence.
Day = HH - sg(z, (H
(10)
prediction loss LLEN, respectively. Based on this, gradients are computed to update various parameters of the model, including the encoder parameters, decoder parameters, and the embedding vectors eg of the SQS.
where Hy represents the hidden representation output by the last layer of the decoder, which is consistent with the dimension of SQS. Note that zq (H.) is calcu- lated N.
1 DN' 2g (hx,i) and 2n (H,)) is calculated
Aligned Reordering Intuitively. the input to the decoder, S SX)),, is strongly correlated with the word order of the source sequence rather than the target sequence. These rep- resentations may not initially adhere to the correct syntactic order with respect to the target language. The Aligned Reordering (AR) mechanism" addresses this by adjust- ing the syntactic discrete representations 2g (H..) from the SQS, corresponding to words in the source sequence, as illustrated in Fig 3. Specifically, we define an align- ment probability distribution matrix A. Given the input 2g (mm) {h1, hy,
Ny E ( (11g))))
Translation Maximum Likelihood Loss LNAT To ensure the model generates complete translations while preserving the benefits of Glancing training, we design learning strat- egy compatible with Glancing Targets (Qian et al., 2021). Specifically, LNAT maximizes the conditional probability of the target sequence Y given the input sequence X:
= = GS(SQS(SeSe) D(Y,, D(Y, F))
(11)
T NNAT= = Σ log (V)))
from the SQS and the ground truth Y [v1, U2, .... Un }. the alignment probability distri- bution matrix A is computed as follows
(12)
where GS(-) denotes Glancing Sampling Strategy (Qian et al., 2021), D(Y, Y) is used to determine the number of samples?. Note that what is replaced here is the discrete vector, rather than the target tokens. SQ(-) is the quantized input after masking. Finally, the overall training loss L is obtained by
A = softmaxx (Yz, (H,,),)
(14)
where A E <nnxm is the alignment probability distribution matrix normalized by rows, which captures the alignment probabilities between the input representation 2, (H,)) and the ground truth Y. The AR process uses these probabilities to reorder the input sequence into the correct syntactic order, as indicated by the reordered sequence (2, (H,)) (ha, ha, hs, h4, h2). The detailed formulation is presented in Appendix C.
L - LNAT + eLSQa + 8LLEN
(13)
where 8 and are the hyperparameters to control the impact of Semantic Quantization Alignment loss LSQA and length
'The sampling number is computed as = A >E (vi # vi). where the sampling ratio X E [0, 1] is a hypermeter.
"Note that Aligned Reordering adjusts the word order without altering the word vectors themselves.
4

---

Adaptive End-to-end Quantization Alignment Training Framework for NAT
Table Performance comparison between our models and existing methods. The speedup measured on WMT 14 EN++DE set with batch size 1. Lave denotes the number of iterations at inference time, Adv means adaptive and m is the number of length reranking candidates. Results of prior work are quoted from respective papers. NPD represents noisy parallel decoding. Reordering denotes the aligned reordering mechanism. DSLP (Huang al., 2022a) denotes deep supervision and feed additional layer-wise predictions. Best performance of non-iterative NATs (Idec=1) are bolded. * indicates results our re-implementationn. denotes results of out implementations.
WMT14 EN-DE
WMT14 DE-EN
WMT16 EN-RO
WMT16 RO-EN
Model
Jace
Speedup
Raw
KD
Raw
KD
Raw
KD
Raw
KD
Transformer (Vaswani et al., 2017)
M
27.37
27.48
31.33
31.34
33.89
33.65
34.12
34.00
1.0x
Transformer (Ours)
M
28.11
28.37
31.90
31.88
33.70
33.68
34.39
34.05
1.0x
CMLM (Ghazvininejad al., 2019)
10
24.61
27.03
29.40
30.53
32.86
33,08
33.31
1.7x
JM-NAT (Guo et al., 2020)
10
27.69
32.24
33.52
33.72
5.7x
SMART (Ghazvininejad al., 2020b)
10
25.10
27.65
29.58
31.27
2.2x
DisCo (Kasai al., 2020)
Adv
27.34
31.31
33.22
33.25
2.6x
Multi-Task Bask NAT (Hao et al., 2021)
10
25.79
27.98
30.32
31.27
33.80
33.60
2.6x
RewriteNAT (Geng et al., 2021)
Adv
27.83
31.52
33.63
34.09
CMLMC (Huang al., 2022d)
26.40
28.37
30,92
31.41
34.14
34.57
34.13
34.14
Con-NAT (Cheng & /hanng 2022)
10
25.60
27.93
30.05
31.57
33.88
34.18
Vanilla NAT (Gu et al., 2018)
10.84
18.21
15.85*
24.33*
19.82*
27.29
21.93*
29.06
15.6x
CTC (Libovický & Helel, 2018)
17.73*
25.52
21.44*
28.73
23.12
32.60
25.05
33.46
14.6x
AXE (Ghazvininejad al., 2020a)
20.40
23.53
24.90
27.90
30.47
30.75
31,42
31.54
14.5x
GLAT (Qian et al., 2021)
18.94
25.21
25.71*
29.84
26.38*
31.19
27.99
32.04
14.2x
OaXE (Du et al., 2021)
22.40
26.10
26.80
30.20
25,43
32.40
28.17
33.30
12.4x
DAT (Huang et al., 2022c)
26.57
27.49
30.68
31.37
32,71*
32.79*
33.25
33.85
13.9x
MgMO (Li al. 2022)
26.40
30.30
32.90
33.60
DePA (Zhan et al., 2023)
26.43
30.42
33.07
33.82
15.1x 1
Renew NAT (Guo et al., 2023)
26.65
30.65
33.02
33.74
11.2x
FA-DAT (Ma et al., 2023)
27.47
27.17
31.44
13.2x
CMLM-rephraser (Shao et al., 2023)
23.12
26.65
27.44
30.70
32.30
32.72
32.07
33.03
15.0x
DAT* (Li et al., 2024)
26.48*
27.01
30.62*
31.15*
33.18
33.25
33.02*
33.14*
12.0x
fAFQA-NAII
25.96
26.24
28.78
29.04
30.83
31.11
31.29
31.67
17.0x
1AEQA-NAT w/ Reordering
26.82
27.04
29.61
29.85
32.75
32.80
32.07
32.11
17.0x
tARQA-NAT w/ NPD (m=5)
26.87
27.20
30.14
30.53
32.78
32.79
32.25
32.59
13.5x
++DSLP
1
26.22
26.29
28.95
29.14
28.97
29.16
32.01
32.34
15.3x
1ATQA-NAT Reordering
26.95
27.21
30,25
30.34
32.80
32.83
32.96
33.27
15.3x
AAKQA-NAT w/ NPD (m=5)
27.10
27.30
30.71
31.42
33.26
33.31
33.42
33.76
12.2x
fAEQA-DAT
27.03
27.62
30.94
31.70
33.09
33.37
33.56
33,89
9.1x
2.5. Inference
3. Experiments
AEQA-NAT utilizes the source text and the guantization alignment representations during inference, which is consis- tent with the training process. Specifically, we obtain
Dataset We validate our proposed models on four widely used translation benchmarks, i.e., WMT14 EN++DE (4.0M), WMT16 EN++RO (610K), WMT17 ZH+-EN (20M) and 1WSLT16 DE EEN (153K), where we follow (Zhou et al. 2020, Lee al. 2018a, Kasai al. 2020) for pre-processing. Consistent with previous work (Gu et al. 2018, Qian et al. 2021), we employ sequence-level knowledge distillation for all datasets, refer to Appendix B.
ÿ = arg max log Po (Y)SQ(X),x)
(15)
Evaluation Following prior works, we compute tokenized BLEU (Papineni al., 2002) for WMT14 EN++DE and WMT16 EN+ROO, while using SacreBLEU (Post, 2018) for WMT17 ZH+EN. To comprehensively assess translation quality, utilize multiple additional metrics, including the rule-based metric chrF (Popović, 2015) and two model-
benefiting from the fact that no additional target information introduced during training, AEQA-NAT seamlessly gener- alizes the knowledge from the training data the inference stage".
"During inference, SQ(-) utilizes a fixed sampling rate.
5

---

Adaptive End-to-end Quantization Alignment Training Framework for NAT
210
1000
16.5
260
20-
800
AT
CMLM
13.5
Count
600
25.0
Transformer
400
DAT
AEDA-DAT
D41
AEQU-DAI
AEQA-NAT
DAT:
200
DAI
18
12300
(20.000
4o 50.000
0-
3
10
Sequence Length
Regeating gram
Figure Effect of the size factor L on WMT14 EN-DE, The graph size or the out- put length is times the source length.
Figure 5. The BLEU
WMT14 EN-DE bucketed by the reference length.
Figure 6. N-gram Repetition of different models.
based metrics: COMET (Rei et al., 2020) and BLEURT (Sel- lam et al., 2020). Specifically, for COMET, we use the wmt22-comet-dd model (Rei et al., 2022), and for BLEURT, we adopt the BLEURT-20 model (Pu et al., 2021).
Table 2. Results of AT and NAT models trained with (or with- out) knowledge distillation on WMT14 EN+>DE and IWSLT16 DE-SEN.
Implementations Our models generally use the hyperpa- rameters of transformer-bat (Vaswani et al., 2017). We set the size of SQS to 2048. During pre-aligning, we set in Eq (16) to 0.5. We set the dropout rate to 0.1 and use Adam opti- mizer (Kingma & Ba, 2014) with 3 = (0.9, 0.999). During training, we set e and ö in Eq (13) to 0.25 and 0.1, respec- tively. We apply weight decay 0.01 and label smoothing 1= 0.1. We train the model with the batches of 64K/8K to- kens for WMT/IWSLT datasets, respectively. The learning rate warms up to 5e 4 in 4K steps and gradually decays according to inverse square root schedule. For hyparameter X, we adopt linear annealing from 0.6 to 0.4 for training and a fixed value of 0.4 inference. We apply noisy parallel de- coding denoted as NPD (Gu et al., 2018) and set the length beam as We extend our method to DAT (Huang et al., 2022c) by setting the graph size factor L. and removing the length prediction module, refer to Appendix D. All models are implemented on fairsen reer (Ott et al., 2019).
Methods
WMT14
IWSLT16
Avg Gap A
EN-DE
DE-
DE-EN
Vanilla NAT
10.84
15.85
17.93
47.24
w/ KD
18.21
24.33
23,81
GLAT
18.94
25.71
29.20
+4.21
w/ KD
25.21
29.84
31.43
DAT
26.57
30.68
31.57
+0.81
KD
27.49
31.37
32.40
AFQA-NAT
26.87
30.14
32.34
+0,29
KD
27.20
30.53
32.50
Transformer
28.11
31,90
32.92
+0.03
KD
28.37
31.88
32.78
benchmarks, surpassing other NAT models and indicating its ability to generate translations that are semantically and syntactically more aligned with the target language.
2) In terms of decoding speed. AEQA-NAT shows clear advantage over AT and iterative NAT models, achieving maximum speedup of 17.0x, which is also superior to other fully NAT models. Even when integrated with NPD tech- niques, AEQA-NA maintains a high decoding speed with a speedup ratio of 12.2x, retaining its edge over other NAT models. This demonstrates that AEQA-NAT achieves an effective balance between translation quality and decoding efficiency.
4. Main Results
The main results on the benchmarks are presented in Ta- ble AEQA-NAT demonstrates significant advantages in translation performance. Our method enhances conditional dependency modeling at the decoder through the Seman- tic Quantization Space (SQS), effectively eliminating the training-inference gap and enabling the model to achieve its full potential. The SQS ensures that the model can directly learn complex raw data distributions, thereby providing promising solution to eliminate the reliance knowledge distillation.
3) In terms of portability, AEQA-NAT exhibits exceptional performance. It can seamlessly integrate advanced NAT techniques, such as NPD and DSLP, without significantly increasing computational overhead. This highlights the effi ciency and flexibility of the AEQA-NAT framework.
1) In terms of translation quality, AEQA-NAT achieves sfate-of-the-and results across multiple translation directions, demonstrating significant performance advantages com- pared to fully NAT models. Compared to AT, AEQA-NAT exhibits a performance gap of less than I BLEU across all
4.1. Analysis
AEQA Enhances Dependency Modeling on Raw Data Knowledge distillation (KD) is widely employed to enhance NAT learning. Distilled data constitutes mixed marginal
6

---

Adaptive End-to-end Quantization Alignment Training Framework for NAT
Table Performances WMT14 EN-DE and WMT17
Table 5. Results of different translation models on WMT16 EN-RO. We encompass a wide range of metrics including rule- based metrics (BLEU and chrf) and model- basel--bsed metrics (COMET and BLEURT).
ZH-EN with the fixed sampling ratio in inference.
Sampling ratio A
0.1
0.2
0.3
0.4
0.5
0.6
WMT14 EN-DE 18.34
20.71
24.53
26.82
25.64
23.87
WMT17 ZH-EN
12.96
15.37
19.11
23.18
23.26
20.95
Methods
BLEU chrft COMETT BLEURTY Speedupt
Raw Data
Vanilla NAT
19.82
50.65
65.22
53.17
15.6x
Table Results WMT14 EN-+DE test with differrnt her of references (c.g., "Single" and "Multiple"). "A" indicates the performance gap over the Vanilla NAT.
GLAT
26.38
56.34
73.53
62.89
14.2x
DAT
32.71
57.28
76.08
66.45
13.9x
AEQA-NAT
32.75
57.40
77.12
67.64
17.0x
Distillation
Single
Multiple
Vanilla NAT
27.29
56.38
72.16
62.01
15.6x
Methods
GLAT
31.19
57.07
75.23
65.17
14.2x
BLEU
A
BLEU
A
DAT
32.79
57.81
76.52
66.84
13.9x
Raw Data
AEQA-NAT
32.80
57.80
77.01
66.92
17.0x
Vanilla NAT
10.8
25.0
CMLM
11.0
+0.2
28.1
+3.1
GLAT
18.9
+8.1
51.5
+26.5
AEQA-NAT
26.8
+16.0
71.3
+46.3
intervals, achieving results comparable to state-of-the-art
NAT models. For AEQA-DAT, its translation quality sur- passes that of DAT in every interval, further validating the effectiveness and stability of AEQA.
Distillation
Vanilla NAT
15.9
41.9
CMLM
18.6
+2.7
50.7
+8.8
GLAT
25.2
+9.3
65.3
423.4
AEQA-NAT
27.0
+11.1
73.5
+31.6
N-gram Repetition We evaluated the ability of AEQA to handle n-gram repetition. As illustrated in the Fig 6, AEQA-DAT demonstrates significant advantage over the DAT models reducing n-gram repetition. This improve- ment can be attributed to AEQA's reduction of input length, where the graph size decreases from 8 to 3, thereby stream- lining the number of references on the directed acyclic graph. This effect is particularly pronounced for n-grams with sizes smaller than 4. Compared to all NATs, AEQA- NAT consistently maintains a lower n-gram repetition count, demonstrating its robust capability to handle multimodal challenges effectively.
distribution derived from raw data and the teacher model's distribution. As illustrated in Table 2, the Transformer's 's translation quality improves by merely 0.03 BLEU after applying KD, underscoring its robust ability to model raw data and minimal dependence on distilled data. Intuitively, a model's capacity to capture data distribution characteris- ties inversely correlates with its reliance on distilled data. Vanilla NAT shows significant dependence on distilled data, achieving BLEU score improvement of 7.24 post- KD.. DAT markedly reduces this gap to 0,81, while AEQA-NAT achieves further reduction to 0.29 for the first time. These results demonstrate that AEQA effectively captures raw data distribution features and exhibits advanced dependency modeling capabilities.
Sampling Ratio in Inference The results across varying sampling rates demonstrate that AEQA-NAY maintains ro- bust performance under diverse data sampling conditions during inference, as illustrated Table This underscores that the SQS effectively addresses the limitations of tradi- tional methods reliant on explicit target word dependencies for token-level relationship modeling, thereby affirming the efficacy of NATs in leveraging semantic consistency spaces. For WMT14 En-De, model performance improves progressively as the sampling rate À increases from 0.1 to 0.4, reaching its peak at À = 0.4. Beyond this point, perfor- mance declines, indicating that excessively high sampling rates may introduce redundancy or noise. A comparable pattern is observed for WMT17 Zh-En, with optimal per- formance achieved at À 0.5. These findings suggest that optimal sampling rates are dataset-dependee emphasizing the necessity of tailoring À to specific data distributions.
Graph Size The original DAT requires a pre-defined graph size significantly larger than the source sequence length to model translation references, typically set L = 8. As illustrated in Fig 4, AEQA-DAT achieves peak performance with only I, = 3, whereas DAT and DAT* exhibit gradual performance improvements as L increases, yet their final performance remains inferior to that of AEQA-DAT Overall, AEQA enables DAT streamline the graph size, thereby enhancing training efficiency.
Different Lengths To analyze the impact of varying lengths on model performance, we categorized references into differ- ent length intervals and evaluated translation quality within each interval. As shown in Fig 5, AEQA-NAT demonstrates strong performance across all reference sequence length
Multiple References To assess the translation quality of AEQA-NAT from a multimodal perspective, we evaluated

---

Adaptive End-to-end Quantization Alignment Training Framework for NAT
its performance on the dataset released by Ott et al. 2018, which includes ten reference translations for each of the 500 sentences from the WMT14 EN-DE test set. As demon- strated in Table 4, AEQA-NAT significantly surpasses other models in multi-reference translation tasks on raw data. This superiority arises from its ability to effectively capture mul- timodal references, allowing to fully exploit its robust diversity generation capability in handling "one-to-many" mapping relationships. Notably, while all models show per- formance gains after distillation, AEQA-NAT'S advantage becomes less pronounced compared to its performance on raw data. This further underscores that AEQA-NAT sub- stantially reduces its dependence on knowledge distillation.
Table 6. Ablation on WMT14 EN test set with different com- binations of techniques. "AR" denotes Aligned Reordering.
K
Sampling
Line
AR DSLP BLEU
512
1024 2048 4096 Uniform Adaptive
1
10.84
2
19.93
3
20.62
4
21.85
5
22.69
6
22.83
7
23,51
8
25.14
9
25.96
Comprehensive Performance Evaluation To comprehen- sively evaluate the performance of AEQA-NAT, we con- ducted assessments across multiple key benchmarks. As shown in Table 5, on raw data, AEQA-NAT outperforms other NAT models on both rule-based metrics (BLEU and chrF) and model-based metrics (COMET and BLEURT), indicating its ability to generate more coherent and higher- quality translations. On distilled data, AEQA-NAT achieves superior performance on model-based metries, suggesting that the introduction of the Semantic Quantization Space (SQS) better captures the semantic relationships between the source and target languages. This is because model- -ased metrics evaluate translation quality by measuring semantic relevance between sentences using parametric knowledge.
10
4
26.82
11
26.95
2048, and 4096, respectively. These findings indicate that the impact of sampling strategies on model performance decreases with larger K values.
Aligned Reordering When evaluating only the predefined K values and sampling methods without integrating AR and DSLP, the BLEU score is 25.96, as indicated in Line 9. With the inclusion of AR (Line 10), the BLEU score rises to 26.82. This improvement suggests that the AR mechanism further enhances the model's translation performance, poten- tially by refining syntactic structures improving semantic alignment, thereby elevating the quality of the generated translations.
4.2. Ablation Study
AEQA significantly enhances model performance. As presented in Table 6, the incorporation of the Semantic Quantization Space (SQS) results in a performance gain exceeding BLEU points (Line 1 vs. Line 2). This sub- stantial improvement underscores the critical role of AFQA training in enhancing the model's capacity to capture data distribution features. Furthermore, we observe a consistent trend of improved translation quality with an increase in the categorical number K of the SQS, with optimal perfor- mance attained at K = 2048. Specifically, as demonstrated in Line and Line 9, the BLEU scores achieve 25.14 and 25.96 under distinct sampling strategies, respectively. These results highlight the importance of selecting an appropriate K value for maximizing model efficacy.
5. Related Work
The optimization strategies for NAT systems can be broadly categorized into two dimensions: 1) Input-side enhance- ment, which incorporates explicit target information to im- prove the model's s ability to capture data distributions, as exemplified by techniques as Conditional Masked Language Model (CMLM, Ghazvininejad et al. 2019; Du et al. 2021; Cheng & Zhang 2022; Shao et al. 2023) and Glancing Trans- former (GLAT, Qian et al. 2021: Bao al. 2022: Schmidt et al. 2022; An al. 2023); and 2) Target-side optimiza- tion, which modifies learning objectives alleviate training difficulties, including methods like Aligned Cross Entropy (AXE, Ghazvininejad et al. 2020a), Order-Agnostic Cross Entropy (OAXE, Du et al. 2021), Muanuarrry Opti- mization (MgMO, Li et al. 2022) and Non-Monotonie La- tent Aligmments (NMLA, Shao & Feng 2022). Target-side optimization techniques, such as AXE and OAXE, attempt to accommodate NAT training by relaxing strict word or- der alignment. Furthermore, several significant Aligned Reordering approaches have been applied to NAT. Reorder- NAT (Ran et al., 2021) incorporates a dedicated reorder- ing module to explicitly model word rearrangement infor- mation during decoding. AligNART (Song et al., 2021)
Influence of Sampling Strategies. The experimental results reveal that Adaptive Sampling exhibits a marginally better performance trend than Uniform Sampling under identical K value configurations. At 512, the performance gap between the two exceeds 1 BLEU point (Line 2 vs. Line 3). As K increases, this gap progressively narrows, with BLEU score differences of 0.84, 0.82, and 0.68 for K 1024,
http://iiiiteooooooooooottttttan nmt
8

---

Adaptive End-to-end Quantization Alignment Training Framework for NAT
achieves one-to-one mapping between encoder hidden rep- resentations and target information through alignment esti- mation, thereby reducing the modality of target distributions. AEQA-NAT leverages the synergistic effect of multiple loss components to jointly consider both the input and decoding phases, guiding the model to adaptively align the source and target texts.
X., Chen, J.,, and Li, L. latent-GLAT: Glancing at latent variables for parallel text generation. In Muresan, S., Nakov, P., and Villavicencio, A. (eds.), Proceedings of the 60th Annual Meeting of the Association for Compu- tational Linguisties (Volume I: Long Papers), pp. 8398- 8409, Dublin, Ireland, May 2022. Association for Com- putational Linguisties. doi: 10. 1/018863/11200222clll www.eaaaannooooooooo///22222222. wrg.//22..
575. URL acl acl-long.0. 575/.
6. Conclusion
In this work, we propose the AEQA-NAT training frame- work for non-autoregressive translation. By introducing se- mantic quantization space, AEQA-NAT eliminates reliance on target information and effectively bridges the training- inference gap in NAT. Experimental results demonstrate that AEQA-NAT achieves state-of-the-art performance among fully non-autoregresssvve models across multiple translation benchmarks, while maintaining decoding speeds compara- ble to Vanilla NAT. Furthermore, our approach enhances the ability of NAT models to learn from raw data distribu- tions, reducing the performance gap between raw data and knowledge distillation to 0.29 BLEU score.
Chen, B. and Cherry, C. A systematic comparison of smooth- ing techniques for sentence-level bleu. In Proceedings of the ninth workshop on statistical machine translation, pp. 362-367, 2014.
Cheng, H. and Zhang, Z. Con-NAT: Contrastive non- autoregressive neural machine translation. In Goldberg, Y., Kozareva, Z., and Zhang, Y. (eds.), Findings of the Association for Computational Linguistics: EMNLP 2022, pp. 6219-6231, Abu Dhabi, United Arab Emi- rates, December 2022. Association for Computational Linguisties. doi: 100.18653/1122222.fniiii 463. URL https: a haa/aaaaaannttoogog/o org//022. f findings-emm11 463/
Impact Statement
Du, C., Tu, Z., and Jiang, J. Order-agnostic cross entropy for non-autoress aunaautoreeressvve machine translation. In International conference on machine learning, pp. 2849 28599. PMLR, 2021.
This work presents novel approach to bridge the training- inference gap in NAT models. By reducing the dependency on knowledge distillation, our method enhances the effi- ciency and scalability of NAT training. The proposed frame- work has the potential to enable more effective and efficient NAT deployment in real-world applications, contributing to the advancement of machine translation and other sequence generation tasks.
Geng, X., Feng, X., and Qin, B. Learning rewrite for non-autoregressive neural machine translation. In Moens, M.-F., Huang, X., Specia, L.,, and Yih, S. W.-t. (eds.), Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing. pp. 3297-3308, Online and Punta Cana, Dominican Republic, November 2021. Association for Computa- tional Linguistics. doi: 10. 10:18633/1122221emlppmmm 265. URL http:///aaaaatthhooooeo emnlp-main. ..m.mmaii....6......m
References
An, C., Feng, J., Huang, F., Qiu, X., and Kong, L. Opti- mizing non-autoregressive transformers with contrastive learning, 2023. URL https:://axii org/ abs/ 2305 13667.
Ghazvininejad, M., Levy, O., Liu, Y., and Zettlemoyer, L. Mask-predict: Parallel decoding of conditional masked language models. In Inui, K., Jiang, J., Ng. V., and Wan, X. (eds.), Proceedings of the 2019 Conference on Em- pirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Lan- guage Processing (EMNLP-IJCNLP), pp. 6112-6121, Hong Kong, China. November 2019. Association for Computational Linguistics. doi: 10.18653/v1/D19-16633 org/ URL https: ht//ccllalnntc http:///////ccccccoooooooo//////
Bao, Y., Huang, S., Xiao, T., Wang, D., Dai, X., and Chen, J. Non-autoregressive translation by learning tar- get categorical codes. In Toutanova, K., Rumshisky, A., Zettlemoyer, L., Hakkani- Tur, D., Beltagy, I., Bethard, S., Cotterell, R., Chakraborty, T., and Zhou, Y. (eds.), Proceedings of the 2021 Conference of the North Amer- ican Chapter of the Association for Computational Lin- guistics: Human Language Technologies, pp. 5749- 5759, Online, June 2021. Association for Computa- tional Linguistics. doi: 10. 10.18663/1//0021.aaccl 458. URL https: https////caaathoooo.ooooooo. naacl -ma inaiinn.. 458/.
Ghazvininejad, M., Karpukhin, V., Zettlemoyer, L., and Levy, O. Aligned cross entropy for non-autoregressive machine translation. In International Conference on Ma- chine Learning, pp. 3515-3523. PMLR, 2020a.
Bao, Y., Zhou, H., Huang, S., Wang, D., Qian, L., Dai,

---

Adaptive End-to-end Quantization Alignment Training Framework for NAT
Ghazvininejad, M., Levy, O., and Zettlemoyer, L. Semi- autoregressive training improves mask-predict decoding. arXiv preprint arXiv:2001.08788. 2020b.
naacl-main.313. URL httpp:://aclaaeee
org/2021 wtt.:/22.naaclmmii.....
Huang, Zhou, H., Zaïane, O. R.,, Mou, I.,, and Li, I.. Non- autoregressive translation with layer-wise prediction and deep supervision. In Proceedings of the AAAI conference on artificial intelligence, volume 36, pp. 10776-10784, 2022a.
Gu, J. and Kong. X. Fully non-autoregressive neural ma- chine translation: Tricks the trade. Zong, C., Xia, F,, Li, W., and Navigli, R. (eds.), Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, pp. 120- 133, Online, August 2021. Association for Compu- tational Linguistics. doi: 10. 1118655/112021.fmndniii 11. URL https: ht//aaaaanncllooooo wrg. 11.
Huang, F., Tao, T., Zhou, H., Li, L., and Huang, M. On the learning of non-autoregressive transformers. In Interna- tional Conference on Machine Learning, pp. 9356-9376.
PMLR, 2022b.
Gu, J., Bradbury, J., Xiong, C., Li, V. O., and Socher, R. Non-autoorerressive neural machine translation. In International Conference Learning Representations, 2018. URL htttttt///peereee///// http://pppneeeeeeeeeeeee net/ net/morum? id-B118Bt1Cb.
Huang, F., Zhou, H., Liu, Y., Li, H., and Huang, M. Di- rected acyclic transformer for non-autoregressive ma- chine translation. In Chaudhuri, K., Jegelka, S., Song, L., Szepesvari. C., Niu, G., and Sabato, S. (eds.), Pro- ceedings of the 39th International Conference on Ma- chine Learning, volume 162 of Proceedings of Ma- chine Learning Research, pp- 9410-9428. PMLR, 17- 23 Jul 2022c. URL https: /ttp:s://prroeoe /prpceedingng mllr. press/ /vlsss/vlle vtlvvvv22.2/ 62 http/huang22200 2m.ann22mm.mmmmmm html.
Guo, J. Tan, x., He, D., Qin, T., Xu, L., and Liu, T.-Y. Non-autoregressive neural machine translation with enhanced decoder input. In Proceedings of the Thirty-Third AAAI Conference on Artificial Intelli- gence and Tirst Innovative Applications of Ar- tificial Intelligence Conference and Ninth AAAI Sym- posium on Educational Advances in Artificial Intelli- gence, AAAI' 19/IAAI1 19/EAAI 19. AAAI Press, 2019.
Huang, X. S., Perez, F,, and Volkovs, M. Improving non- autoregressive translation models without distillation. In International Conference on Learning Representations, 2022d.
ISBN 978-1-57735-809-1.
doi: 11..69/aaiiv333000 33013723. URL https::////// /tt:://oiioooo org/ 100. 1609/ aaai Vtt.vv333...3311377
Kaiser, L.,, Bengio, S., Roy, A., Vaswani, A., Par- mar, N., Uszkoreit, J., and Shazeer, N. Fast de- coding sequence models using discrete latent vari- ables. In Dy, J. and Krause, A. (eds.), Proceed- ings of the 35th International Conference Machine Learning, volume 80 of Proceedings of Machine Learn- ing Research, pp. 2390-2399. PMLR, 10-15 Jul 2018. URL https: / pro.eedincennsesee www. pra.s/rres/v80 kaiser18a html.
Guo, J., Xu, L.. and Chen, E. Jointly masked sequence- to-sequence model for non-autoregressive neural ma- chine translation. In Jurafsky, D., Chai, J., Schluter, N., and Tetreault, (eds.), Proceedings of the 58th An- mual Meeting of the Association for Computational Lin- guistics, pp. 376-385, Online, July 2020. Association for Computational Linguistics. doi: 10. 10.11863//1/200.. acl acl-main.36. URL httpt:::s//aaaaaaooootttett 2020. acl - ma mm...36/.............
Kasai, J., Cross, J., Ghazvininejad, M., and Gu, J. Non- autoregressive machine translation with disentangled con- text transformer. In International conference on machine learning, pp. 5144-5155. PMLR, 2020.
Guo, P., Xiao, Y., Li, J., and Zhang, M. Renewnat: renewing potential translation for non-autoregressive transformer. In Proceedings of the AAAI Conference on Artificial In- telligence, volume 37, pp. 12854-12862, 2023.
Kim, Y. and Rush, A. M. Sequence- -evel knowledge dis- tillation. In Proceedings of the 2016 Conference on Em- pirical Methods in Natural Language Processing, pp. 1317-1327, 2016.
Hao, Y., He, S., Jiao, W., Tu, Z., Lyu, M., and Wang, X. Multi-task learning with shared encoder for non- autoregressive machine translation. In Toutanova, K., Rumshisky, A., Zettlemoyer, L., Hakkani-Tur. D., Belt- agy, I., Bethard, S., Cotterell, R., Chakraborty, T., and Zhou, Y. (eds.), Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguisties: Human Language Technolo- gies. pp. 3989-3996, Online, June 2021. Association for Computational Linguistics. doi: 10. 18653/11/222111
Kingma, D. P. and Ba, J. Adam: A method for stochastic optimization. CoRR, abs/1412.6980. 2014. URL https::/aat /ppi. semanttisccoolar org http//orpuII:::22:0000
Lee, J., Mansimov, E., and Cho, K. Deterministic non- autoregressive neural sequence modeling by iterative re- finement. In Riloff, E.,, Chiang, D., Hockenmaier, J.,
10

---

Adaptive End-to-end Quantization Alignment Training Framework for NAT
and Tsuji,, J. (eds.), Proceedings of the 2018 Confer- ence on Empirical Methods in Natural Language Pro- cessing, pp. 1173-1182, Brussels, Belgium, October- November 20. Association for Computational Lin- guistics. doi: 10. 10538//////118114444 1149. URL https: //aclanthology org/ 1149/.
Oit, M., Auli, M., Grangier, D., and Ranzato, M. Analyzing uncertainty in neural machine translation. In Interna- tional Conference Machine Learning, 2018.
Ott, M., Edunov, S., Bacvski, A., Fan, A., Gross, S., Ng, N., Grangier, D., and Auli, M. fairseq: A fast, extensible toolkit for sequence modeling. In Ammar, W., Louis, A., and Mostafazadeh, N. (eds.), Proceed- ings of the 2019 Conference of the North American Chapter of the Association for Computational Linguis- lies (Demonstrations), pp. 48-53, Minneapolis, Min- nesota, June 2019. Association for Computational Lin- guistics. doi: 10. 10.18653/v1/NN19-40000 / //aclanthology org N1 9 4-4009/
Lee, J., Mansimov, E., and Cho, K. Deterministic non- autoregressive neural sequence modeling by iterative re- finement, 2018b. URL wwt.tttt////iiii.taa org/ abs/ 1802. 06901.
Li, Y. Cui, L., Yin, Y., and Zhang, Y. Multi-granularity opti- mization for non-autoregresssve translation. In Goldberg, Y., Kozareva, Z., and Zhang, Y. (eds.). Proceedings of the 2022 Conference on Empirical Methods in Natural Lan- guage Processing, pp. 5073-5084, Abu Dhabi, United Arab Emirates, December 2022. Association for Compu- tational Linguistics. doi: 10.18655//1/20222.mmmpp main. 339. URL https: haa.aatooaoooooy......... or2/2022. emn 1 p-main 339.
URL https:
Papineni, K., Roukos, S., Ward, T., and Zhu, W.-J. Bleu: a method for automatic evaluation of machine translation. In Isabelle, P., Charniak, E., and Lin, D. (cds.), Proceed- ings of the 40th Annual Meeting of the Association for Computational Linguistics, pp. 311-318, Philadelphia, Pennsylvania, USA, July 2002. Association for Computa- tional Linguistics. doi: 10.3115/1073383310722 https: / htt///ccaantthooo/ wttp://hoo..oo..oooo/-/00--00000 wptppo/-
URL
Li, Y. Zhang, H., Yan, J., Yin, Y., and Zhang. Y. What have we achieved on non-autoregressive trans- lation? In Ku, L.-W., Martins, A., and Srikumar, V. (eds.), Findings of the Association for Computa- tional Linguistics: ACL 2024, pp. 7585-7606, Bangkok, Thailand, August 2024. Association for Computa- tional Linguistics. doi: 10. 10.18653/112004.fmmiing..
Popović, M. chrF: character n-gram F-score for automatic MT evaluation. In Bojar, O., Chatterjee, R., Federmann, C., Haddow, B., Hokamp, C., Huck, M., Logacheva, V., and Pecina, P. (eds.), Proceedings of the Tenth Workshop on Statistical Machine Translation, pp. 392-395, Lis- bon, Portugal, September 2015. Association for Compu- tational Linguisties. doi: 10. 10.18653/1//155-34499 URL https: ptttps://aclatteee /ataaaaaaanccooo www.aaattoooogy.oog//.----------
452. URL https: http://caaaattooooooo/////0 findings aaiggs-acl... acl.452/:
Li, Z., He, D., Tian, F., Qin, T., Wang, L., and Liu, T.- Y. Hint-based training for non-autoregressive translation. 2018.
Post, M. A call for clarity in reporting BLEU scores. In Bojar, O., Chatterjee, R., Federmann, C., Fishel, M., Graham, Y., Haddow, B., Huck, M., Yepes, A. J., Koehn, P., Monz, C., Negri, M., Névéol, A., Neves, M., Post, M., Specia, L., Turchi, M., and Verspoor, K. (eds.), Proceedings of the Third Conference on Ma- chine Translation: Research Papers, pp. 186-191, Brus- sels, Belgium. October 2018. Association for Computa- tional Linguistics. doi: 10. 10.18653/v1/W16 111////111-65 6319. URL https:///acaaarthhooo.oo
Libovicky, J. and Helel, J. End-to-end non-autoregressive neural machine translation with connectionist temporal classification. In Riloff, E., Chiang, D., Hockenmaier, J., and Tsujii, J. (eds.), Proceedings of the 2018 Confer- ence on Empirical Methods in Natural Language Pro- cessing, pp. 3016-30211 Brussels, Belgium, October- November 2018. Association for Computational Lin- guistics. doi: 10. 10.188633v1/D18. 1336. URL https: //aclanthologe hwwpollooyooogg/////oo www.or/g//////////eeeooeeee 1336//
org/w1 6319/.
Pu, A., Chung, H. W., Parikh, A., Gehrmann, S., and Sel- lam, T. Learning compact metrics for MT. In Moens, M.-F., Huang, X., Specia, I.., and Yih, S. W.-t. (eds.), Proceedings of the 2021 Conference on Empirical Meth- ods in Natural Language Processing, pp. 751-762, On- line and Punta Cana, Dominican Republic, November 2021. Association for Computational Linguistics. doi: 10.18653/v1/2011 hwt.://022..mmmmmmmmm.....8.. a acl antho 1 ogy- org/ / 22021 emn 1ammllpmminm i. 58/.
Liu, Y., Gu, J., Goyal, N., Li, X., Edunov, S., Ghazvininejad, M., Lewis, M., and Zettlemoyer, I.. Multilingual denois- ing pre-training for neural machine translation. 2020.
Ma, Z., Shao, C., Gui, S., Zhang, M., and Feng, Y. Fuzzy alignments in directed acyclic graph for non- autoregressive machine translation. In The Eleventh International Conference on Learning Representations, /ttpt:t::////peeeeee http:///ppeeeeeeeee/eeee hww.:/eettrorrt/
URL https://
Qian, L., Zhou, H., Bao, Y., Wang, M., Qiu, L., Zhang, W., Yu, Y., and Li, L. Glancing transformer for non- autoregressive neural machine translation. In Zong,
2023. URL
1d=1sz

---

Adaptive End-to-end Quantization Alignment Training Framework for NAT
C., Xia, F., Li, W., and Navigli, R. (eds.), Proceed- ings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th Interna- tional Joint Conference on Natural Language Process- ing (Volume I: Long Papers), pp. 1993-2003, Online, August 2021. Association for Computational Linguis- tics. doi: 10. 11/6663//1120222aa-lllooo 155. URL https: // //aclanthology htt.hhoooooo...g..oeeoe/ aww.accl-looo 1oo.---oonn....... 155.
(eds.), Proceedings of the 2022 Conference on Empiri- cal Methods in Natural Language Processing, pp. 2785-
2799, Abu Dhabi, United Arab Emirates, December
2022. Association for Computational Linguisties. doi:
10. 111865531122222222mpmmmmaiii aclanthology.o wwwph//yo.oor..ooooooe 2022. emm.z///2.....pppppp.mmmm emnlp wmw.ppmmin..111 main.179.
URL https://
Sellam, T., Das, D., and Parikh, A. BLEURT: Learn- ing robust metrics for text generation. In Jurafsky, D., Chai, J., Schluter, N., and Tetreault, J. (eds.), Pro- ceedings of the 58th Annual Meeting of the Associa- tion for Computational Linguistics, pp. 7881-7892 On- line, July 2020. Association for Computational Linguis- tics. doi: 10. 10.16653/1120222.ammmmm..... 704. URL https: / //aclanthologe
Ran, Q. Lin, Y. Li, P., and Zhou, J. Guiding non-autoregressivee neural machine translation decod- ing with reordering information. Proceedings of the AAAI Conference on Artificial Intelligence, 35(15): 13727-13735, May 2021. doi: 10.1609/aaai.v351155 17618. URL https://ojs aaai httpAAAA//artiicl///eee//// aaai. org index.
wwwpooggg///.... acl-mama...cmmmamm in. 704/-
Shao, C. and Feng, Y. Non-monotonic latent alignments for ete-based non-auloregressive machine translation. Ad- vances in Neural Information Processing Systems, 35: 81 8159-8173, 2022.
Rei, R., Stewart, C., Farinha, A. C., and Lavie, A. COMET: A neural framework for MT evaluation. In Webber, B. Cohn, T, He, Y., and Liu, Y. (eds.), Proceed- ings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 2685- 2702, Online, November 2020. Association for Computa- tional Linguistics. doi: 10.18663/v1/2020.mmmll http://022..mmmmmaaa.... main.
Shao, C., Zhang, J., Zhou, J., and Feng, Y. Rephrasing the reference for non- non-autoregressive machine translation. In Proceedings of the AAAI Conference on Artificial Intelli- gence, volume 37, pp. 13538 13546, 2023.
213. URL https: /tap::://aaaaae h//aaccaaatoooogyy owg.o/og/000 emnlp-maing 213/.
Shu, R., Lee, J., Nakayama, H., and Cho, K. Latent- variable non-autoregressive neural machine translation with deterministic inference using a delta posterior. Pro- ceedings of the AAAI Conference on Artificial Intelli- gence, 34(05):8846 8853, Apr. 2020. doi: 10.1609/ aaai.v34i05 6613. URL /os.:ttppss///s.aaaaaaaaaaa index. phdex.php/ AAA./article//veew///////etttt// 64111
Rei, R., C. de Souza, J. G., Alves, D., Zerva, C., Farinha,
A. C., Glushkova, T., Lavie, A., Coheur, L.,, and Martins, A. F. T. COMET-22: Unbabel-IST 2022 submission for the metrics shared task. In Koehn, P., Barrault, L., Bo- jar, O., Bougares, F., Chatterjee, R., Costa-jusså, M. R., Federmann, C., Fishel, M., Fraser, A., Freitag, M., Gra- ham, Y., Grundkiewicz, R., Guzman, P., Haddow, B., Huck, M., Jimeno Yepes, A., Koemi, T., Martins, A., Morishita, M., Monz, C., Nagata, M., Nakazawa, T., Ne- gri, M., Névéol, A., Neves, M., Popel, M., Turchi, M., and Zampieri, M. (eds.), Proceedings of the Seventh Confer- ence on Machine Translation (WMT), pp. 578-585, Abu Dhabi, United Arab Emirates (Hybrid), December 2022. Association for Computational Linguistics. URL. https: //aclanthology org/ 2022. wmt 1. 125/
Song, J., Kim, S., and Yoon, S. AligNART: Non- autoregressive neural machine translation by jointly leam- ing to estimate alignment and translate. In Moens, M.-F., Huang, X., Specia, L.,, and Yih, S. W.-t. (eds.), Pro- ceedings of the 2021 Conference on Empirical Meth- ods in Natural Language Processing, pp. 1-14, On- line and Punta Cana, Dominican Republic, Novem- ber 2021. Association for Computational Linguistics. doi: 10. 10.18655ww/20221emmmp-mm / //aclanthology wrgp://oo.o..gg///0 wrgg//21g/222 minni-mmain1//
URL https:
Saharia, C., Chan, W., Saxena, S., and Norouzi, M. Non- autoregressive machine translation with latent alignments. In Webber. B., Cohn, T., He, Y., and Liu, Y. (eds.), Pro- ceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 1098- 1108, Online, November 2020. Association for Computa- tional Linguistics. doi: 10. 12..86661112022.emmll./ 83. URL https: http:///caaatthooyy.ooo org. 2020. emnlp-mate 83.
Van Den Oord, A., Vinyals, O., et al. Neural discrete rep- resentation learning. Advances in neural information processing systems, 30, 2017.
Vaswani, A., Shazeer, N.. Parmar, N., Uszkoreit. J., Jones, L., Gomez, A. N., Kaiser, L. u., and Polosukhin, I. Attention is all you need. In Guyon, I., Luxburg, U. V., Bengio, S., Wallach, H., Fergus, R., Vishwanathan, S., and Garnett, R. (eds.), Advances in Neural Information Processing Systems, volume 30. Curran Associates, Inc., 2017. URL httpps: nww.ips....p....
Schmidt, R., Pires, T., Peitz, S., and Lööf, J. Non- autoregressive neural machine translation: A call for clarity. In Goldberg, Y., Kozareva, Z.,, and Zhang, Y.
12

---

Adaptive End-to-end Quantization Alignment Training Framework for NAT
pdf.
Wei, B., Wang, M., Zhou, H., Lin, J., and Sun, X. Im- itation learning for non-autoregressive neural machine translation. In Korhonen, A., Traum, D., and Marquez, L. (eds.), Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pp. 1304- 1312, Florence, Italy, July 2019. Association for Compu- tational Linguistics. doi: 10. 1001666611//119-1222 URL https: //laaaaannttooooo/e wttp://oog..or/11111111101
Xiao, Y.. Wu. L., Guo, J., Li. J., Zhang, M., Qin, T., and Liu, T.Y. A survey ation for neural machine translation and beyond. IEEE Trans. Pattern Anal. Mach. Intell., doi: 10.1 October 2023. ISSN 0162-8828.
aon-autoregressive gener-
TPAMM.2023.3277122. URL https: //w////ii//////////////.... org/10. 1109
Zhan, J., Chen, Q., Chen, B., Wang, W., Bai, Y., and Gao, DePA: Improving eunreuooorereeevve transla- tion with dependency-awee Federico, M., and Carpuat, M. (eds.), Proceedings of the 20th International Conference Spoken Language Translation (IWSLT 2023), pp. 478-490 Toronto, Canada (in-person and online), July 2023. Association for Com- putational Linguistics. doi: 10. 1/058/63 47. URL 1ws1t-1.47/.
decoder. In Salesky, E.,
Zhou, C., Gu, J., and Neubig, G. Understanding knowledge distillation in non-autoregressive machine translation. In International Conference on Learning Representations, 2020. URL https:::p/.........t. / id=BygFVAEKDH.
wwt. forum?
13

---

Adaptive End-to-end Quantization Alignment Training Framework for NAT
A. Pre-aligned SQS
A.1. VAE-based NAT
Latent Transformer (LT, Kaiser et al. 2018) is the first to use the vector quantization (VQ) technique to improve NAT learning. During inference, VAE-based NATs first predict the latent variable, then con-autoregressively produce the entire target sequence y conditioned on the latent sequence (Kaiser et al., 2018; Shu et al., 2020; Bao al., 2021; 2022). In contrast, we do not introduce additional network model the distributions of latent variables. We design NAT-based Semantic Quantization Space (SQS) for aligning discrete representations of source and target languages. Specifically, ABQA-NAT differs from latent-GL.n in several key aspects:
AEQA-NAT introduces a novel Semantic Quantization Space (SQS) that jointly models discrete latent variables for both source and target texts during pre-alignment. This bilateral alignment mechanism, driven by the collaborative effect of multiple loss terms, establishes a unified semantic quantification space, ensuring cross-lingual consistency between training and inference. In contrast, latent-GLAT adopts a unilateral approach, encoding only target-side information and requiring auxiliary network for latent variable prediction during inference.
While latent- GLAT employs glancing laanning-opimmzzin masked token reconstruction by training on both latent variables and explicit tokens-it directly predicts complete translations during inference. Conversely, AEQA-NAT maintains trainingiifferncce inference parity by directly generating full translations in both phases, eliminating architectural discrepancies.
A.2. Optimizing Semantic Quantization Alignment
To ensure effective alignment, we design a fine-tuning task on the pre-trained multilingual model mBART (Liu et al., 2020) for optimizing the pre-aligned SQS, as shown in Fig 2. For the sake of brevity, only the details related to the SQS are presented here. Given a language pair (X, Y), and through the mBART encoder, we obtain the intermediate representation H.. Note that SQS contains the pre-alignn aligned embeddings 2q (H,) and 2y (H,)). To ensure translation quality, we prefer 2y (H2) keep the original embedding position and zy (H₂) to cluster zy (Hy). We define the training objective for the alignment of the semantic quantization CAL
CAL = La + aLx
(16)
C. = 1 H₂)) sg( HH,
(17)
Cx = |He sg(=z (H,))
(18)
where a is the hyperparameter to control the effect of Lz, sg(-) the stop-gradient operation and H. the intermediate representation output by the encoder for the target text, having the same dimension (H,)) Note that zy (H,) is 1 Σ 2n (ha.i) and 2n (H,) is calculated as N₂ E/'1 (g(hya), which represent the closet embedding calculated as No
optima for X and Y in SQS, respectively.
The specific meaning of each loss term is as follows:
La: Ensures that the representation (H,)) of the source text is semantically aligned with the representation 2g (H,)) of the target text quantization. This helps the model establish accurate semantic associations between different texts, enhancing the accuracy of cross-text semantic transfer and facilitating the subsequent generation of semantically coherent texts.
Le: Maintains a reasonable relationship between the original intermediate representation H,, of the source text and its corresponding quantized representation zy (after
(after stop- dop-gradient processing), preventing the quantization process from excessively distorting the original semantics and ensuring that the quantized representation 2y (H,,) can still effectively carry the key semantic information of the source text.
B. Knowledge Distillation
Existing NAT techniques rely on knowledge distillation (Kim & Rush, 2016; Zhou et al., 2020) to mitigate the multimodality problem. Formally, the distillation data is mixture of marginal distribution approximating the original data distribution and
14

---

Adaptive End-to-end Quantization Alignment Training Framework for NAT
the teacher model's distribution
arg max sim(ū: (ū
GET
(19)
23 arg max sim y)
уеTk
where sim a function measuring closeness by sentence-level BLEU (Chen & Cherry, 2014), Tk is the K-best list from beam search and go is the teacher model. The distribution learned by the student model should match the mixture distribution
(xD( 5) (1 a)Daata (2, (x.y) v) + oxgo(ilz)
(20)
Notably, the argmax of this mixture distribution is unlikely to correspond to either y (the ground truth the original data) or Y (the beam search output). Thus, distilled data serves as a compromise between real data and data suitable for NAT training, rather than representing real sentences applicable real-world scenarios. Furthermore, distilled data has inherent limitations, such its dependence guidance from a teacher model, compared the richness and potential of the original corpus.
C. Details of Model Components
C.1. Correcting Word Order via Alignment Probability Matrix
The alignment probability distribution matrix A used to reorder the words the input sequence 2, (Hm) match the syntactic order of the target sequence. The steps are as follows:
Matrix Computation
The alignment probability matrix defined as:
A = softmax (H,)))
where each element Ay denotes the probability of aligning input word hy with target position
Algorithm for Reordering
Algorithm Aligned Reordering Process
Require: Aligmment probability matrix A, Input sequence 2g (Hm) = {h1, h2, hy, hs, hs}
Ensure: Reordered sequence AIR(2,, (H,.))
1: Initialize an empty list reordered sequeenee
2: for each target position in the range to (loop target sequence positions y1) do
3: Extract row from A to get alignment probabilities for h2, lgg,.., ...; Amm-
4: Identify column index j with the maximum probability in row i, which corresponds to the alignment of y with hg-
5: Place h2 in position i of reordered sequencee
6: end for
7: return reordered
Outcome
The final seeueere aligns with the syntactic structure of the ground truth sequence and returned as the corrected order.
C.2. Length Predictor
Length prediction can be formulated as a classification problem based on the intermediate representations generated by the encoder. Following (Lee et al., 2018b), we predict the target sequence length m. Given m, the decoder inputs H,, himm
15

---

Adaptive End-to-end Quantization Alignment Training Framework for NAT
computed using Softcopy (Li al., 2018; Wei al., 2019) as:
Wiii softmax(-1 -
(21)
h Σ Wighz,in
where the weight Wiy is determined by the positional distance between the source position i and the target position j. and T a hyperparameter controlling the sharpness of the softmax function.
D. DA-Transformer
The strict position alignment between predicted and target tokens in vanilla NAT models struggles to capture multimodal data distributions, often leading to generated tokens with mixed modality and repeated words. To address this, the directed acyclic decoder state length is upsampled to I., and Hr = [h,, .2....,.. ...,...,,, represents the decoder output hidden states, defined as vertex states. The probability of a path is redefined as the position transition probability:
(a II pe(as+1
II E Em.0.1
(22)
where E E RLLL is the transition matrix normalized by rows. Here, a = fai, a2,..., ,ar) is a set of decoder position indices sorted in ascending order, with size |a| = n, and contains all possible a with a size of (*). For example, if the target length 3 and L = 5, I contains (5) the transition matrix is computed as:
(3) = 10 possible paths, such as {0, 1,2, {0, 1, 1,3). (2, 3, 4}. Specifically,
QK?
softmax
Vd
(23)
Q = HWQ. K HW KG
where is the hidden size, and WQ and WK are learnable parameters. DAT applies lower triangular masking to E, restricting transitions to vertices with smaller indices to larger indices. Conditioned on the vertex states in and the selected path a, posterior probability of y calculated as:
Pe Pe(y(a,z)
II pe (vilas, x) 1=1
(24)
II 1=1
softmax(W, W,ha),,
where ba, is the representation of the i-th vertex the path
E. More Analyses
E.1. Sampling Rate in Training
Table 7. Results IWSLT16 with decreasing sampling ratio.
Sampling Ratio
A
BLEU
0.6
0
26.28
0.6
0.1
26.75
Decreasing
0.6
0.2
29.83
0.6
0.3
31.10
0.6
0.4
32.34
0.6
0.5
30.36
Drawing inspiration from the GLAT methodology, we reduce the sampling rate during training. In contrast to GLAT, our initial state begins at X, 0.6, as AEQA does not depend on explicit target token inputs for dependency learning and
16

---

Adaptive End-to-end Quantization Alignment Training Framework for NAT
16
Relative Decoding Speedup
14
20
Transfommer
CTC
GLAT
SMART
MMLM
DAT
Multi-Task NAT
CMLMrreprreser
10.0
12.5
15.0
17.5
20.0
22.5
25.0
27.5
Performance (BLEU)
Figure 7. The tradeoff between Speedup and BLEU on WMT14 EN-DE.
necessitates richer semantic consistency information. Table 7 demonstrates the influence of different termination values Ae on model performance. As Ae increases from to 0.4, the BLEU score exhibits notable upward trend, reaching its peak of 32.34 at 0.4. Therefore, we establish the training termination value at Ae = 0.4 to fully exploit the model's capabilities.
E.2. Tradeoff
As illustrated in Fig 7, AEQA-NAT demonstrates exceptional performance in terms of relative decoding speed, achieving a speedup that surpasses all other NAT's This result reinforces the advantage of NAT models decoding speed, indicating that ABQA-NAT possesses significant superiority in decoding efficiency. Regarding translation quality, AEQA-NAL also excels, with its BLEU score surpassing that of other NAT models. This indicates that the translations generated by AEQA-NAT closer in quality to the reference translations, further narrowing the performance gap with AT models. Overall, AEQA-NAT achieves optimal performance-speed tradeoff.
17


Reviews:
[{"review_id": "0r5iUCvhFJ", "reviewer": "Reviewer_e42P", "summary": "This paper argues that there is a training-inference gap in Non-autoregressive Transformers (NATs), where NATs sample target words during training to enhance input but have no access to target information during inference. To address this, they propose an Adaptive End-to-end Quantization Alignment (AEQA) training framework, which introduces a semantic consistency space to eliminate the need for target information during inference. Experimental results demonstrate the effectiveness of AEQA, especially on raw data.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your comments. Your suggestions have been very helpful to us.\n1. In the revised manuscript, we have added citations to the relevant literature you mentioned and revised the conclusions in both the abstract and the experimental sections accordingly. \n2. In response to your suggestion, we have changed \"AR\" to \"w/ reordering\" in Table 1 to avoid any potential confusion.", "claims_and_evidence": "No. The claim \"Experimental results demonstrate that our method achieves state-of-the-art performance among fully NAT models on major WMT benchmarks\" is problematic. The state-of-the-art method FA-DAT [1] is ignored, which outperforms AEQA-NAT on raw data.\n\n[1] Fuzzy Alignments in Directed Acyclic Graph for Non-Autoregressive Machine Translation, ICLR 2023.", "methods_and_evaluation_criteria": "Yes. The evaluation covers a wide range of benchmark datasets and metrics.", "theoretical_claims": "There are no theoretical claims.", "experimental_designs_or_analyses": "Yes.", "relation_to_broader_scientific_literature": "Currently, there is limited interest on non-autoregressive machine translation, yet the studies conducted in this area have the potential to inspire broader fields of research, such as the LLM community, which primarily relies on the next token prediction mechanism.", "essential_references_not_discussed": "AEQA-NAT employs an aligned reordering mechanism to align the word order of source and target sentences. The concept of aligned reordering is not new in NAT [1,2], but they are not cited.\n\n[1] AligNART: Non-autoregressive Neural Machine Translation by Jointly Learning to Estimate Alignment and Translate, EMNLP 2021.\n[2] Guiding Non-Autoregressive Neural Machine Translation Decoding with Reordering Information, AAAI 2021.\n\nThe state-of-the-art fully-NAT method on raw data is ignored [3].\n\n[3] Fuzzy Alignments in Directed Acyclic Graph for Non-Autoregressive Machine Translation, ICLR 2023.", "other_strengths_and_weaknesses": "Pros:\n1. This article identifies the training-inference gap issue in NAT, which makes sense and has been overlooked in previous research. The proposed Quantization Alignment method cleverly addresses this issue.\n2. AEQA-NAT achieves impressive experimental results, especially in enabling vanilla NAT w/o distillation to perform nearly as well as the autoregressive baseline.\n3. Both the experiments and analyses are very comprehensive.\n\nCons:\nI do not find significant weaknesses of this paper.", "other_comments_or_suggestions": "The abbreviation \"AR\" for \"aligned reordering\" can be misleading, as AR usually refers to \"autoregressive\". I recommend using a different abbreviation or simply labeling it as \"w/ reordering\" in Table 1.", "questions_for_authors": "None", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "cf800io348", "reviewer": "Reviewer_ifHW", "summary": "This paper works on non-autoregressive machine translation. It bridges the gap by introducing the latent variables and applying glancing training over the latent codes. In addition, order alignment of latent code is also introduced. The empirical results are very good.", "questions": "", "limitations": "", "rating": 2, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your comments.\n## 1. Clarification on Motivation-Method Alignment \nTo address your concern regarding the alignment between our motivation and the proposed method, we would like to clarify the following: We highly value the role of GLAT in NAR systems, which is why we chose it as our baseline model. Our intention to *bridge the training-inference gap* is not to reduce Glancing Sampling to zero (and in our work, we do not aim to eliminate Glancing Sampling), but rather to maintain consistent use of it during both training and inference. Specifically, we predict complete translations in both phases, as opposed to the mismatch seen in training  ($X+Y_{obs} \\rightarrow Y_{mask}$) and inference  ($X \\rightarrow Y$). Therefore, AEQA-NAT is designed to ensure consistency between training and inference, as detailed in Eq. (11) and Eq. (15). To summarize, our motivation is that the training-inference gap hinders NAT from fully realizing its potential, and we have empirically demonstrated this in Fig. 1. We propose AEQA-NAT, which introduces a semantic consistency space to semantically quantize and align the source and target texts. This approach preserves the MLM-based NAT modeling capability while overcoming the explicit reliance on target words, thereby achieving unified prediction of complete translations in both training and inference stages. Thus, our motivation is aligned with the proposed method.\n \n## 2. Differentiation from Latent-GLAT \nWe appreciate your attention to our methodological innovation and would like to clarify the significant differences between AEQA-NAT and Latent-GLAT (as cited on p.9). The following outlines the three key technical distinctions:\n\n- **Latent variable design**: AEQA-NAT introduces a novel *Semantic Quantization Space* (SQS) that jointly models discrete latent variables for both source and target texts during the pre-alignment phase. This bilateral alignment mechanism, driven by the collaborative effect of different loss terms, establishes a unified semantic quantification space that maintains cross-lingual consistency between training and inference phases. In contrast, latent-GLAT employs a unilateral approach that only encodes target-side information, requiring an additional network for latent variable prediction during inference.\n- **Training objective**: Latent-GLAT employs glancing training on both latent variables and explicit tokens during the training phase, optimizing for masked token reconstruction. During inference, it directly predicts complete translations. In contrast, AEQA-NAT directly predicts complete translations in both the training and inference phases, ensuring parity between training and inference.\n- **Architectural foundation**: As detailed in Appendix A.1 (p.13), AEQA-NAT diverges from VAE-based approaches that learn latent features through auxiliary networks. Instead of introducing additional networks to learn latent variables, our Semantic Quantization Space (SQS) **aligns vectorized discrete representations between source and target languages during the pre-alignment phase.** \n \n## 3. Performance Attribution Analysis \nIn response to your concern that \"the main factor of the performance gain is from the latent variables, and has nothing to do with GLAT,\" our manuscript argues that the training paradigm of MLM-based NAT creates a training-inference gap. Therefore, our baseline primarily compares to MLM-based NAT systems. Thus, we focus on comparing the translation performance under Uniform sampling and Adaptive sampling methods, as shown in Table 6. We also present in Table 3 the impact of the sampling rate of GLAT during the inference phase on performance (with a fixed sampling rate during inference). The results demonstrate the effectiveness of the GLAT sampling method.\n \n## 4. Hyperparameters of Glancing Training and Metric\nRegarding your comment on \"the hyperparameters of glancing training not being explored,\" we have discussed the hyperparameters of glancing training in our settings: \n- Training-phase sampling rates: See Appendix E.1 \n- Inference-phase sampling strategies: See Table 3\n \nFor fair comparisons with previous work, we follow established practices from prior research and use sacreBLEU for WMT 17 EN-ZH, and tokenized BLEU for other benchmarks. The experimental results reported using sacreBLEU can be found in Table 3.\n \n## 5. Other Comments and Questions\nLatency comparisons with vanilla Transformer were conducted on identical NVIDIA A40 GPU configurations. DSLP denotes deep supervision and additional layer-wise predictions. We appreciate your comment about the explanation and citation of DSLP, and we have now revised the manuscript to address this issue.", "claims_and_evidence": "The main claim is that the proposed method bridges the training-inference gap caused by Glat. The improved scores on standard datasets support this claim. \n\nThe main claim is not well-justified because the solution is a latent variable model, which naturally enhances the sequence modeling capability. It is not certain if glancing is useful at all in the proposed method. \n\nMy best understanding is that the proposed method is a well-designed latent variable model for NAR machine translation, but I think the motivation and the proposed method are not aligned. \n\nAdditional note: Glat, a training technique for many NAR systems, uses the training-inference gap. In fact, the gap is essential to maintaining performance (reducing it to 0 degrades performance). Thus, the gap may not be a drawback but rather a feature of Glat.\n\nIn fact, there is also a gap between the training and the inference in the proposed method. First, the glancing of the hidden codes are based on the groundtruth tokens, which are not available during the inference. The order alignment also needs ground truth during training.", "methods_and_evaluation_criteria": "This paper uses tokenized bleu for most datasets and includes chrf, COMET, and BLEURT for evaluating translation quality. It is generally valid and comprehensive. \n\nHowever, using tokenized BLEU scores in NAR systems has been criticized over the years, and the paper should report SacreBLEU when possible (Tables 4 and 5).", "theoretical_claims": "There are no poofs in this work.", "experimental_designs_or_analyses": "As mentioned before, the proposed system introduces latent variables and claims to bridge the gap between the training and inference of glat training. This brings the concern that the main factor of the performance gain is from the latent variables, and has nothing to do with glat. \nThere should be at least a baseline line indicating that the glancing of latent code is not used (i.e., No SQ(x) in Eqns 11 and 12).", "relation_to_broader_scientific_literature": "This paper relates to non-autoregressive machine translations, which involve methods like Glat, DAT, NPD, and DSLP.\nIn addition, VQ-VAE, which has broader applications, is also related.", "essential_references_not_discussed": "The latent-Glat is closely related to this work. In fact, many aspects are the same:\n1. Codebook, 2) glancing over latent codes, 3) non-autoregressive latent variables\n\nYu Bao, Hao Zhou, Shujian Huang, Dongqi Wang, Lihua Qian, Xinyu Dai, Jiajun Chen, Lei Li, latent-GLAT: Glancing at Latent Variables for Parallel Text Generation , ACL 2022", "other_strengths_and_weaknesses": "The hyper-parameters of glancing training are not explored, which is an issue as improving glancing is a major claim.", "other_comments_or_suggestions": "The proposed system achieves good performance with relatively low FLOPs compared with models based on DAT or CTC. The authors may consider adding analysis to this regard.\nThe acronym DSLP is not explained nor cited.", "questions_for_authors": "When you compute the latency, what are the specs of the machine, including the GPU and CPU?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "g9mTYwWctY", "reviewer": "Reviewer_EKGr", "summary": "Non-autoregressive transformers (NATs) are attractive due to computational efficiency for machine translation workloads. However, existing approaches fail to completely close training-inference mismatches for these systems. This work proposes Adaptive End-to-End Quantization Alignment Training for NATs (AEQA-NATs) to reduce this gap, optimizing for a few novel training goals and better leveraging a joint semantic embedding space. When successfully applied, AEQA-NATs achieve the fastest decoding throughput of any NAT model and see improvements in minimizing the BLEU score gap between autoregressive and non-autoregressive transformers.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your valuable comments. Your feedback is important to us.  \n1. In response to your suggestion, we have added further descriptions to the sequence of expressions in Section 2.3 to more clearly explain the data flow process:  \n- The source text $(x_1,x_2, \\dots, x_n)$ is input into the encoder $f_{\\text{enc}}$ to obtain the hidden representations $(h_{x,1},h_{x,2}, \\dots, h_{x,n})$.  \n- The length prediction module and the Softcopy mechanism transform the hidden representations into intermediate representations $(h_1,h_2, \\dots, h_m)$.  \n- The semantic consistency representations $(z_q(h_1),z_q(h_2), \\dots, z_q(h_m))$ are obtained by querying the Semantic Quantization Space (SQS).  \n- Finally, the semantic consistency representations $(z_q(h_1),z_q(h_2), \\dots, z_q(h_m))$ and the encoder's hidden representations \n$(h_{x,1},h_{x,2}, \\dots, h_{x,n})$ are fed into the decoder $f_{\\text{dec}}$ for the decoding process.\n2. Regarding your suggestion to \"choose a more modern and improved autoregressive Transformer architecture as the autoregressive (AT) baseline,\" although existing NAT systems have made significant progress in translation quality, their overall performance still lags behind that of the vanilla Transformer. Therefore, the vast majority of NAT systems use the original Transformer architecture as the baseline for AT systems for comparison. We have adopted this approach as well. Your suggestion is valuable and will be an important direction for future exploration.", "claims_and_evidence": "The core claims of this work are clearly outlined in earlier sections and then supported by the methodology and data in Section 3. The ablation results later in this work serve to further support the core claims of this paper.", "methods_and_evaluation_criteria": "Chosen evaluation criteria seem fairly standard for this application space. BLEU, COMET, and similar, typical translation metrics are employed on typical datasets (WMT ’14, ’17, etc.).", "theoretical_claims": "This is largely an empirical work, no significant theoretical claims are made that need to be substantiated.", "experimental_designs_or_analyses": "The experimental design appears fairly standard for machine translation works. As mentioned before, typical metrics are employed. A range of similar NAT models are compared against in a seemingly comprehensive manner. Additionally, a thorough and credible ablation study is conducted.", "relation_to_broader_scientific_literature": "Based on the results observed in this work, the described approach is largely a step forward for machine translation NATs. While somewhat inspired by certain VAEs, the significance of this work appears to be constrained only to efficient machine translation.", "essential_references_not_discussed": "n/a", "other_strengths_and_weaknesses": "Regarding strengths, the paper is very will written, which is particularly helpful as it is fairly information-dense. The proposed approach makes intuitive sense and the shared SQS is a sensible improvement. Additionally, the obtained results make a strong case for the viability of the proposed approach. When it comes to weaknesses, it is a bit unclear how stable the provided results are (i.e. do different training runs result in significant differences in end evaluations). Additionally, using the original transformer architecture as an autoregressive baseline is dubious, although this obviously does not impact the significance of demonstrated improvement upon NAT methods.", "other_comments_or_suggestions": "The series of expressions under Section 2.3 is not well set-up and is an area for improvement on revision passes. While clear upon reinspection, it is dense and a little difficult to initially parse.\n\nTypos: entend -> extend in Section 3 under Implementation", "questions_for_authors": "Why was the original transformer architecture chosen as an autoregressive baselines? With nearly a decade of architectural improvements, surely a more modern baseline could have been chosen for a more reasonable comparison.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "7XvGJoTyGr", "reviewer": "Reviewer_rBqh", "summary": "The paper presents AEQA-NAT, a novel Non-Autoregressive Machine Translation (NAT) framework that introduces a Semantic Quantization Space (SQS) inspired by VQ-VAE. The key components include:\n- Pre-aligned Semantic Quantization Space (SQS) leveraging mBART.\n- Semantic Quantization Alignment Loss (LSQA) to enforce consistency.\n- Aligned Reordering (AR) to improve syntactic alignment.\n\nThese innovations help AEQA-NAT mitigate the training-inference gap, enhancing translation quality while maintaining high decoding efficiency.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your comments. Your feedback has been very helpful to us.\n \n1. The cost of aligned reordering is $O(n \\cdot m \\cdot d)$, and we do not use an additional aligner model during the inference phase. We appreciate your interest in the intuition behind our method. When we recognized the existence of the training-inference gap in MLM-based NAT systems, we naturally hypothesized that this gap would hinder the performance of NAT, and our experimental results confirmed this hypothesis. Inspired by VQ-VAE, we quantize the semantic representation vectors, which forces the model to learn more compact semantic consistency representations. This not only facilitates bridging the training-inference gap but also mitigates the multimodal distribution characteristics in language mapping. Based on the discretized semantic representations, we model the word order correspondence through the aligned reordering mechanism, ensuring the syntactical accuracy of the discrete representations, which further alleviates the multimodal problem in NAT.\n\n2. This work conducts a detailed study on typical datasets widely used in NAT systems (such as WMT ’14, ’16, etc.). We appreciate your mention of low-resource languages in relation to the reliability of our method, and this will be one of the key areas of focus in our future work.", "claims_and_evidence": "The paper claims that AEQA-NAT eliminates the training inference gap and achieves state-of-the-art performance among NAT models. These claims are supported by:\n- Comprehensive experiments comparing AEQA-NAT against strong NAT baselines.\n- Demonstrated performance improvements in BLEU scores, particularly on raw data.\n- Reduced dependency on knowledge distillation, indicating improved generalization.\n\nHowever, AEQA-NAT still lags behind the basic Autoregressive baseline, Transformer, in translation quality.", "methods_and_evaluation_criteria": "The methodology is well-motivated, combining quantization-based semantic alignment with adaptive reordering to address NAT’s dependency modeling issues. The evaluation on standard benchmarks (WMT14, WMT16, etc.) is appropriate, and comparisons with strong NAT baselines validate its performance. \n\nOne concern I have with the method is its reliance on Aligned Reordering (AR), especially for low-resource languages where alignment might be less reliable.", "theoretical_claims": "N/A (this is an empirical paper)", "experimental_designs_or_analyses": "The experiments are extensive across different WMT benchmarks and include ablation studies on SQS size, sampling strategies, and length effects.", "relation_to_broader_scientific_literature": "The paper builds upon previous NAT improvements, particularly knowledge distillation-based approaches and Directed Acyclic Transformers (DAT). Instead of distillation, AEQA-NAT introduces a semantic quantization approach similar to latent variable modeling in VQ-VAE. The work is well-positioned within the broader trend of improving NAT dependency modeling, offering an alternative path to addressing multimodality beyond KD and DAG-based methods.", "essential_references_not_discussed": "N/A", "other_strengths_and_weaknesses": "N/A", "other_comments_or_suggestions": "N/A", "questions_for_authors": "To use the aligned reordering technique, what is the computational cost? Do you compute the alignment with some aligner model on the fly during inference?\n\nPrevious work address the multimodality problem in NAT by applying knowledge distillation or using DAG (as in DAT) to generate coherent translation. In your work, I guess your semantic quantization and aligned reordering do a similar thing right? Can you provide more intuition behind this?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: The paper presents AEQA-NAT, a novel Non-Autoregressive Machine Translation (NAT) framework that introduces a Semantic Quantization Space (SQS) inspired by VQ-VAE. The key components include:

Pre-aligned Semantic Quantization Space (SQS) leveraging mBART.
Semantic Quantization Alignment Loss (LSQA) to enforce consistency.
Aligned Reordering (AR) to improve syntactic alignment.
These innovations help AEQA-NAT mitigate the training-inference gap, enhancing translation quality while maintaining high decoding efficiency.


This article identifies the training-inference gap issue in NAT, which makes sense and has been overlooked in previous research. The proposed Quantization Alignment method cleverly addresses this issue. AEQA-NAT achieves impressive experimental results, especially in enabling vanilla NAT w/o distillation to perform nearly as well as the autoregressive baseline.

For the revision: please include more discussion on Latent GLAT in the revision and VQ-VAE.
Decision: Accept (poster)