Quantization-Aware Diffusion Models For Maximum Likelihood Training
Authors: Shohei Taniguchi
Masahiro Suzuki
Yutaka Matsuo
Venue: ICLR 2026 Poster (2026)
Keywords: diffusion model
dequantization
Primary area: generative models
TL;DR: Quantization-aware diffusion (QDPM) makes the reverse SDE provably land on discrete pixel values, enabling maximum-likelihood training and achieving SoTA NLL.

Abstract: Diffusion models are powerful generative models for continuous signals, such as images and videos.
However, real-world digital data are quantized; hence, they take not continuous values but only a finite set of discrete values.
For example, pixels in 8‑bit images can take only 256 discrete values.
In existing diffusion models, quantization is either ignored by treating data as continuous, or handled by adding small noise to make the data continuous.
Neither approach guarantees that samples from the model will converge to the finite set of quantized points.
In this work, we propose a methodology to explicitly account for quantization within diffusion models.
Specifically, by adopting a particular form of parameterization, we guarantee that samples from the reverse diffusion process converge to quantized points.
In experiments, we demonstrate that our quantization-aware model can substantially improve the performance of diffusion models for density estimation, and achieve state‑of‑the‑art results on pixel‑level image generation in likelihood evaluation.
In particular, for CIFAR‑10 image generation, the negative log‑likelihood improves substantially from 2.42 to 0.27, approaching the theoretical lower bound.

Published as a conference paper at ICLR 2026
QUANTIZATION- DIFFUSION MODELS FOR MAXIMUM LIKELIHOOD TRAINING
Shohei Taniguchi, Masahiro Suzuki & Yutaka Matsuo
Department of Technology Management for Innovation The University of Tokyo
7-3-1 Hongo, Bunkyo-ku, Tokyo, Japan (taniguchi, masa, matsuoo]ewbbles t. w.w.t.....o.........oo
ABSTRACT
Diffusion models are powerful generative models for continuous signals, such as images and videos. However, real-world digital data quantized; hence, they take not continuous values but only a finite set of discrete values. For example, pixels in 8-bit images take only 256 discrete values. In existing diffusion models, quantization is either ignored by treating data as continuous, or handled by adding small noise make the data continuous. Neither approach guarantees that samples from the model will converge to the finite set of quantized points. In this work, propose methodology to explicitly account for quantization within diffusion models. Specifically, by adopting a particular form of parame- terization, we guarantee that samples from the reverse diffusion process converge to quantized points. In experiments, we demonstrate that our quantization-awal model can substantially improve the performance of diffusion models for density estimation, and achieve state-of-the-art results on pixel-level image generation in likelihood evaluation. particular, for CIFAR-I0 image generation, the negative log-likelihood improves substantially from 2.42 to 0.27, approaching the theoret- ical lower bound.
INTRODUCTION
Diffusion probabilistic models (Sohi-Dickstein al. 2015 Ho et al. 2020) and score-based gen- erative models (Song & Ermon 20199; 2020) have achieved ofate-of-the-ar performance in terms of sample quality and density estimation (Kingma et al. 2021 Sahoo et al. 2024) for image gen- eration. Both models consider to perturb data with sequence of noise distributions, and generate samples by learning to reverse the diffusion process from noise to data. [Song et al. (2020b) have shown that these two types of models can be interpreted using single framework, which refer to as diffusion models in this paper.
The framework of diffusion models (Song et al. 2020b) involves gradually diffusing the data dis- tribution towards a simple noise distribution, such as a Gaussian distribution, using a stochastic differential equation (SDE), and learning the time reversal of this SDE for generative modeling. The reverse-time SDE has an analytic expression which only depends on a time-dependent score func- tion of the perturbed data distribution. This score function can be efficiently estimated by training neural network (called a score-based model (Song & Ermon 2019 2019)2020) with a weighted combi- nation of score matching losses (Hyvärinen Dayan 2005 Vincent 2011 Song et al.) (020a) as the objective. After training, we can obtain samples from the model by simulating the reverse SDE from a simple noise using estimated score function.
By definition, diffusion models are designed for continuous-valued data, and have been applied to continuous domains (e.g., images (Croitoru et al. 2023), videos (Ho et al., 2022b and au- dio (Zhang et al.] 2023 Huang et al., 2023)). However, even in such domains, digital data do not take continuous values in a precise sense due to quantization. For example, a pixel in 8-bit images can take only an integer value from 0 to 255 rather than real continuous value. In many previous works of diffusion models, quantization is largely ignored during training, since its effect seems negligible the performance, and they only apply it to generated samples during inference as post-

---

Published as a conference paper at ICLR 2026
Reverse SDE
X
Quantization via Post-processing
Diffusion
QDPM (ours)
Quantization Inside Reverse SDE
Figure 1: Overview of our QDPM. In the traditional diffusion models (above), the reverse SDE does converge quantized points the data lie on, quantization is performed post-processing after SDE simulation. On the other hand, our QDPM (below) incorporates the quantization process inside the reverse SDE, which explicitly guarantee convergence to the quantized point without post- processing.
Test NLL for CIFAR-10
2.9
2.83
2.65
2.56
2.55
2.49
2.42
2.25
0.75
0.27
Image
i-DODE w/ aug.
i-DODE
Transformer
W/ aug.
(ours)
Figure 2: Comparison of the test negative log- ligeliidld of our QDPM for CIFAR- 10 with existing methods. QDPM achieves the state-of-the-at performance by large margin.
processing (Karras al. 2022). In some studies, they deal with quantization by adding small noise to quantized data before applying them to diffusion model so that the data lie on a continu- space. However, such preprocessing leads to the trained model generating noisy data that may deteriorate the performance.
In this paper, propose method that directly incorporates quantization into the design of score- based diffusion models, which can discard such ad hoc pre- or post-processing. Specifically. we derive sufficient condition for the reverse-time SDE converge to quantized value shown Figure [ and propose a specific parameterization to meet the condition. We refer to the model trained under this framework the Quantizing Diffusion Probabilistic Model (QDPM). In the ex- periments, we show that our QDPM can substantially improve the performance of pixel-level image generation. In particular, QDPM achieves a negative ligeelilioood of 0.27 bpd on CIFAR-10, which beats the current state-of-the oftte-of-hhe-ar result of 2.42 by i-DODE (Zheng et al. 2023) by a large margin.
BACKGROUND
2.1 DIFFUSION MODELS
Diffusion models (Sng al. 2020b) generative models defined via stochastic differential equation (SDE) known as the diffusion process. For example, letting the data be Xn E Rdd, consider the following variance-exploding (VF) SDE.
2

---

Published as a conference paper at ICLR 2026
dx; 22t dw
(1)
Let de and for denote the marginal distribution time L and the transition density of this SDE, respectively.
qui (x; xo) N (x,; PI)
The time-reverse process of this forward SDE can be analytically derived as follows:
dx, -t S, (xe) dt + 22t dw,
(3)
where 8t is the score function of the noisy data distribution ge at time i, i.e., (x{) = Vx, log de (x,). If we can simulate this reverse process, we can model the generative process of the data distribution. However, the score function is unknown and must be estimated. To that end, we consider model Sy (x,, t) that estimates the score function and train it so that the estimated reverse SDE matches the true SDE.
dx; = 21 Sq (Xe, () + 22 dw
Let Ps denote the marginal distribution at time for this (estimated) reverse SDE. Since it suffices that the scores match at any time t, we minimize the following score-matching loss.
ISM (0) = ExorpPaata
w (t) [80 (xx) - so (xt, t)]l
(5)
Amin
where w(t) is a positive weighting function over time, i.e., w(t) > 0, 0 < 4 < max < oo, and I1-I1 denotes the L²-norm. Because 8, unknown, we cannot compute this directly; however, is known that minimizing JsM is equivalent to minimizing the denoising score-maie matteman loss below.
JDSM (0) = E Expexpaatu e~N(0.I)
w (t) - 6 so (x+
(x:xxxx = + t. c.t) dt
(6)
Amin
Afmax w (1)
Exorppaaaa e>N()
|le én (x==x + 1)11
t2
Amin
Jsm (0) + const.
Here set éø -t $8. In other words, if we train model éo predict the added noise from noisy data Xx, we can effectively estimate the underlying score function. If, instcad of a noise predictor, define signal predictor Xe- t. Eo. then JOSM be rewritten follows:
fima w (t)
JOSM (0) Exorxpanta e0.()
8xo X0 (x: xo e, 1)1) di
1
Furthermore, when w(t) it is known that minimizing JSM corresponds maximizing lower bound on the likelihood of po-
[
ffmax
- llg Po 0) < Ee-Nio.)
||xo - Xo (X+ Xo + l. 6)]] di
+ ci +
(10)
Amin 13
x l?
C1 = -Ea-N10.1 [log po (Xo Ximin = Xo + Emin <)] C2
(11)
21
max
Up this point considered VE-SDE, but by applying change of variables x, derive analogous forms for other SDEs such as variance- prieieceeereerviiggg (VP) and straight- (ST) SDEs. For example, if we set Yt = x.VI++/ the process follows a VP-SDE, while y: = x,/(1 + t) yields ST-SDE.
3

---

Published as a conference paper at ICLR 2026
2.2 DEQUANTIZATION FOR DIFFUSION MODELS
Standard diffusion models assume that the data X take continuous values. In contrast, real-world digital data such as images quantized and therefore, take only finite set of values rather than true continuous values. This quantization is ignored in many prior works, which typically treat the data directly as continuous (Karras et 2022) or use diffusion models in the continuous latent space in the VAE (Rombach et al. 2022 Peebles & Xie 2023).
When one wants evaluate models by likelihood, however, it is common handle quantization explicitly in order to compare against quantization-awan
models such as autoregressive models (e.g., PixelCNN (Van Den Oord et al. [2066] Salimans et al. (017)) The simplest and most common approach is to treat the data with small added noise as and evaluate the likelihood of the quantized data via a variational bound (Song et al., 2021 Zheng et al. 2023). For example, with 8-bit images whose pixel values are integers in 0 ~ 255, pixel values to make them continuous. As a related alternative, one sets the start time of the diffusion process min to small positive value and defines Pg (Xo Xtmin as probability distribution over the quantized discrete space (Kingma et al. 2021 Sahoo et al. 2024). In this case, po (Xo Xtm.n) becomes categorical distribution:
often first adds ii.d. uniform noise in [0, 1) to the
Pe (Xo Xtmin
II pe ((xo); x(mm))
(12)
(zz(k) )i)
po ((xo): = (8) ((mmn) softmax K {=(*)] k=1 In the case of an 8-bit image pixel, K = 256 and (A) = k While these methods are convenient for likelihood evaluation, they are ad-hoc and do not guarantee that the estimated reverse SDE will converge to quantized points.
(13)
&
21
where we suppose each element of the data (e.g., each RGB pixel) is quantized to value in
3 METHOD: QUANTIZING DIFFUSION PROBABILISTIC MODELS
We propose methodology that designs quantization aware score function so that the reverse SDE explicitly converges to quantized points; we call this approach Quantizing Diffusion Probabilistic Models (QDPM). To guarantee this, first analyze the solution of the reverse SDE and derive sufficient conditions. Based on it, we then design a score function that satisfies those conditions.
3.1 QUANTIZATION-AWARE PARAMETERIZATION
Here we consider the case where a signal predictor &e is parameterized. Given an initial condition x,, the solution for x, at time t of the reverse SDE for 0 stss< co can be derived as follows:
22
2 X (x., r)
(x,, dr 8 f.
2 [ V dww.
X
22
(14)
See the appendix for the derivation. For the reverse SDE to converge to quantized point, we have to ensure that the limit of this solution at 0 always lies a quantized point. As regards the limit point Xo. the following fact holds under some regularity conditions:
Proposition 1 (Convergence to a fixed point, informal). For the stochastic differential equation defined by Eq. (4)), the solution X at meets the following equality almost surely.
Xy xx (Xo, 0).
(15)
This means that the limit point xo is always fixed point of the signal predictor Xa (-,)). To verify
whether this holds in practice, we measure the fixed-point error, i.c., Xx, Xs (Xe, f /d, each
time step when sampling from the pretrained EDM (Karass et al. 2022) CIFAR- 10 and AFHQ
Note that, in many cases, pixel values are scaled and shifted to standardized range (e.g.. [-1, 1) rather than indices via preprocessing. but we omit it for simplicity here.
4

---

Published as a conference paper at ICLR 2026
6000
6000
5000
MSE
3000
3000
2000
2000
1000
1000
40
60
0
20
60
80
(a) CIFAR-I0
(b) AFHQ
Figure 3: Fixed-point error of the pretrained EDM on CIFAR-10 and AFHQ at sampling steps. The error bars show the standard deviation of 64 samples.
and plot it in Figure As can be seen from it, the fixed-point error converges to zero -> 0, confirming that Proposition holds true in practice. Therefore, by limiting the fixed points of the guarantee that the solution of the reverse SDE at t + lies on them. Based on this fact, we can derive a sufficient condition for the SDE to converge to quantized points as follows:
signal predictor &e to quantized points,
Proposition 2 (Sufficient condition for convergence to a quantized point, informal), If the signal predictor Ka meets the following equality for any E Rd, the solution Xo at takes value in a set of quantized points, ie.,, Ω = (12)())) jk=1) almosst surely.
Eg xx, 0) round (x) == arg min ||x 15---- yll
(16)
yen
When this condition is satisfied, it is obvious that the limit point always lies at quantized point due to Eq. (15). fact, we can construct a signal predictor &a that satisfies the above condition as follows:
Sy (xx, t) softround (xx. -1/12² 80 (xe, t),
(17)
(softround (x,, t)), = Σ softmax (- (_((())(xx))) (xx)))
x (k)
(18)
2t2
where (-), is the i-th element of vector, and d is some parametric function, e.g., neural network. Intuitively, softround (.) smooth version of round (.) and converges to as - Since the second term in Eq. (17 diminishes + 0, the sufficient condition is always satisfied under this parameterization.
3.2
MAXIMUM LIKELIOOOD LOSs FOR QDPM
As described in Section 2.1 the negative log- ligllikel of diffusion models admits the upper bound in Eq. (TO). Taking the limits bmin + and Imax + Do yields Cı, - 0, so we obtain the following (negative) evidence lower bound (ELBO):
[
- log (Xo: 0) S EE-N(O.))
Xo
So xx xo + e) at = L (xo, 0)
(19)
If we omit the terms in L that are independent of the model de, we can rewrite the loss as follows:
e 1/12
L (Xo, 0) E
83
1/- d0
25. dt
+c,
(20)
|
where 5, Xg softround (x,, t),
dt
(21)
t
5

---

Published as a conference paper at ICLR 2026
Algorithm @DPM-Soller-1
Xs, U, <- 0,
for i + to .M do
ut us 1/M
t + 1/v - log Us, 1/v- log
Xo (x,,8) softround (x,; + us d0 (x.: 8)
X, ~ W x +
&o (X.: 8), 12 (1- 1)
Ug ues Xs t- X,
end for return X₁
With the change of variable u = e -1/t² this can be further written in a simple form:
L (xo, 0) = E
000
e
1/t²
50
1/12 5.
-26,
dt
+
(22)
t
0
1/t²
2e 1/12
1/P
E
8
6g
8
dt
+
(23)
2
13
E
8
80
5,
du
+ E
8T
11 80
51)]
(24)
2
where (0, t 1/v- du 2/-1/e2 dt. Therefore, aware diffusion model with this maximum likelihood objective.
train the quantization-
Regarding the design of 8g. we use a U-Net-type architecture following the prior works (Ho et al, 2020 Kingma al. 2021 Karras et al. 2022). However, since our formulation is based on VE-SDE, the noised data x, can take large value especially when ( is large. To normalize the magnitude, we input x, x./vI+ + 1 to the neural net instead of the
Similarly, also use e 11/t as input instead of f, since u has a bounded range of [o, 1] whereas unbounded.
3.3 EFFICIENT SDE SOLVER FOR QDPM
Since already have the exact solution for the reverse SDE in Eq. (14, can construct an efficient solver rather than using a naive Muler-Maruyama solver. The simplest way is to use the first-order approximation for the drift term follows:
t
{
2 p
2 dw,
X,
82
8
Xg
(x,,
r) +1
(25)
12
12 ["
Xo (x,, (),,))r dr +2 {" pd
X
p3
dw,
(26)
12
W x, +
(1-
1
X0 8), la
(1-5))
(27)
By running it for discretized points in L [0, oo). variable can take value in an unbounded range, which is difficult to discretize, we choose the discretized points in the space of 1//² E [0, 1), and run the update of x; by transform it into The algorithm of this QDPM-Solver is summarized Algorithm We can also derive a second- order alternative of this solver using the scheme of Runge Kutta methods shown in Algorithm 2
simulate the reverse SDE. Since the time
In fact, derived SDE solver is closely related to DPM-Solver++ (Lu et al. 2022b). DPM- Solver++ is also derived using a closed-form solution using the signal predictor Xg. but its formula- tion is based on VP-SDE instead of VE-SDE. In addition, DPM-Solver++ chooses the discretization points in the space of
log whereas take them in the space of u e 1l/t? In our QDPM setting, t can take a value in [0, oo), so X-based discretization is not feasible because also takes value unbounded range (-00,00).

---

Published as a conference paper at ICLR 2026
Algorithm 2 QDDM-Sollve-2
Xs. U, <- 0,
for i + to MM do
Ur 4t U2
2/M,, Us 1/M
7, t - 1/v- log Usa
1/ - log Up, 1/ log uh
&, N 0,
5 E: ~ N S (1- 1)
80 (x,, 5) softround (x2, 8) + U, do (x.: 8)
Xr + x2 (1 X0 (x,, s) + &r
Xo (x,,) softround (xy: + Uy 80 (x,, r)
X, x= + # ( (x., 5) 9&00 r)) + E.
U4 - ur, X, f X
end for return X₁
4 RELATED WORK
4.1 DEALING WITH DATA QUANTIZATION IN DIFFUSION MODELS
Although many existing works ignore the data quantization, and treat the raw data continuous signal, some studies take it into account for their model design, especially when they want to perform the likelihood- based evaluation. For example, the original DDPM (Ho et al. 2020) uses discritized Gaussian distribution for Pa (Xo Xtm..), and explicitly quantize the output of the diffusion model, even though they use discrete-time model instead of continuous L. VDM (Kingma al., 2021) also
categorical distribution over the quantized points as in Eq. (13). Some works (Song et al., 2020b 202I) use a technique called dequantization, in which small continuous noise is added to data before passing it to the models, and the likelihood the original quantized data is calculated as a variational bound. For example, with 8-bit images whose pixel values are integers in 0 ~ 255, uniform noise in [0, 1) to the pixel values to make them continuous. In this case, the negative log- likelihood (NLL) can be upper bounded using Jensen's 's inequality as follows:
explicitly deals with the quantization by simply defining po (xo Xt
often first adds i.i.d.
logg (x; 0) = - log {
p = x u: 0) du I -Eu-2((0.1) [logp = =x=xxx x u: u) (28) /..1"
In many cases, the dequantized data is scaled and shifted in range [-1, 1]: hence, by defining y x/128 1,
treat the (continuous) NLL of y as an upper bound of the NLL of the
original quantized data x as follows:
log (x; < -Eu-2 blogp (x u;²)]
(29)
-EE (0,)((
logp y
x
1:00
+ log det dy
(30)
128
dx
(0.1))
log
y
X +
1:0)
}:
+ 7d.
(31)
128
The dequantization method is not only applied to diffusion models but also to any generative models for continuous-values data, such as normalizing flows (Ho al., 2019). More sophisticated noise, such as truncated Gaussians, rather than uniform noise is often used for dequantization (Zheng et al., 2023 Song et al. (021) in prior works. In contrast these works, our QDPM incorporates the quantization process inside the design of reverse SDE, which can eliminate any ad hoc operations from both training and inference.
4.2
MAXIMUM LIKELIHOOD TRAINING OF DIFFUSION MODELS
Originally, Ho et al. 2020 derived maximum likelihood objective for the discrete-time diffusion model. but they experimentally show that a simpler noise prediction loss performs better in terms of the sample quality. After (Sng al. (20205) reformulate the continuuus-timm diffusion model using
7

---

Published as a conference paper at ICLR 2026
stochastic differential equations, Song et al. (2021) and Huang et al. (2021) derive the correspond- ing ELBO objective for it. In previous works of maximum likelihood training for continuous time diffusion models, the time variable t is typically truncated within a predefined bounded region in [Imin: 4max), because training tends to suffer from numerical instability, otherwise. We assume that such numerical instability is related to the lack of quantization awareness of diffusion models in prior work. As can be seen from the ELBO objective in Eq. (TO). the upper bound of negative log-likelihood can easily diverge if min approaches zero, since the coefficient t-3 will be divergent. Therefore, training with the ELBO objective will be intractable when we want to set small min- Kim al. (2022) try alleviate this numerical instability by introducing technique called soft fruncalion, in which the truncation time mmin is randomly chosen during training. Although soft truncation alleviates the numerical instability during training, it still requires the choice of a mini- mum truncation time 'min- On the other hand, our QDPM parameterization in Eq. (17) ensures that the signal predictor Xg approaches to the original quantized signal X as + 0 at super- expeeenapeet rate by definition. Hence, the divergent behavior of the ELBO objective can be avoided, making maximum likelihood training feasible without time truncation.
4.3 PARAMETERIZAATION OF DIFFUSION MODELS
In the original paper by Song et al. (220b)), the noise predictor éω directly parameterized by neu- ral network (e.g., U-Net), and many subsequent works follow that parameterization. However, some variants are also proposed in the previous works, such as the signal predictor (Karras et al.) 2022 Wei et al. 2023), the velocity predictor (Salimans & Ho 2022 Zheng et al. (023). In our QDPM parameterization, we can interpret that the function oa predicts d, which is the difference between the data signal Xg and the smoothly rounded value of the current state, i.c., softround (x,, t), with a scale of u. since the loss function in Eq. (24) can be rewritten as follows:
5. $1
C (xo, 0) E
8,
(32)
2z
This quantization-awaat parameterization contributes to a drastic improvement in density estimation for quantized signal data (e.g., images) as described in the next section.
4.4 SDE SOLVERS FOR DIFFUSION MODELS
Originally, Song et al. (2200b) use hybrid approach that combines the Euler-Maruyama solver and Langevin dynamics, which they call prdiccor-correctoo sampling. In subsequent studies, more sophisticated approaches have been proposed. Karras et al.) (2022) uses second-order solver based on Heun's ' method to decrease the discretization errors. Lu et al. (2022b) proposes a SDE solver named DPM-Solver based on the exact solution of the reverse SDE, which is similar QDPM solver except for the choice of discretization points. Although we focus on SDE solvers in this paper, ODE solvers for diffusion models (and flow matching) have been proposed in many prior works (Karras et al. 2022, Lu et al. (2022a). We leave the derivation of an ODE alternative of our QDPM-Solver for future work.
EXPERIMENTS
Datasets: To demonstrate the effectiveness of our QDPM, we perform experiments of image gen- eration and density estimation tasks. In our experiment, we use the CIFAR-10 10 and downsampled ImageNet-32 32 (Deng al., 2009) datasets for density estimation. In some previous works (Parmar et al. 2018 Chen et al.] 2018 [Song et al.] (021). the old version of ImageNet-32 is used, which is no longer available, so we clarity it in the table. We also train QDPM for Facckr-Face-HH (FFHQ. and Animal-Faces-HC HQ (AFHQ) datasets with 64 x 64 resolution for qualitative evaluation. We do not employ any data augmentation during training, since we observe that QDPM can achieve good density estimation results without it. Regarding the model architecture, use the U-Net-based model used in Kingma et al.] (2021), which is one of the most commonly used models for image density estimation (Zheng et 2023] Sahoo et al. 2024).
Evaluation: We evaluate the model performance with the negative log- ligellioddood (NLL) of the reverse SDE Following prior works, we report the biss-per-dimension (BPD) score, which corre-
8

---

Published as a conference paper at ICLR 2026
Density estimation performance CIFAR-10 and ImageNet-32. Negative log log-likelihood (NLL) in bits per dimension (BPD) is reported.
Table 1:
Corresponds to the old version of ImageNet-32.
Model
CIFAR-10
ImageNet
Image Transformer Parmar et al.
2018
2.90
3.77
DistAug (Jun al.
2020)
2.53
Very Deep VAE (Child,
2021 2021
2.87
3.80"
PixeISNAIL (Chen et 2018
2.85
3.80*
ScoreFlow (Song et al. 2021)
2.83
3.76
VDM (Kingma et al. 202T
2.65
3.72"
VDM (w/ uniform dequantization)
2.85
3.76
VDM (w/ data augmentation)
2.49
iDDODE (Zheng et al. 2023)
2.56
3.43
i-DODE (W/ data augmentation)
2.42
MULAN (Sahoo et al. 2024
2,55
3.67
QDPM (ours)
0.27
0.32
Theoretical lower bound
0.0043
0.0051
(a) FFHQ
(b) CIFAR-10
(c) AFHQ
(d) ImageNet-32
Figure 4: Non-cherrypicked samples of QDPM for (a) Flickr-Faces-HC (FFHQ) (b) CIFAR-IO, (c)
Anisal-Faces-HQ (AFHQ), and (d) ImageNet-32 dataset.
sponds to the NLL scaled by the data dimension d with the log base 2 as follows:
BPD -
E Paatn [ogg, P (x: = dllog E (x:
(33)
For our QDPM, we calculate the upper bound of the BPD using Eq. 24 We also report the Fréchet inception distance (FID) of the generated images via SDE solver for the evaluation of perceptual quality, although we do not tune our QDPM for it, since our main focus on density estimation. Since the negative lig-likelihood for the reverse SDE is intractable, we report its upper bound cal- culated with Eq. (19
The results summarized in Table Our QDPM demonstrates the state-of-the-arr result for both CIFAR-10 and ImageNet-32 dataset in terms the test likelihood. The test negative log- logllikel (NLL) scores improved from the previous SoTA methods over 2.0 (bpd). Since CIFAR-10 has 10,000 test images with the resolution of 32 32 of RGB color channels, the theoretical lower bound of the test NLL. is log, (10000) (3 x 322) 0.0043 (see Appendix C for the derivation). To the

---

Published as a conference paper at ICLR 2026
(a) NFE=4
(b) NF
(c) NFE=64
(d) NFE-256
Figure 5: FFHQ samples by QDPM-Solver-2 with number of function evaluations (NFEs) in a range of 4 ~ 256.
best of our knowledge, our QDPM is the first result that achieves the test NLL under 1.0, approaching the theoretical lower bound. For qualitative evaluation, generated samples by our QDPMs are shown in Figure It can be seen that our QDPM can generate high-quality samples without cherrypicking. Regarding the FID score, we observe 5.60 for CIFAR-10 10 and 8.89 for ImageNet-32. These are still worse than the SoTA FID scores, such as 1.54 of GDD (Zheng
Yang 2024) for CIFAR-10. Previous works also have reported that the test NLL and FID score do not tend to correlate, and diffusion models trained with a maximum likelihood objective tend to show worse FID scores (Ho) al. 2020 Song et al. 2021 Zheng et al.) 2023); hence, this is natural phenomenon. We also provide a qualitative comparison of generated samples when the number (NFEs) changes in Figure 5 We use the second-order QDPM-Solver,
function evaluations NFE corresponds to twice the number of discretization steps (i.e., 2 x M in Algorithm 2 It can be observed that our QDPM- Solver can generate high-quality samples even with relatively small NFEs (e.g., 16 64).
6 CONCLUSION
In this paper, we propose the Quantizing Diffusion Probabilistic Model (QDPM), whose generated samples are guaranteed to converge to a quantized point. This is achieved by designing the signal prediction model so that its fixed points corresponds to quantized points described in Section 3 We also provide efficient SDE solvers for QDPM, which are derived using the closed-form solution of QDPM's reverse SDE. We experimentally show that QDPM can substantially improve the performance of diffusion model for image density estimation, and achieves the state-of-the-art re- sults for test negative log-likelihood by large margin. While we have focused the SDE-based formulation throughout the paper, we believe it can be extended to an ODE-based formulation as in flow matching; we leave this for future work. In addition, developing more efficient solvers with higher-order convergence guarantee is also a promising direction for further improvement.
REFERENCES
Björ Barz and Joachim Denzler. Do we train on test data? purging cifar of near-duplicates. Journal of Imaging, 6(6):41, 2020.
Xi Chen, Nikhil Mishra, Mostafa Rohaninejad, and Pieter Abbeel. Pixelsnail: An improved au- toregressive generative model. In International conference on machine learning, pp. 864-872. PMLR, 2018
10

---

Published as a conference paper at ICLR 2026
Rewon Child. Very deep generalize autoregressive models and can outperform them images. In International Conference on Learning Representations, 2021.
Florinel-Alin Croitoru, Vlad Hondru, Radu Tudor Ionescu, and Mubarak Shah. Diffusion models in vision: A survey. IEEE Iransactions on paltern analysis and machine intelligence, 45(9): 10850-10869, 2023.
J. Deng. W. Dong, R. Socher, L.-J. Li. 1. K. Li, and L. Fei-Fei. ImageNet: A Large-Scale Hicrarchical Image Database. CVPR09, 2009.
Jonathan Ho, Xi Chen, Aravind Srinivas, Yan Duan, and Pieter Abbeel. Flow++: Improving flow- based generative models with variational dequantization and architecture design. In International conference on machine learning, pp. 2722-2730. PMLR, 2019.
Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in neural information processing systems, 33:6840-6851, 2020.
Jonathan Ho, William Chan, Chitwan Saharia, Jay Whang, Ruiqi Gao, Alexey Gritsenko, Diederik P Kingma, Ben Poole, Mohammad Norouzi, David J Fleet, et al. Imagen video: High definition video generation with diffusion models. arXiv preprint rrXiv:22/0.0..000 2022a.
Jonathan Ho, Tim Salimans, Alexey Gritsenko, William Chan, Mohammad Norouzi, and David Fleet. Video diffusion models. Advances in neural information processing systems, 35:8633- 8646, 2022b.
Chin-Wei Huang, Jae Hyun Lim, and Aaron C Courville. A variational perspective diffusion-based generative models and score matching. In M. Ranzato, A. Beygelz- imer, Y. Dauphin, P.S. Liang, and Wortman Vaughan (eds.), Advances in Neural In- formation Processing Systems, volume 34, pp. 22863-22876. Curran Associates, Inc.,
2021. URL httpp::/pproceeeiigggs.... neurips cctppper.rrpeeeeeeeeeeeeeeeee files/paper/2021/
Rongjie Huang, Jiawei Huang, Dongchao Yang, Yi Ren, Luping Liu, Mingze Li, Zhenhui Ye, Jinglin Liu, Xiang Yin, and Zhou Zhao. Make-an-audio: Text-to-audio generation with prompt-enhanced diffusion models. In International Conference on Machine Learning, pp. 13916-13932. PMLR, 2023.
Aapo Hyvärinen and Peter Dayan, Estimation of non-normalized statistical models by score match- ing. Journal of Machine Learning Research, 6(4), 2005.
Heewoo Jun, Rewon Child, Mark Chen, John Schulman, Aditya Ramesh, Alec Radford, and Ilya Sutskever. Distribution augmentation for generative modeling. In International Conference on Machine Learning. pp. 5006-5015 PMLR, 2020.
Tero Karras, Miika Aittala, Timo Aila, and Samuli Laine. Elucidating the design space of diffusion-based generative models. In Alice H. Oh, Alekh Agarwal, Danielle Belgrave, and Kyunghyun Cho (eds.), Advances in Neural Information Processing Systems, 2022. URL https:://ppeeevieeee .et/ feet/forum's
Dongjun Kim, Seungjae Shin, Kyungwoo Song, Wanmo Kang, and II-Chul Moon. Soft truncation: A universal training technique of score-based diffusion model for high precision score estima- tion. In Kamalika Chaudhuri, Stefanic Jegelka, Le Song, Csaba Szepesvari, Gang Niu, and Sivan Sabato (eds.), Proceedings of the 39th International Conference on Machine Learning, volume 162 of Proceedings of Machine Learning Research, pp. 11201-11228, PMLR, 17-23 Jul 2022. URL https: http://pocccccccee/e////p/aa U
www.r.press/vee vilss//v122////22 wttps//s/vv22/immm2222 html
Diederik Kingma, Tim Salimans, Ben Poole, and Jonathan Ho. Variational diffusion models. In M. Ranzato, A. Beygelzimer, Y. Dauphin, P.S. Liang, and Wortman Vaughan (eds.), Advances in Neural Information Processing Systems, volume 34, pp. 21707 Curran Associates, Inc., 2021. URL wtt.:ttsss/pppcceein....ee.ooooo haap:r/000AAaaaa1aaa322222aaaaa0
filpaapppp_ele//eapp__aaaaeaasae files/

---

Published as a conference paper at ICLR 2026
Cheng Lu. Yuhao Zhou, Fan Bao, Jianfei Chen, Chongxuan LI, and Jun Zhu. Dpm-solver: A fast ode solver for diffusion probabilistic model sampling in around 10 steps. In S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh (eds.), Advances in Neu- ral Information Processing Systems, volume 35, pp. 5775-5787. Curran Associates, Inc., 2022a. URL httpppss///prrrceddngg.gg. neurips cc wwtpppppppppppeeeeeeeee saappper_rfleeeeee httppppper_felepppppp////0 FlLc720b.4ccce2ssaccbbbeeeeeeeee Paper Con.eernccnrrreceeep.oo
Cheng Lu, Yuhao Zhou, Fan Bao, Jianfei Chen, Chongxuan Li, and Jun Zhu. Dpm-solver++: Fast solver for guided sampling of diffusion probabilistic models. arXiv preprint arXiv:2211.01095, 2022b.
Niki Parmar, Ashish Vaswani, Jakob Uszkoreit, Lukasz Kaiser, Noam Shazeer, Alexander Ku, and Dustin Tran. Image transformer. International conference on machine learning, pp. 4055- 4064. PMLR, 2018.
William Peebles and Saining Xie. Scalable diffusion models with transformers. In Proceedings of the IEEE/CVF international conference on computer vision, pp. 4195-4205, 2023.
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bjöm Ommer. High- resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF confer- ence on computer vision and pattern recognition, pp. 10684-10695, 2022.
Subham Sahoo, Aaron Gokaslan, Christopher M De Sa, and Volodymyr Kuleshov. Diffusion models with learned adaptive noise. Advances in Neural Information Processing Systems, 37:105730- 105779, 2024.
Tim Salimans and Jonathan Ho. Progressive distillation for fast sampling of diffusion models. In International Conference on Learning Representations, 2022. URL ptttt::://opereeeiee
net bet/Eoruue htt//oorummiid TIdIXIpzhoI
Tim Salimans, Andrej Karpathy, Xi Chen, and Diederik P Kingma. Pixelcnn++: Improving the pixelenn with discretized logistic mixture likelihood and other modifications. In International Conference on Learning Representations, 2017.
Jascha Sohl-Dickstein, Eric Weiss, Niru Maheswaranathan, and Surya Ganguli. Deep unsupervised learning using nonequilibrium thermodynamics. In International conference on machine learn- ing. pp. 2256-2265, PMLR, 2015.
Yang Song and Stefano Ermon. Generative modeling by estimating gradients of the data distribution. Advances in neural information processing systems, 32, 2019.
Yang Song and Stefano Ermon. Improved techniques for training score-based generative models. Advances in neural information processing systems, 33:12438-1221 12448, 2020.
Yang Song, Sahaj Garg, Jiaxin Shi, and Stefano Ermon. Sliced score matching: A scalable approach to density and score estimation. In Uncertainty in Artificial Intelligence, pp. 574-584. PMIR, 2020a.
Yang Song, Jascha Sohl-Dickstein, Diederik Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations. In Interna- lional Conference on Learning Representations, 2020b.
Yang Song, Conor Durkan, Iain Murray, and Stefano Ermon. Maximum likelihood training of score-based diffusion models. Advances in neural information processing systems, 34:1415- 1428, 2021.
Aäron Van Den Oord, Nal Kalchbrenner, and Koray Kavukcuoglu. Pixel recurrent neural networks. In International conference machine learning, pp. 1747-1756. PMLR, 2016.
Pascal Vincent. A connection between score matching and denoising autoencoders. Neural compu- tation, 23(7):1661-1674, 2011.
Runqian Wang and Yilun Du. Equilibrium matching: Generative modeling with implicit energy- based models. arXiv preprint arXiv:2510.023000 2025.
12

---

Published as a conference paper at ICLR 2026
Chen Wei, Karttikeya Mangalam, Yao Huang, Yanghao Li, Haoqi Fan, Hu Xu, Huiyu Wang, Ci- hang Xie, Alan Yuille, and Christoph Feichtenhofer. Diffusion models as masked autoencoders. In Proceedings of the IEEE/CVF International Conference Computer Vision, pp. 16284-16294, 2023.
Chenshuang Zhang, Chaoning Zhang, Sheng Zheng, Mengehun Zhang, Maryam Qamar, Sung-Ho Bae, and So Kweon. survey on audio diffusion models: Text speech synthesis and en- hancement in generative ai. arXiv preprint arXiv:2303.13336, 2023.
Bowen Zheng and Tianming Yang. Diffusion models are innate one-step generators. arXiv preprint 2XXv:2200..27700 2024.
Kaiwen Zheng, Cheng Lu, Jianfei Chen, and Jun Zhu. Improved techniques for maximum likelihood estimation for diffusion ODEs. In Andreas Krause, Emma Brunskill, Kyunghyun Cho, Barbara Engelhardt, Sivan Sabato, and Jonathan Scarlett (eds.). Proceedings of the 40th International Conference on Machine Learning, volume 202 of Proceedings of Machine Learning Research, pp. 42389, PMLR, 23-29 Jul 2023. URL ottps::/pproeeciigss..ttt .ml. press/ v202 / zheng23c.htm.
Jun
13

---

Published as a conference paper at ICLR 2026
A PROOFS
Proposition 3 (Formal restatement of Proposition D.. Let Xt be solution on [0, 2o) of the SDE
2 (x; X6 (x,, t)) dt + V2ldw,, dx;-
(34)
and assume:
1. X, admits a (random) continuous extension at t = 0: x, - Xg as O almost surely.
2. Xg Rdd x [0, oc) > TR is continuous at (xo, 0) and locally bounded in neighborhood of (xo.0)
Then, the following holds almost surely:
X0 = X2 (xo. 0)-
(35)
co. Let gt) = 2² (deterministic, C' on [0, oo) and apply the integration by-parts for semimartingales:
Proof. Fix 0
d (o(1)x)) g(()dx, + x,dg(() + d(g, x||-
(36)
Since g has finite variation, its quadratic covariation with any continuous semimartingale is zero, i.e. [g, x] = Using dg(()= -1 3dt and the given SDE,
2 t Ro (xt, dtt +
(37)
12
We used the semimartingale product rule and the facts that variation processes have quadratic variation and that [g, x]
0. Integrating from to L yields.
2 [ p Ro (x,, rdr [ -22-3/2 3/2 dw..
(38)
Multiplying by s2 and rearrange:
t²
2 [
p 80 (xr, r) dr + 12 S. /2 V
(39)
We now let t with s > 0 fixed and evaluate each term:
1 0 almost surely.
For the first term,
3 X,
For the stochastic term, by Itō isometry.
)
t1 = [ 2r -r -2
(1-
t2
t
/2r 12r-3/22 2r2-3/2dw, 3/2dw,
0.
(40)
s2
<10
so this term converges to 0 in 2 (hence in probability, along a subsequence almost surely).
For the deterministic integral term, set h(r) := *o(2r, r). By the second assumption and X, - Xo. we have l(r) + *0(xo: 0) as r O. and locally bounded near O. For any small 8 E (0,
e J. h(r)dr- 21 l. (( 21
("
h(() p3
dr.
(41)
The second integral tends to 0 since it is bounded by 2Ct² 1 3rr For the first, write ((r) = /200) + =(r) with E(r) + 0 as 1. O. Then
2t J
h(0) dr 2t2 / e((r) dr h(0) (1
12
c(() y
2x2
dr.
(42)
82
Given 1j > O, choose so small that supoerss (e(r) < my; then the last term bounded by
n (1 () ) which can be made < n7 uniformly in t. Hence the whole expression tends to h()) = xx(x, 0) as O.
14

---

Published as a conference paper at ICLR 2026
Taking limits in Eq. (39 and using x, Xo by the first assumption yields xu Xzy (xoo 0) almost surely.
(1x(1)) )==)) R., be fixed.
Proposition 4 (Formal restatement of Proposition 2. Ler Ω
by round(æ) E arg minyen - with a deterministic tie bre-breaking rule so that round is single-valued (equivalently, a single-valued selection of the metric projection onto the finite set Q2). Under the same assumptions with Proposition &o (x, 0) round (z), the limit Xo takes values in the codebook, i.e., Xo E Ω almost surely.
Define rounding map round
3 if for all & TRd
Proof. By Proposition any such solution satisfies Xg Sg (Xo, 0) almost surely. Invoking the fact & (x, 0) = round r)) the following holds:
Xg round (xo) Ω a.s.
(43)
Finally, because round maps Rd into Ω the equality round(x) x can hold only when x E Ω: Q2 then (x) l o x. Hence Xo E Ω almost surely.
B RELATION TO EQILIBRIUM MATCHING
Recently, Wang & Du 2025 have proposed the equilibrium matching model (EqM), whose loss function is defined follows:
F = E [1.o (X,) - (E - xn) (x) (x)).c(>ll'' <<>))]]
(44)
where 7 ~ (((0 and x, = 7X0 + (1 () E. By defining xo 7) = X, -17
(T7) fo(x),, this can
be transformed into the form of the signal prediction loss as follows:
fo (8,) (1- (xx - 7780) - xx)) -c(7)
LEqM = E
(45)
1 1- (Xe, - xo) 17
c(7)
E
So (X.,
(46)
c(>)
1-r fo (Xcr)
E
1-7 Xo
X,
(47)
c(7)
c(r)2
(1 (1- ||x0 - 20 (Xy,, 7)1
(48)
In addition, by defining t = (1 7//// 8 and Xx f) = *x(X, x1/(1 + t), 1/(1 t)) with a little abuse of notation, we can recover a similar form to Eq. 19):
LESM L/' c(r)²
(1 7)²² |xo
Xo (X,, 7)1 ()1, dy
(49)
1 c11 (1+
|xx0 xo (xx, t)| dt.
(50)
f2
When <(>) between EqM and our QDPM can be absorbed into the design choices of the signal predictor X, and the weighting function
17 this is exactly the same with Eq.
19
Therefore, the difference
C DERIVATION OF THEORETICAL LOWER BOUND OF NLL
First, we show that the log-likelihood of the model distribution p(x) for a finite set of data x[1),... x[N] is maximized when it corresponds the empirical distribution of x[I]....,,[(
15

---

Published as a conference paper at ICLR 2026
Proposition 5. Ler X {a1..... aam) be a finite set, and let
x x[N] E X
(51)
be observed data points. For each m. E [ M) define the empirical frequency
Tmm == # (I,..., x = am) f((m) == N
Bym
(52)
Thus p is the empirical distribution of the sample. Consider any probability mass function on X
satisfying
p(am) > 0 whenever 1ym. O.
(53)
Define the empirical average log- lillikkllioodd of p
L(p)
Σ log (x(()) N
1
(54)
Then L(p) is maximized over all such p uniquely at p= p. that is,
L(p) S L(p) for all admissible ps
(55)
with equality if and only i P.
Proof. First rewrite Lip) in terms of the empirical distribution. Using the counts Ymm
M
1 L(p) N
log (xg](x(())
Tem log (lgg((mm)
Σ
p(am) log p(am).
(56)
Now consider the Kullback-Leibhe divergence from ý to pt
(KL(fll)) = Σ p(am) log plam)
(57)
(mam)
m=1
This can be expanded
M
M Σ m=
DKL(f1p)
Σ p(am) p(am)
pamm) logplam)
(58)
m=1
Rearranging gives
M
M
Σ pam) ((am) = Σ
p(am) log((am) - (1.p..
(59)
m=11
The left-hand side is precisely L(p), so
L(p) Σ p(am) lla - DKL. (f [)
(60)
The first term En p((m) log p(am) depends only on the empirical distribution á and not on pt hence, it is a constant with respect to
On the other hand, by the non-negativity of KL divergence,
DkL(fllp) 0,
(61)
with equality if and only if p. Therefore,
L(p) (constant) DKL (p p) < (constant) 0 = L()),
(62)
and equality holds if and only if DRI. p) = 0, i.c., = p. Thus, L(p) is maximized uniquely at the empirical distribution p p.
16

---

Published as a conference paper at ICLR 2026
Since CIFAR-10 have 10,000 test images, the likelihood of the test set is maximized when the following holds:
10000 x Xxest
p (x:
(63)
otherwise,
where Xeest set of test examples with [Xtstt| 10000. Note that we assume that there no overlap inside the dataset, which is confirmed in prior work for CIFAR-10 (Barz & Denzler 2020 Therefore, the theoretical lower bound of the test NLL as follows:
NLL*
Σ
logp" (x;
(64)
X X
log 10000.
(65)
By transforming it to the scale of bits per dimension (BPD), can derive the theoretical lower bound NLL in the BPD scale as follows:
BPD
Σ
loga p" (x; 0)
(66)
[teat dim (x)
xEX
dim (x) log NLL"
(67)
3 x 32 x 32 log2 10000
(68)
~ 0.0043.
(69)
Similarly, the NLL lower bound for ImageNer-32, which has 50,000 test images, can be derived as follows:
BPD*
loga 50000
(70)
x x 32
0.0051.
(71)
D
DETAILS OF EXPERIMENTAL SETUPS CODE
D.1
Our implementation for the experiment is available at seience//zgggmm-1c1200000
4oper.
D.2
TOTAL AMOUNT OF COMPUTE
We run our experiments mainly cloud GPU instances with 8x H100. It took approximately 120 hours for our experiments in total.
D.3 LICENSE OF ASSETS
Datasets: The terms of access for the CIFAR-10 database is provided hhttps://www Cs cs.toronto. wtttt/o.dddd..ii/iffaata.aao
The terms of access for the ImageNet database The terms of access for the Flickr-Faces HQ (FFHQ) database is provided at https:://itth:
provided https: www. www.mmmaaaaaa...m net wwwp///wg//eeeooo
com/NVlabs/ txt The terms of access for the Animal-Faces-
HQ (AFHQ) database is provided at htttp:://githu com/ cl oolloovaiii/ sta v2 b blob/maateer/ATCENSS
Code: Our implementation is based on the official Jax code of Kingma et al. 2021 provided
at wihttps: /ttp://///tttt///e wwtc com/google- research/vdml We reimplement with Py Porch, and modify for our experimental settings.
For ImageNet, there is no clear evidense that there is no overlap to the best of our knowledge, the BPD lower bound can be little smaller than 0.0051.
17

---

Published as a conference paper at ICLR 2026
NLL vs FID
Likelihood loss
No
Yes
QDPM
ScoreFlow
3
Improved DDPM
DDPM
NCSN+
2
0.5
1.0
1.5
2.0
2.5
3.0
3.5
NLL
Figure 6: Trade-off between negative log- le-likkiiihod (NLL) and Frechet Inception Distance (FID).
E
THE USE OF LARGE LANGUAGE MODELS
We used large language model (ChatGPT; "GPT-5 Thinking") only for English- polishing, i... grammar, phrasing, and style improvements, and light copy-editing (e.g., reducing redundancy, harmonizing terminology). The model was not used to generate ideas, derivations, proofs, experi- mental designs, results, figures, tables, or code, nor to select or fabricate references. All technical content and claims were authored and verified by the human authors, who edited every LLM sug- gestion and take full responsibility for the final text. No non-public data, confidential information, personally identifiable information were provided to the model beyond portions of the manuscript text necessary for editing.
18


Reviews:
[{"review_id": "UnJw2cZxuX", "reviewer": "Reviewer_poSH", "summary": "- This paper introduces Quantizing Diffusion Probabilistic Models (QDPM), a diffusion model that explicitly integrates quantization into the  reverse diffusion process. \n - Unlike traditional models that treat data as continuous or rely on dequantization noise, QDPM enforces convergence to quantized points through a carefully designed signal predictor. \n - The authors prove theoretical guarantees, derive a quantization-aware maximum likelihood objective, and develop efficient SDE solvers. \n - Experiments on CIFAR-10 and ImageNet-32 show remarkable improvements in negative log-likelihood (NLL), from prior SoTA of 2.42 bpd to 0.27 bpd, nearing the theoretical lower bound.", "questions": "Please check the weaknesses and answer the questions.", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. The paper provides a clear theoretical foundation for incorporating quantization into the reverse SDE. The use of softround + decaying correction (Eq. 17) is interpretable.\n2. The analysis of convergence (Propositions 1 and 2) and the closed-form derivation of the SDE solution demonstrate theoretical grounding.\n3. Achieving a 0.27 bpd test NLL on CIFAR-10 is a good improvement and suggests genuine progress in likelihood-based training of diffusion models.\n4. The proposed QDPM framework could influence how quantization is treated in generative modeling, especially for discrete data domains (e.g., images, audio).", "weaknesses": "1. The paper emphasizes log-likelihood but gives limited discussion on sample quality (FID ≈ 5.6 / 8.9). While the authors claim NLL–FID trade-off is expected, a deeper analysis (e.g., perceptual vs likelihood trade-off curve) would strengthen claims.\n2. Although the authors cite dequantization-based diffusion models, there are no direct experimental comparisons using the same architecture but with traditional dequantization. This would isolate the quantization-aware effect.\n3. The paper reports approaching the theoretical lower bound (~0.0051 bpd), but this bound assumes uniform discrete entropy over the test set; a short derivation or clarification would help readers assess the significance.\n4. There are no ablations on components such as the softround function, or solver order. These would be important to understand where the gain originates.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "egAfJW50IB", "reviewer": "Reviewer_Pzwu", "summary": "The work introduces a quantization-aware diffusion parameterization that guarantees convergence to discrete codebook points at \n$t \\downarrow 0$. It derives a reverse-SDE solution specific to this parameterization, proposes first/second-order solvers, and trains with a likelihood-motivated objective that avoids explicit dequantization. Results show very low bits-per-dim (BPD) on CIFAR-10 and ImageNet-32.\n\nIn more detail, the paper proposes a parameterization that forces the reverse SDE to converge to quantized points by making $x_\\theta(\\cdot,0)$ equal a (soft) rounding operator; formally, the limit $x_0$ is a fixed point of $x_\\theta(\\cdot,0)$, and they limit the set of fixed points to a finite set, hence the output is quantized. The training objective is presented as an ELBO-style upper bound on $-\\log p_0(x_0)$ with $t_{\\min} \\to 0$, $t_{\\max} \\to\\infty$ (Eq. 19), then reparameterized via $u=e^{-1/t^2}$ to yield a simple Monte-Carlo form (Eq. 22). They evaluate an upper bound on NLL for the reverse SDE and report extremely low BPD on CIFAR-10/ImageNet-32 (0.27 / 0.32), claiming closeness to a “theoretical lower bound”.", "questions": "### 0. A few typos\n\nThroughout, there are expressions of the form $\\Omega = \\prod_{i=1}^d \\{ x^{(k)}\\}_{k=1}^K$ where $i$ is not used\n\n### 1. Comparability of the “likelihood” metric\n\nPrior continuous-model papers typically report *dequantized* likelihoods (e.g., uniform $[0,1)$ or learned dequantization), or use a categorical $p_\\theta(x_0 \\mid x_{t_{\\min}})$ at a positive $t_{\\min}$. Here the bound integrates to $t_{\\min} \\to 0$ with a quantized fixed-point parameterization.  It isn’t obvious this produces the same quantity other works report.  It would help to clarify whether the contribution is primarily the bound, the trained model, or both.  \n\n**(1a) Type of bound**\n\nPlease state at a high level whether your BPD corresponds to:\n\n(i) discrete likelihood (categorical over 256 bins per channel),  \n(ii) dequantized continuous likelihood, or  \n\n\n**(1b) Logp → BPD conversion**\n\nRight now the text says you “report [the] upper bound calculated with Eq. (19)” but does not specify the unit conversion or definition used to go from log-likelihood to BPD.  Assuming the logp is computed correctly, could you specify the exact computation you used to convert from your logp bound to your BPD bound?  \n\n**(1c) Evaluating other models with your bound, and evaluating your model with other bounds**\n\n- (1c.i) Would your bound, if used to evaluate another model, also report numbers <=1.0? Especially given that rounding is included in the likelihood evaluation. Regardless of how another model checkpoint was trained, does this bound (with rounding) give that model a better BPD too? Could this be evaluated on existing checkpoints from Song, Kingma, etc.?  \n\n- (1c.ii) Conversely, what would the *standard* dequantization bound (Eq. 26) report for your model, without the rounding step?  \n\n### 2. (Important!) Change-of-variables detail in Eq. (22)\n\nQuestion 1 concerned defining the BPD (assuming logp computation is correct).  This question is about the correctness of the logp computation itself. When moving from Eq. (20) to Eq. (22) with the substitution $u=e^{-1/t^2}$, the weighting induced by $du/dt$ must be accounted for; otherwise the Monte-Carlo estimator could be biased and underestimate NLL.   Could you please show the exact steps (including Jacobian factors) to justify the *unweighted* expectation over $u \\sim \\mathrm{Unif}(0,1)$ in Eq. (22)?  \n\n### 3. “Theoretical lower bound” – Q1\n\nThe lower bound $\\log_2(50000)/(3 \\cdot 32^2) \\approx 0.0051$ appears to be the code-length of a uniform categorical over the 50k images divided by dimension.  This is a property of the finite dataset, not the underlying data distribution. Please justify using this as a target for generative modeling and clarify whether your evaluation implicitly allows near-memorization (e.g., concentrating mass on quantized locations). Is it a theoretical lower bound on the 50k images?\n\n### 4. “Theoretical lower bound” – Q2\n\nWhy is the word “test” used here? For CIFAR there are 50k train images and 10k test images; for ImageNet, ~1.2M train and 50k test.  \nWas there a mix-up? The other numbers reported in Table 1 usually refer to the CIFAR *test* set of 10,000 datapoints (even though FID is often reported on the training set). Which sets were your BPD numbers computed on?  Why do we compare CIFAR test BPDs to a lower bound computed on 50k training points?\n\n### 5. Definition of $s_t$\n\nCould you explicitly define $s_t(x_t) = \\nabla_{x_t} \\log q_{0t}(x_t \\mid x_0)$ around Eqs. (2)–(4) for readability? It may be clear to experts, but all mathematical symbols should be defined.  \n\n### 6. Code availability\n\nThe anonymous code link in Appx. B.1 is currently empty.  It would be helpful to share working code so others can confirm the BPD computation protocol.  \n\n### 7. Sensitivity of reported BPD\n\nHow sensitive are the reported BPD values to the time-discretization schedule? In my experience, sampling 1,000 vs. 10,000 uniform points in (0,1) can drastically affect dequantized-diffusion BPDs for standard noise schedules and common checkpoints (e.g., those from Song, Durkan & Song, Kingma, etc.).", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- Clear mechanism to force quantized fixed points; the softround + $e^{-1/t^2}\\delta_\\theta$ parameterization is simple and elegant.  \n- Derivation of an SDE solution enables specialized solvers with straightforward implementations.  \n- Ambitious likelihood-centric evaluation; if comparable, the reported BPD would be a genuine leap.", "weaknesses": "None listed yet; see **Questions** instead. I prefer to label weaknesses only after the discussion period. Everything is only a question until then.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "HSTMA1qTKE", "reviewer": "Reviewer_Zbvq", "summary": "This paper introduces a new parameterization for diffusion models designed to improve negative log-likelihood (NLL) performance by ensuring that generated samples converge to quantized points. The proposed parameterization can be broadly applied to density estimation tasks involving quantized data. Experimental results demonstrate substantial improvements in NLL across various datasets.", "questions": "* In the proof, should $f$ be $\\hat{x}_{\\theta}$?\n\n* Could you provide more details about the role of the fixed-point requirement in Proposition 1?", "limitations": "", "rating": 6, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "* The paper is clearly written and easy to follow. The motivation and methodological design are well articulated, and the experiments convincingly demonstrate the proposed approach’s effectiveness.\n\n* The method is supported by theoretical analysis for both the parameterization and the associated solvers, though there are some issues with the description in Proposition 1 (see Weaknesses).\n\n* The experiments show strong performance on density estimation for discrete signal data, outperforming state-of-the-art methods. For example, the improvement on CIFAR-10 reduces NLL from 2.42 to 0.27.", "weaknesses": "* Proposition 1 claims that the SDE defined in Eq. (14) has a fixed point at time step 0; however, this does not appear to hold in the general case. Constructing denoised score matching for diffusion models requires defining the score field over discrete data through noise injection. As discussed in [1], when the model distribution approaches the real data distribution, the gradient becomes steeper due to the small Gaussian variance, making the boundedness assumption not always valid. While the proposed parameterization may alleviate this issue, a more detailed explanation covering broader cases would strengthen the argument. Moreover, the role of the fixed-point requirement in Proposition 1 is unclear. \n\n* Beyond the explanation in Proposition 1, an ablation study validating this proposition, for instance, by reporting the magnitude of the fixed-point error, would significantly improve the paper’s empirical evidence.\n​\n\n[1] Song, Yang, et al. \"Sliced Score Matching: A Scalable Approach to Density and Score Estimation.\" Uncertainty in Artificial Intelligence. PMLR, 2020.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "5bJ1IZ75S0", "reviewer": "Reviewer_12ST", "summary": "This work introduces a new parameterization technique for diffusion models aimed at improving negative log-likelihood performance by ensuring generated outputs lie on quantized values. The approach is broadly applicable to density modeling on discretized data and experimental results show significant NLL gains.", "questions": "NA", "limitations": "", "rating": 6, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The paper is clearly written with well-motivated methodology. The rationale behind the approach and its implementation is easy to understand, and the experiments effectively highlight its advantages.\n\n2. The method is supported by theoretical analysis regarding both the parameterization and associated solvers, though there may be some concerns with the proposition as noted below.\n\n3. Strong experimental results are demonstrated on discrete-signal density estimation tasks, outperforming prior state-of-the-art methods. Notably, CIFAR-10 results show substantial improvement.", "weaknesses": "A recent work [1] proposes an equilibrium formulation for image generation rather than traditional density estimation. Although their approach does not strictly ensure convergence to quantized points, it explicitly defines a fixed point for the generative process. It would be valuable to discuss how this relates to your method and whether their training strategy could potentially benefit or be adapted to your framework. In addition, that work argues that standard diffusion models do not exhibit a valid fixed point at t=0. This observation seems relevant to Proposition 1 in your paper and merits further discussion or clarification.\n\n2. In the proof, the function labeled $f$ should be $\\hat{x}_\\theta$\n\n[1] Wang, Runqian, and Yilun Du. \"Equilibrium Matching: Generative Modeling with Implicit Energy-Based Models.\" arXiv preprint arXiv:2510.02300 (2025).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)