Progressive Compression with Universally Quantized Diffusion Models
Authors: Yibo Yang
Justus Will
Stephan Mandt
Venue: ICLR 2025 Oral (2025)
Keywords: diffusion
generative modeling
compression
universal quantization
Primary area: generative models
TL;DR: We improve practical compression with an unconditional diffusion model, proposing a new form of diffusion based on uniform noise instead of Gaussian noise.

Abstract: Diffusion probabilistic models have achieved mainstream success in many generative modeling tasks, from image generation to inverse problem solving. A distinct feature of these models is that they correspond to deep hierarchical latent variable models optimizing a variational evidence lower bound (ELBO) on the data likelihood. Drawing on a basic connection between likelihood modeling and compression, we explore the potential of diffusion models for progressive coding, resulting in a sequence of bits that can be incrementally transmitted and decoded with progressively improving reconstruction quality. Unlike prior work based on Gaussian diffusion or conditional diffusion models, we propose a new form of diffusion model with uniform noise in the forward process, whose negative ELBO corresponds to the end-to-end compression cost using universal quantization. We obtain promising first results on image compression, achieving competitive rate-distortion-realism results on a wide range of bit-rates with a single model, bringing neural codecs a step closer to practical deployment. Our code can be found at https://github.com/mandt-lab/uqdm.

Published as a conference paper at ICLR 2025
PROGRESSIVE COMPRESSION WITH UNIVERSALLY QUANTIZED DIFFUSION MODELS
Yibo Yang" Justus C. Will"
Stephan Mandt
Department of Computer Science University of California, Irvine {yibo. yanb, yang, jewill, mandt focitttuucieeem
ABSTRACT
Diffusion probabilistic models have achieved mainstream success in many gen- erative modeling tasks, from image generation to inverse problem solving. A distinct feature these models is that they correspond to deep hierarchical latent variable models optimizing a variational evidence lower bound (ELBO) on the data likelihood. Drawing on a basic connection between likelihood modeling and compression, explore the potential of diffusion models for progressive coding, resulting in a sequence of bits that can be incrementally transmitted and decoded with progressively improving reconstruction quality. Unlike prior work based Gaussian diffusion or conditional diffusion models, we propose new form of diffusion model with uniform noise in the forward process, whose negative ELBO corresponds to the end-to-end compression cost using universal quantization. We obtain promising first results on image compression, achieving competitive rate- distortion and rate-realism results on wide range of bit-rates with single model, bringing neural codecs a step closer to practical deployment. Our code can be found at https: / h/g/ttttubbb www.oooo/eeeeoooooooooc lab /uqdm.
INTRODUCTION
A diffusion probabilistic model can be equivalently viewed as deep latent-variable model (Sohl- Dickstein et al., 2015; Ho et al., 2020; Kingma et al., 2021), a cascade of denoising autoencoders that perform score matching different noise levels (Vincent, 2011: I; Song & Ermon, 2019), or neural SDE (Song et al., 2021b). Here we take the latent-variable model view and explore the potential of diffusion models for communicating information. Given the strong performance of these models likelihood estimation (Kingma et al., 2021; Nichol & Dhariwal, 2021), it is natural to ask whether they also excel in the closely related task of data compression (MacKay, 2003; Yang et al., 2023).
Ho et al. (2020); Theis et al. (2022) first suggested a progressive compression method based on an unconditional diffusion model and demonstrated its strong potential for data compression. Such progressive codec is desirable as it allows us to decode data reconstructions from partial bit-streams, starting from lossy reconstructions at low bit-rates perfect (lossless) reconstructions high bit- rates, all with single model. The ability to decode intermediate reconstructions without having to wait for all bits be received is a highly useful feature present in many traditional codecs, such as JPEG. The use of difffusion models has the additional advantage that can, in theory, obtain perfectly realistic reconstructions (Theis et al., 2022), even at ultra-low bit-rates. Unfortunately, the proposed method requires the communication of Gaussian samples across many steps, which remains intractable because the exponential runtime complexity of channel simulation (Goc & Flamich, 2024).
In this work, take first steps towards diffusion-basse progressive codee that is computationally tractable. The key idea is to replace Gaussian distributions in the forward process with suitable uniform distributions and adjust the reverse process distributions accordingly. These modifications allow the application of universal quantization (Zamir & Feder, 1992) for simulating uniform noise channels, avoiding the intractability of Gaussian channel simulation in (Theis et al., 2022).
Specifically,, our contributions are as follows:
"qqual contribution

---

Published as conference paper at ICLR 2025
original
JPG
BPG
CDC (p=0.9)
CDC
CTC
UQDM (ancestral)
UQDM (flow-based)
UQDM (denoise)
(p=0)
bpp=1 1app=1.81. PSNR-28.90 9.97. S5IM-0.91
bpp=1 85, P6 SSIM-0.9e 98
b PSNR-27 24,
Upp=3 58,
bpp= 880
bpp=1100 PSMA-30 06.
b 95,
bpp= 1ipp=1,77
PS 80.
PSNR- 32-37, SSIM-0.95
PS 32.61. SSIM-0.96
PSNR- 32 89. 5SIM-0.96
SSIM=O.aa 0.93
S5IM-0.5 0.96
SSIM-
bpp-2.21
PSNR=32
PSNR=
PSNR=25
PSNR=27
PSNR-
PSNR
PSMR-32
PSMR-
SSIM
SSIM=0...
SSIM=0 92
SSIM 94
SSiMm0
5SIM=0 92
SSIM=O
SSIM
bpp-2.15,
bpp-2.14. PSNR=29 96,
bop=4.11 4.16.
boo 18,
bpp-2 15.
bpp-2.11, PSNR=25 SSIM-0.84
boo-2 49,
bpp-2.00 2.08.
PSNR=25 69
PS 43.
PSNR=25
PSNA=26 63.
PSNR=228:1 SSIM-0.97
PSAR=20 22.
5SIM-0.94
55IM-0.79
SSIM-0.88
Figure 1: Example reconstructions from several traditional and neural codecs, chosen at roughly similar bitrates. At high bitrates, our UQDM method preserves details (e.g. shape and color pattern of the spider, sharpness of the calligraphy) better than other neural codecs. Note that among the methods considered here, only ours and CTC (Jeon et al., 2023) implement progressive coding,
1. We introduce a new form of diffusion model, Universally Quantized Diffusion Model (UQDM), that is suitable for end-to-end learned progressive data compression. Unlike the closely-related Gaussian diffusion model (Kingma et al., 2021), compression with UQDM is performed efficiently with universal quantization, avoiding the generally exponential runtime of relative entropy coding (Agustsson & Theis, 2020; Goe & Flamich, 2024).
2. We investigate design choices of UQDM, specifying its forward and reverse processes largely by matching the moments of those Gaussian diffiusion, and obtain the best results when we learn the reverse-process variance as inspired by Nichol & Dhariwal (2021).
3. We provide theoretical insight into UQDM in relation VDM, and derive the continuous- time limit of its forward process approaching that of the Gaussian diffusion. These results may inspire future research in improving the modeling formalism and training efficiency.
4. We apply UQDM image compression, and obtain competitive rate-distortion and rate- realism results which exceed existing progressive codecs at wide range of bit-rates (up to lossless compression), all with single model. Our results demonstrate, for the first time, the high potential of an unconditional diffusion model as practical progressive codec.
BACKGROUND
Diffusion models Diffusion probabilistic models learn to model data by inverting Gaussian noising process. Following the discrete-time setup of VDM (Kingma et al., 2021), the forward noising process begins with data observation x and defines sequence of increasingly noisy latent variables z, with conditional Gaussian distribution,
q(z, xx ((axx,
t t I...,,
Here at and of are positive scalar- valued functions of time, with a strictly monotonically increasing signal-to-noise-aute SNR(t) = a² / The variance-pred pressevvng process of DDPM (Ho al., 2020) corresponds to the choice of = 1 oi. The reverse-time generative model is defined by collection of conditional distributions (/zz-1/z,). a prior p(zr) = (O, (O,I), I), and likelihood model Xx (z, t)) are chosen to have
p(x[zo). The conditional distributions p(z(-1/z)) the same distributional form as the "forward posterior" distribution (2,,-1 (2,,)). with x estimated from its noisy version 2, through the learned denoising model &p. Further details the forward and
2

---

Published as a conference paper at ICLR 2025
backward processes can be found in Appendix A and B. Throughout the paper the logarithms use base 2. The model is trained by minimizing the negative ELBO (Evidence Lower BOund),
[(x) ([((gar))))lp(aaa + +E| log p( [
x
[KT 11,, x) 11 pp(z [2:)]] (1)
2/40
where the expectations are taken with respect to the forward process g(oo:T x). Kingma al. (2021) showed that a larger T' corresponds to a tighter bound on the marginal likelihood log p(x), and as
T - oc the loss approaches the loss of a class of continuous-ting diffusion models that includes the ones considered by Song et al. (2021b).
Relative Entropy Coding (REC) Relative Entropy Coding (REC) deals with the problem of efficiently communicating single sample from a target distribution q using a coding distribution p. Suppose two parties in communication have access to a common "prior" distribution p and pseudo-random number generators with common seed; a Relative Entropy Coding (REC) method (Flamich et al., 2020) allows the sender to transmit a sample z ~ q using close to (L(g ll p) bits on average. If q arises from a conditional distribution, e-g., 9x q( x) is the inference distribution of a VAE (which can be viewed as a noisy channel), a reverse channel coding or channel simulation (Theis & Ahmed, 2022) algorithm then allows the sender to transmit
9x with x p(x) using close to [xxp(x) (g x) p(z))] bits on average. At a high level, a typical REC method works as follows. The sender generates a (possibly large) number of candidate samples from the prior p.
Zp ~ P, Y1 1, 1,2,3,...
and appropriately chooses index K such that x is fair sample from the target distribution, i.e., ZK q. The chosen index K E N is then converted to binary and transmitted to the receiver. The receiver recovers zk by drawing the same sequence of z candidates from p (made possible by using a pseudo-random number generator with the same seed as the sender) and stopping at the K'th one.
A major challenge of REC algorithms is that their computational complexity generally scales ex- ponentially with the amount of information being communicated (Agustsson & Theis, 2020; Goc & Flamich, 2024). As an example. the MRC algorithm (Cuff, 2008; Havasi et al., 2018) draws M candidate samples and selects K E (1, ...... M) with a probability proportional to the importance weights, v(x.))/((..), n 1, ...,, M; similarly to importance sampling. M needs to be on the order of (Tl(p)) for zg to be (approximately) a fair sample from q, thus requiring a number of drawn samples that scales exponentially with the relative entropy (L(gllp) (the cost of transmitting K is thus log M A (gllp)) bits). The exponential complexity prevents, e.g., naively communicating the entire latent tensor Z in Gaussian VAE for lossy compression, the relative entropy KL(g(z) x) p(z)) casily exceeds thousands of bits, even for a small image. This difficulty can be partly remedied by performing REC on sub-problems with lower dimensions (Flamich al., 2020; 2022) for which computationally viable REC algorithms exist (Flamich et al., 2024; Flamich, 2024), but at the expense of worse bitrate efficiency due the accumulation of codelength overhead across the dimensions.
Progressive Coding with Diffusion A progressive compression algorithm allows for lossy recon- structions with improving quality as more bits are sent, up till a lossless reconstruction. This results in variable-rate compression with a single bitstream, and is highly desirable in practical applications.
As we will explain, the NELBO of a diffusion model (eq. (1)) naturally corresponds to the lossless coding cost of progressive codec, which can be optimized end-to-end on the data distribution of interest. Given a trained diffusion model, a REC algorithm, and a data point x, we can perform progressive compression follows (Ho et al., 2020; Theis et al., 2022): Initially, at time T. the sender transmits a sample of q(zr|x) (x) under the prior p(zT), using Lr bits on average. At each subsequent time step t. the sender transmits sample of q z [z, x) given the previously transmitted 2,, under the (conditional) prior p(z,-1/za IZ,), using approximately L,-1 bits. Finally, given 2, at t = 0, x can be transmitted losslessly under the model p(x (2)) by entropy coding algorithm (e.g., arithmetic coding), with a codelength close to Lx bits (Polyanskiy & Wu, 2022, Chapter 13.1). Thus, the overall cost of losslessly compressing x sums up to [(x) bits, as in the NELBO in eq. (1). Crucially, at z; to obtain lossy data reconstruction &
any time t, the receiver can use the moot-reeently-receenn For this, several options are possible: Ho et al. (2020) consider using the diffusion model's denoising prediction & (z(2,) while Theis et al. (2022) consider sampling X, p(x(2.). either by ancestral
3

---

Published as a conference paper at ICLR 2025
sampling probability flow ODE (Song et al., 2021b). Note that if the reverse generative model captures the data distribution perfectly, then X2 ~ p(x|z,) follows the same marginal distribution as the data and has the desirable property of perfect realism, ie., being indistinguishable from real data (Theis et al., 2022).
REC algorithms suffer from exponential run- time (Agustsson & Theis, 2020; Goc & Flamich, 2024), efficient REC algorithms exist if willing to restrict the kinds of target and coding distributions allowed (Flamich et al., 2022; 2024). Here, we focus on the special case where the target distribution q is given by uniform noise channel, which is solved efficiently by Universal Quantization (UQ) (Roberts, 1962; Zamir& & Feder, 1992; Agustsson & Theis, 2020). Specifically, suppose we (the sender) have access to a scalar r.v. ~ py, and would like to communicate noise ceise-perturbed version of it,
Universal Quantization Although
Y ==Y- Y-Y+
where U (-(-2/2, A/2) is an independent r.v. with a uniform distribution on the interval [-4/2, A/2]. UQ accomplishes this as follows: Step 1. Perturb Y by adding another independent noise U ~ 4(-4/2, A/2), and quantize the result to the closet quantization point K on a uniform grid of width A, i.e., computing K == A | where [-] denotes rounding to the nearest integer. Step Entropy code and transmit K under the conditional distribution of K given U". Step 3. The receiver draws the same U' by using the same random number generator and obtains a reconstruction Y = K U' A|=+ U'. Zamir & Feder (1992) showed that Y indeed has the same distribution as Y, and the entropy coding cost of K related to the differential entropy of Y via
H[K]U") T( A(²) -log(A).
In the above, the optimal entropy coding distribution IP(KIU" u') is obtained by discretizing Py := py (1(--/2 A/2) on grid of width and offset by U" u' (Zamir & Feder, 1992), where denotes convolution. If the true py is unknown, we can replace it with surrogate density model fū) during entropy coding and incur a higher coding cost,
Ey (KT(((-WV) (v) [[)( 2 I(Y;)),
where u(-ly) denotes the density function of the uniform noise channel gr1r= 1/y can be shown that the optimal choice of fe is the convolution of py with ((- 4/2). Therefore, as in prior work (Agustsson & Theis, 2020; Ballé et al., 2018), we will choose fo to have the form of another underlying density model go convolved with uniform noise, i.e.
fo(-) = g-(() * u(; A/2).
(3)
3 UNIVERSALLY QUANTIZED DIFFUSION MODELS
We follow the same conceptual framework of progressive compression with diffusion models as in (Ho et al., 2020; Theis et al., 2022), reviewed in the previous section. While Theis et al. (2022) use Gaussian diffusion, relying the communication of Gaussian samples which remains intractable in higher dimensions, we want to apply UQ to similarly achieve a compression cost given by the NELBO, while remaining computationally efficient. We therefore introduce a new model with modified forward process and reverse process, which we term universally quantized diffusion model (UQDM), substituting Gaussian noise channels for uniform noise channels.
3.1 FORWARD PROCESS
The forward process of a standard diffusion model is often given by the transition kernel q(zr+1 (z,) (Ho et al., 2020) or perturbation kernel q(z, x) (Kingma et al., 2021), which in turn determines the conditional (reverse-tima)
NELBO in eq. (1). As distributions [zr[x) and {q(zt-1/ze x)]t 1.... T} appearing in the interested in operationalizing and optimizing the coding cost associated
with eq. (1), we will directly specify these conditional distributions to be compatible with UQ rather than deriving them from transition/perttuaatioo kernel. We thus specify the forward process with
4

---

Published as a conference paper at ICLR 2025
the same factorization in DDIM (Song al., 2021a) via ((zo:](x) q(zz (X) IT-= Q(2(-1/2(,,)) and consider a discrete- time non-Markovian process as follows,
[((zr x) = ((rxx, I) g(zz-1 12. == u ((t))z c(t)x -(0) b(t)zz c(t)x A10) .t=
+ 1, 1,,,, 2,.,,
(4)
where b(t), c(t), and (t) are scalar-valued functions of time. Note that unlike in Gaussian diffusion, our ((z Zt. x) is chosen to be a uniform distribution so that it can be efficiently simulated with UQ
(as result, our (Zz xx) for any * T does not admit simple distributional form). There is freedom in these choices of the forward process, but for simplicity we base them closely on the Gaussian case:
we choose standard isotropic Gaussian g(zr|x), and b((), c(t), A()) so that g(ze-1 (ee, x) has the same mean and variance as in the Gaussian case (see Appendix A for more details):
c(t) = -o b(t) = an T:-1 a-1 of
af
0tt-1 A(t) = V120 of
0t-1 with -ir-1 = o² ai of
de
We note here that g(z,) kr, x) can be written as a sum of uniform distributions, which as we increase T + oc, converges in distribution to Gaussian by the Central Limit Theorem. This implies that {(z,|x) x) also converges to a Gaussian for every 4, and that our forward process has the same underlying continuous-time limit as in VDM (Kingma al., 2021). We give the precise statement and proof in Appendix A.3.
As in VDM (Kingma et al., 2021), the forward process schedules (i.e., Ot and oz, as well as b(t), (t)) can be learned end-to-end, e.g., by parameterizing of = sigmoid( where is monotonic neural network. We did not find this to yield significant improvements compared to using& a linear noise schedule similar to the one in Kingma et al. (2021).
3.2 BACKWARD PROCESS
Analogously the Gaussian case, we want to define conditional distribution p(z,-1 (z:) that leverages denoising model & {o(,,,)) f) and closely matches the forward "posterior" g(z [z,, x). In our case, the forward "posterior" corresponds to a uniform noise channel with width A (t), i.e..
2t-1 b(()z, + c(()x + (()),,,
(-12/2 1/2); to simulate it with UQ we choose density model for Zt-1 with the same form as the convolution in eq. (3). Specifically, we let
p(2--1 = (0 (2 1: zz, 1) x[(-(00/// A(0/2),
(5)
where ge t) is learned density chosen match q | x). Recall in Gaussian diffusion (Kingma et al., 2021), p(z--1/z;) is chosen to be Gaussian of the form q | x X {z, t)). i.e. the same as q(z [2, x) but with the original data x replaced by denoised prediction x (o t). For simplicity, we base closely on the choice of p(zz-1/z)) in Gaussian diffusion, e.g.,
Ze,
go(2+11 Zz, 1) (b()))z +
(())
or logistic distribution with the same mean and variance.
(0(2;-1; z,, t) Logistic (b(t)z, + (())x
where og (t) is the variance of the Gaussian forward "posterior", and we use the same noise- preeippreintion network for X, as in (Kingma al., 2021). We found the Gaussian and logistic distributions to give similar results, but the logistic to be numerically more stable and therefore adopt it in all our experiments.
Inspired by (Nichol & Dhariwal, 2021), we found that learning per-coordinate variance in the reverse process to significantly improve the log-likelihonn which we demonstrate in See. In practice, this is implemented by doubling the output dimension of the score network to also compute tensor of scaling factors Se [2.), so that the variance of ge is o = (t) t) @ Sø (z.).. Refer to Appendix B.2 for more detailed analysis of the log-likelihood and how a learned variance is beneficial.
We note that other possibilities for go exist besides Gaussian or logistic, e.g., mixture distributions (Cheng et al., 2020), which trade off higher computation cost for increased modeling power. Analyz- ing the time reversal of the our forward process, similarly to (Song et al., 2021a), may also suggest better choices of the reverse-time density model go. We leave these explorations future work.
5

---

(Ballé et al.), and Published as a conference paper at ICLR 2025
We adopt the same form of categorical likelihood model p(x(zo) as in VDM (Kingma et al., 2021), as well as the use of Fourier features.
Algorithm 1 Encoding
Algorithm 2 Decoding
2T p(zr) for t = ....... 2, do
2T p(žr)
b Using shared seed
for = T..... 2, 1.1dd do
Let A. (t), uo - b(1)z) + c(()x.
Let A = (t)).
Compute the parameters of p(z (z,)-
Compute the parameters of p(zı-1/zz)
> Send 1 ~ q(zc-1/Ze. x) with UQ:
u, ((1/22 1/2). & Using shared seed Derive entropy model p(k|z,, u.) by dis- cretizing p( 1
u, ~ ((11/2, 1/2). k. = A, Me + u.l. A:
Entropy-decode k, under p(k|z,, uz). Lossy
Derive entropy model p(k|z,, u,) by dis- cretizing p (z,).
Z:- = k, u.
& = (2, 1). reconstruction
Entropy-encode k, under p(k|z,,u) uz).
Z k, A,u,
end for Entropy-decode x with p(x|zo). D Lossless
end for Entropy-encode x with p(x(zo).
3.3 PROGRESSIVE CODING
Given UQDM trained on the NELBO in eq. (1), we can use for progressive compression similarly to (Ho et al., 2020; Theis et al., 2022), outlined in Section 2.
The initial step T involves transmitting Gaussian Zy. Since we do not assume access to an efficient REC scheme for the Gaussian channel, we will instead draw the same 2T ~ p(ZT) No, I) on both the encoder and decoder side, with the help of shared pseudo-random seed. To avoid train/compression mismatch, we therefore always ensure ((zl(x) s p(zr) and hence Lr 0. At any subsequent step t, instead of sampling 2 b(t)z; + c(t)x + (t))u as in training, we apply UQ to communicate the "forward posterior" mean vector 14Q b(t)z; + c(t)x. Specifically, given Ze the sender computes μQ and the parameters of p(z ( (by evaluating the score network), draws a pscudo-random noise u; ~ ((22/2, 1/2). quantizes #4 to k; = A.L + u,] where A = (l), derives an entropy model p(k[z, u,) (by discretizing p(z,-1 (2)) on grid of width A. and offset by u,), and entropy-encodes k, under p(k|z,, u.). The receiver draws the same pseudo- random u, ~ L(-1/2,, 1/2), entropy-decode
k; under the same entropy model p(k|z;, uz), and computes 2:-1 k; A, and (optionally) a lossy reconstruction x; from Z;-1- Finally, after having transmitted when t = 1, x is losslessly compressed using the entropy model p(x (20). Pseudocode can be found in Algorithms and Note that we can replace the denoised prediction X, = (2 1) with more sophisticated ways to obtain lossy reconstructions such as flow- based reconstruction or ancestral sampling (Theis et al., 2022). As our method is progressive, the algorithm can be stopped at any time and the most recent lossy reconstruction be used as the output. Compared to compression with VDM (Theis et al., 2022). the main difference is that we transmit 2;-1 g(z |2, x) under p(z (2)) using UQ instead of Gaussian channel simulation; the overall computation complexity is now dominated by the evaluation of the denoising network &o (for computing the parameters of p(2,-1 (2.)). which scales linearly with the number of time steps.
We implemented the progressive codec using sensorflow-compzeee
found the actual file size to be within 3% of the theoretical NELBO.
RELATED WORK
Diffusion models (Sohl-Dickstein al., 2015) have achieved impressive results on image generation (Ho et al., 2020; Song et al., 2021a) and density estimation (Kingma et al., 2021; Nichol & Dhariwal, 2021). Our work is closely based on the latent-variable formalism of diffusion models (Ho et al.,
This corresponds to a trivial RBC problem where a sample from g p can he transmitted using K KT((gll))
0 bits.

---

Published as a conference paper at ICLR 2025
2020; Kingma et al., 2021), with our forward and backward processes adapted from the Gaussian case. Our forward process is Mar--aaanv like DDIM (Song et al., 2021a), and our reverse process uses learned variance, inspired by (Nichol & Dhariwal, 2021). Recent research has focused on efficient sampling (Song et al., 2021a; Pandey et al., 2023) and better scalability via latent diffusion (Rombach al., 2022), consisteney models (Song al., 2023), and distillation (Sauer et al., 2024). whereas focus on the compression task. Related to our approach, cold diffusion (Bansal et al., 2024) showed that alternative forward processes other than the Gaussian still produce good image generation results.
Several diffusion-based neural compression methods exist, but they use conditional diffusion models (Yang & Mandt, 2023: Careil et al., 2023; Hoogeboom et al., 2023) which do not permit progressive decoding. Furthermore, they are also less flexible as a separate model has to be trained for each bitrate. Progressive neural compression has so far been mostly achieved by combining non-linear transform coding (for example using VAE) with progressive quantization schemes. Such methods include PLONQ (Lu et al., 2021), which uses nested quantization, DPICT (Lee et al., 2022) and its extension CTC (Jeon et al., 2023), which use trit-plane coding, and DeepHQ (Lee et al., 2024) which uses a learned quantization scheme. Finally, codecs based on hierarchical VAEs (Townsend et al., 2024; Duan al., 2023) closely related but do not directly target the realism criterion.
EXPERIMENTS
We train UQDM end-to-end by directly optimizing the NELBO loss eq. (1), summing up L, across all time steps. This involves simulating the entire forward process {Zo..... Zy' } according to eg. (4) and can be computationally expensive when T is large but can be avoided by using a Monte-Carlo estimate based on single L, as in the diffusion literature (Ho et al., 2020). We found small T (< 10) to give the best compression performance, and therefore leave the investigation of training with single-step Monte- Carlo objective to future work. Note that this would require sampling from the marginal distribution ((z, x), which becomes approximately Gaussian for large (see Sec. 3.1).
When considering the progressive compression performance of VDM and UQDM, we consider three ways of computing progressive reconstructions from Zy denoi se, where = (z(2;1)) is the prediction from the denoising network; ancestral. where x p(x (z) is drawn by ancestral sampling, and flow-based where X ~ p(x[2,) is computed deterministically using the probability flow ODE in (Theis et al., 2022). In VDM, the probability flow ODE produces the same trajectory of marginal distributions as ancestral sampling, but gives improved lossy compression performance (Theis al., 2022). In the case of UQDM, we apply the same update equations and observe similar benefits, likely due to the continuous-time equivalence of the underlying processes of UQDM and VDM. See Appendix B.3 for details. Note that D and DiffC-F (Theis et al., 2022) directly correspond to our VDM results with ancestral and Flow-based reconstructions.
In all experiments involving VDM and UQDM, we always use the same denoising U-net architecture for both, except UQDM uses twice as many output dimensions to additionally predict the reverse- process variance (see Sec. 3). We refer to Appendix Sec. C for further experiment details.
5.1 SWIRL TOY DATA
We obtain initial insights into the behavior of our proposed UQDM by experimenting on toy swirl data (see Appendix C.1 for details) and comparing with the hypothetical performance of VDM (Kingma al., 2021).
First, we train UQDM end-to-end for various values of T E (3, 4, 5, 10, 15, 20, 30), with and without learning the reverse process variance. For comparison, we also train a single VDM with T 1000, but compute the progressive-coding NELBO eq. (1) using different T.. Fig. 2 plots the resulting NELBO values, corresponding to the bits-per-dimension cost of lossless compression. We observe that for UQDM, learning the reverse-process variance significantly improves the NELBO across all T. and a higher is not necessarily better. In fact, there seems to be an optimal T ~ 5, for which we obtain bpd of around 8. The theoretical performance of VDM, by comparison, monotonically improves with T (green curve) until it converges to a bpd of 5.8 at T = 1000, as consistent with theory (Kingma et al., 2021). We also tried initializing UQDM without learned evverseprocess variances to use the pre-trained VDM weights; interestingly, this resulted in very similar performance to the end-to-end trained result (blue curve), and further finetuning gave little improvement.
7

---

Published as conference paper at ICLR 2025
UQDM
0.47
VOM (T-120)
0.06
UQDMI T-19
UQDMTT55
NELBD (ape)
6881
0.05
UOBMIT-20
UODM T-30
15
(T-00)) UQDM
10
UODM T-20
0.01
UQUM T-30
0.00
5
15
20
25
30
(bpd)
Figure 2: Results on swirl data. The VDM curves correspond to the hypothetical performance of
REC that remains computationally intractable. Left: Lossless compression rates v.s. the choice of T, for UQDM with/without learned reverse-process variance (blue/orange) and VDM (green). For UQDM. learning the reverse-process variance significantly improved the NELBO, and an optimal T x 5. Middle, Right: Progressive lossy compression performance for VDM and UQDM, measured in fidelity (PSNR) v.s. bit-rate (middle), or realism (sliced Wasserstein distance) v.s. bit-rate (right).
JPRG
70
JPEG 2000
DPG
102
VOM T-1000, denose
T- ancestral
T-20,
LOON TwA,
PSWA (68)
60
JPEG
20 20:
LODM T-4, donoic
DOOW 1-4, Now- besed
[P2G 2000
BPG
VOM T-1000, derese
VOM T- ancestral
VOM ancestral
T-4
UODM T-4, denoise
30
LigDM Twdd Row hase-
5.0
7.5
Data Oppp)
Rata (opp)
Figure 3: Progressive lossy compression performance of UQDM the CIFARIO dataset, comparing fidelity (PSNR) and realism (FID) with bit-rate per pixel (bpp). using either ancestral sampling or denoised prediction obtain progressive reconstructions indicated. The VDM curve corresponds to hypothetical performance REC that computationally intractable. We achieve better fidelity and realism than JPEG and JPEG2000 across all bit-rates and than BPG in the high bit-rate regime.
This suggests that pretrained VDM can already be used for progressive compression with UQ via our moment-matching scheme (see Section 3), although the compression performance will be much worse compared end-to-end trained UQDM with learned preess-processs variances.
We then examine the lossy compression performance of progressive coding. Here, we train UQDM end-to-end with learned reverse-process variances, and perform progressive reconstruction by ances- tral sampling. Figure 2 plots the results in fidelity v.s. bit-rate and realism v.s. bit-rate. For reference, we also show the theoretical performance of VDM using T 100 discretization steps, assuming hypothetical REC algorithm that operates with no overhead. The results are consistent with those on lossless compression, with similar performance ranking for among UQDM, and gap remains to the hypothetical performance of VDM.
Finally, we examine the quality of unconditional samples from UQDM with varying T. Although our earlier results indicate worse compression performance for T > 5, Figure 7 shows that UQDM's sample quality monotonically improves with increasing T.
5.2 CIFAR10
Next, we apply our method to natural images. We start with the CIFARIO dataset containing 32 x 32 images. We train baseline VDM model with smaller architecture than that used by Kingma et al. (2021), converging to around bits per dimension. We use the noise schedule of = o(x) where % linear in / with learned endpoints "yr and yo. For UQDM model empirically find that T x
8

---

Published as a conference paper at ICLR 2025
60
JPEG
-+-- JPEG 2000
BPG
50
(2+0.3)
100000,
VOM T-20, ancestral
PSWR 008)
VER1 1-20, dericive
BPS
VOM T-20, Tow based
CTC
LggoM Tmd, anrmetral
CDC (p-0.31
UOON denoise
CDC (a-0)
VOM T 1200 deroise
VOM 20
VOM 1m based
UQOM T-4,
ancostral
UKDDM T-m4
denpise
UCDM Twd, nlo based
0.0 5.0
7.5
10.0
12.5
15.0
17.5
0.0 2.5
5.0
7.5
10.0
12.5
15.0
17.5
Rate Copp1
Rate Copp)
Figure 4: Progressive lossy compression performance of UQDM on the Imagenet64 dataset, compar- ing fidelity (PSNR) and realism (FID) with bit-rate per pixel (bpp), using either ancestral sampling or the denoised prediction to obtain progressive reconstructions corresponds to hypothetical performance of REC that remains computationally intractable. While the reconstruction quality of other codees like CDC or BPG plateaus higher bit-rates, our method continues to gradually improve fidelity and realism even at higher bit-rates where it achieves the best results of any baseline. We beat compression performance JPEG, JPEG2000, and CTC across all bit-rates. Note that only UQDM, CTC, and JPEG2000 implement progressive coding.
indicated. The VDM curve
yields the best trade-off between bit-rate and reconstruction quality. We train our model end-to-end on the progressive coding NELBO eq. (1) with learned evvrss-procees variances.
We compare against the mnvelet-basedd codecs JPEG, JPEG2000, and BPG (Bellard, 2018). For JPEG fixed set of quality levels and encode the images independently, for JPEG2000 instead use its progressive compression mode that allows us to set the approximate size reduction in each quality layer and obtain rafe-distortion curve from one bit-stream.
and BPG
As shown in Figure 3, we consistently outperform both JPEG and JPEG2000 over all bit-rates and metrics. Even though BPG, competitive non-progressive codec optimized for rate-distortion performance, achieves better reconstruction fidelity (as measured in PSNR) in the low bit-rate regime, our method closely matches BPG in realism (as measured in FID) and even beats BPG in PSNR at higher bit-rates. The theoretical performance of compression with Gaussian diffusion (c.g.. VDM) (Theis al., 2022), especially with a high number of steps such T 1000, currently computationally infeasible, both due to the large number of neural function evaluations required, and due the intractable runtime of REC algorithms theoretical results both for T 1000 and 7 20, where the latter uses a smaller and more practical number diffussion/progressive reconstruction steps.
the Gaussian case. Still, for reference we report
5.3 IMAGENET 64 x 64
Finally, we present results on the ImageNet 64 64 dataset. We train baseline VDM model with the same architecture as in (Kingma et al., 2021), reproducing their reported BPD of around 3.4; train UQDM of the same architecture with learned reverse-process variances and T 4. In addition to the baselines described in the previous section, we also compare with CTC (Jeon et al., 2023), recent progressive neural codee, and CDC (Yang & Mandt, 2023), non non-progressive neural codec based a conditional diffusion model that can trade-off between distortion and realism via a hyperparameter p. We separately report results for both p O, which purely optimizes the conditional diffusion objective, and p 0.9, which prioritizes more realistic reconstructions that also jointly minimizes a perceptual loss. For CTC we use pre-trained model checkpoints from the official implementation (Jeon et al., 2023); for CDC we fix the architecture but train model for each bit-rate v.s. reconstruction quality/realism trade-off.
The results shown in Figure 4. When obtaining progressive reconstructions from denoised predictions, UQDM again outperforms both JPEG and JPEG2000. Our results are comparable to, not slightly better than, CTC, and even though the reconstruction quality of other codecs plateaus

---

Published as conference paper at ICLR 2025
bpp-0.00 psnr=133.45
bpp=0.45, p0.r=20.92
bpp-2.18 pahn-
bpp-15.55
bpp-
850-0.45,
bpg-2.16
bpp-
bpp-15 10.615.63
ponn=
ponm=131
ponr-17.53
ponr-
ponto
ponn=
bpp-0
hppm0.45,
2pp-2
byaam
bppm15 15
1pp-r0 00.
bigp00.45
1ppm2.16 16.
bpp psnr=38.80
6pp=15 63,
05hr=13.45
20.r=20.92
29.96
psnr
pann-
ponne
ponr=17.93
ponr-
ponr-33.67
Figure 5: Example progressive reconstructions from UQDM trained with T 4, obtained with denoised prediction (left) or ancestral sampling (right). The latter avoids blurriness but introduces graininess at low bit-rates, likely because the UQDM is unable completely capture the data distribution and achieve perfect realism (perfect realism is also difficult to achieve also for Gaussian diffusion, as seen the rate-realism plot of (Theis al., 2022)). Flow-based reconstructions are qualitatively similar to the denoising-basee reconstructions and can be found in Figure 8.
at higher bit-rates, our method continues to improve quality and realism gradually, even at higher bit-rates. Refer to Figures 1, 5 and 8 for qualitative results demonstrating progressive coding and comparison across codecs. At high bit-rates, UQDM preserves details better than other neural codecs. UQDM with denoised predictions tends introduce blurriness, while ancestral sampling introduces graininess at low bit-rates, likely because the UQDM is unable to completely capture the data distribution and achieve perfect realism. Flow-based denoising matches the distortion of denoised predictions but achieves significantly higher realism measured by FID. We note that the ideal of perfect realism (i.e., achieving divergence between the data distribution and model's distribution) remains challenge even for state-of-the diffusion models.
DISCUSSION
In this paper, we presented a new progressive coding scheme based on novel adaptation of the standard diffusion model. Our universally quantized diffusion model (UQDM) implements the idea of progressive compression with unconditional diffusion model (Theis al., 2022) but bypasses the intractability of Gaussian channel simulation by using universal quantization (Zamir & Feder, 1992) instead. We present promising first results that match or outperform classic and neural compression baselines, including a recent progressive neural image compression method (Jeon et al., 2023). Given the practical advantages of progressive neural codec allowing for dynamic trade-offs between rate, distortion and computation, support for both lossy and lossless compression, and potential for high realism, all in single model our approach brings neural compression a step closer towards real-world deployment.
Future work may further improve our approach close the performance gap Gaussian diffusion; the latter represents the ideal lossy compression performance under perfect realism constraint data source (Theis al., 2022). This may require more
for an approximately Gaussian-distrin sophisticated methods for computing progressive reconstructions that can achieve higher quality with fewer steps, or exploring different parameterizations the forward and reverse processes with better thcoretical properties. Finally, expect further improvement in computation efficiency and scalability when combining our method with ideas such as latent diffusion (Rombach al., 2022), distillation (Sauer al., 2024), consistency models (Song et al., 2023).
10

---

Published as a conference paper at ICLR 2025
ETHICS STATEMENT
Our work focuses on the methodology of learning-based data compression method, and thus has no direct ethical implications. The deployment of neural lossy compression however carries with it risks of miscommunication and misrepresentation (Yang et al., 2023), and needs to carefully analyzed and mitigated with future research.
REPRODUCIBILITY STATEMENT
We include proofs for all theoretical results introduced in the main text in Appendix A and B. We include further experimental and implementation details (including model architectures and other hyperparameter choices) in Appendix C. Our code can be found at https: wtggthuuuu..ee com/ mandt-lab/uue /ugdm.
ACKNOWLEDGMENTS
Justus Will and Yibo Yang acknowledge support from the HPI Research Center in Machine Learning and Data Science UC Irvine. Stephan Mandt acknowledges support from the National Science Foundation (NSF) under an NSF CAREER Award IIS-2047418 and IIS 20-70077799 19, the NSF LEAP Center, by the Department of Energy under grant DE-SC0022331, the IARPA WRIVA program, the Hasso Plattnrr Research Center at UCI, the Chan Zuckerberg Initiative, and gifts from Qualcomm and Disney. We thank Kushagra Pandey for feedback the manuscript.
REFERENCES
Eirikur Agustsson and Lucas Theis. Universally Quantized Neural Compression. NeurlPS, 2020.
Johannes Ballé, David Minnen, Saurabh Singh, Sung Jin Hwang, and Nick Johnston. Variational Image Compression with Scale Hyperprior. ICLR, 2018.
Johannes Ballé, Sung Jin Hwang, Nick Johnston, and David Minnen. Tensorloww-ompprsssonn: Data compression in tensorflow. URL https: g thub. http://nnssrrf.eem compression inn.
Arpit Bansal, Eitan Borgnia, Hong-Min Chu, Jie Li, Hamid Kazemi, Furong Huang, Micah Goldblum, Jonas Geiping, and Tom Goldstein. Cold Diffusion: Inverting Arbitrary Image Transforms without Noise. NeurlPS, 2024.
Fabrice Bellard. Bpg image format, 2018. URL
Marlène Carcil. Matthew J Muckley, Jakob Verbeek, and Stéphane Lathuilière. Towards Image Compression with Perfect Realism at Ultra-low Bitrates. ICLR, 2023.
Zhengxue Cheng, Heming Sun, Masaru Takeuchi, and Jiro Katto. Learned Image Compression with Discretized Gaussian Mixture Likelihoods and Attention Modules. In IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 7939-7948, 2020.
Paul Cuff. Communication requirements for generating correlated random variables. In 2008 IEEE International Symposium Information Theory, pp. 1393-1397. IEEE, 2008.
Zhihao Duan, Ming Lu, Zhan Ma, and Fengqing Zhu. Lossy Image Compression with Quantized Hierarchical VAEs. In IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 198-207, 2023.
Rick Durrett. Probability: theory and examples, volume 49. Cambridge university press, 2019.
Gergely Flamich. Greedy Poisson Rejection Sampling. NeurlPS, 2024.
Gergely Flamich, Marton Havasi, and José Miguel Hernández-Lobato. Compressing Images by Encoding their Latent Representations with Relative Entropy Coding, NeurIPS, 2020.
Gergely Flamich, Stratis Markou, and José Miguel Hemmndezz-Lobato. Fast Relative Entropy Coding with * Coding. ICMI., 2022.
11

---

Published as a conference paper at ICLR 2025
Gergely Flamich, Stratis Markou, and José Miguel Hernández-Lobato, Faster Relative Entropy Coding with Greedy Rejection Coding. NeurlPS, 2024.
Daniel Goc and Gergely Flamich. On Channel Simulation with Causal Rejection Samplers. arXiv preprint arXiv:2401.16575 2024.
Marton Havasi, Robert Peharz, and José Miguel HemándezzLobato. Minimal random code learning: Getting bits back from compressed model parameters. arXiv preprint arXiv:1810.0044 00440, 2018.
Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising Diffusion Probabilistic Models. NeurIPS, 2020.
Emiel Hoogeboom, Eirikur Agustsson, Fabian Mentzer, Luca Versari, George Toderici, and Lucas Theis. High-Fidelity Image Compression with Score-Based Generative Models. arXiv preprint 11X11:2305.18231, 2023.
Seungmin Jeon, Kwang Pyo Choi, Youngo Park, and Chang-Su Kim. Context-Based Trit-Plane Coding for Progressive Image Compression, In JEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.
2023.
Diederik Kingma, Tim Salimans, Ben Poole, and Jonathan Ho. Variational Diffusion Models. NeurlPS, 2021.
Jae-Han Lee, Seungmin Jeon, Kwang Pyo Choi, Youngo Park, and Chang-Su Kim. DPICT: Deep Progressive Image Compression using Trin-Planes. In IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 16113-16122, 2022.
Jooyoung Lee, Se Yoon Jeong, and Munchurl Kim. DeepHQ: Learned Hierarchical Quantizer for Progressive Deep Image Coding. arXiv preprint arXiv:2408.1 12150, 2024.
Yadong Lu, Yinhao Zhu, Yang Yang, Amir Said, and Taco S Cohen. Progressive Neural Image Compression with Nested Quantization and Latent Ordering. In IEEE International Conference on Image Processing, pp. 539 339-543,11 2021.
David JC MacKay. Information Theory, Inference and Learning Algorithms. Cambridge University Press, 2003.
Alexander Quinn Nichol and Prafulla Dhariwal. Improved Denoising Diffusion Probabilistic Models. ICML, 2021.
Kushagra Pandey, Maja Rudolph, and Stephan Mandt. Efficient Integrators for Diffusion Generative Models. ICLR, 2023.
Yury Polyanskiy and Yihong Wu. Information theory: From coding to learning. Book draft, 2022.
Lawrence Roberts. Picture Coding using Pseudo-Random Noise. IRE Transactions on Information Theory, pp. 145-154, 1962.
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High- Resolution Image Synthesis with Latent Diffusion Models. In IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 10684-10695, 2022.
Axel Sauer, Frederic Boesel, Tim Dockhorn, Andreas Blattmann, Patrick Esser, and Robin Rombach. Fast High-Resolution Image Synthesis with Latent Adversarial Diffusion Distillation. arXiv preprint arXiv:2403. 2024.
Jascha Sohl-Dickstein, Eric Weiss, Niru Maheswaranathan, and Surya Ganguli. Deep Unsupervised Learning using Nonequilibrium Thermodynamics. ICML, 2015.
Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising Diffusion Implicit Models. ICLR, 2021a.
Yang Song and Stefano Ermon, Generative Modeling by Estimating Gradients of the Data Distribution. NeurIPS, 2019.
12

---

Published as a conference paper at ICLR 2025
Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-Basee Generative Modeling through Stochastic Differential Equations. ICLR, 2021b.
Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations. In International Conference on Learning Representations, 2021c.
Yang Song, Prafulla Dhariwal, Mark Chen, and Ilya Sutskever. Consistency Models. ICML, 2023.
Lucas Theis and Noureldin Y Ahmed. Algorithms for the Communication of Samples. ICML, 2022.
Lucas Theis, Tim Salimans, Matthew D Hoffman, and Fabian Mentzer. Lossy Compression with Gaussian Diffusion. arXiv preprini arXiv:2206. 08889, 2022.
James Townsend, Thomas Bird, Julius Kunze, and David Barber. HiLLoc: Lossless Image Compres-
sion with Hierarchical Latent Variable Models. ICLR, 2024.
Pascal Vincent. A Connection Between Score Matching and Denoising Autoencoders. Neural Computation, pp. 1661-1674, 2011.
Ruihan Yang and Stephan Mandt. Lossy Image Compression with Conditional Diffusion Models. NeurlPS, 2023.
Yibo Yang, Stephan Mandt, Lucas Theis, et al. An Introduction to Neural Data Compression. Foundations and Trends Computer Graphics and Vision, pp. 113-200, 2023.
R. Zamir and M. Feder. On Universal Quantization by Randomized Uniform/Lattice Quantizers. IEEE Transactions on Information Theory, pp. 428-436, 1992.
13

---

Published as a conference paper at ICLR 2025
APPENDIX
A FORWARD PROCESS DETAILS
A.1 GAUSSIAN (DDPM/VDM)
For completeness and reference, we restate the forward process and related conditionals given in (Kingma et al., 2021). The forward process is defined by
q(zz|x)
o(
where a and of positive scalar-valus functions of As in (Kingma et al., 2021), we define the following notation shorthand which are used in the rest of the appendix: for any 8 < I, let
at o = of
of oe a²
bejs = 0tt o of
0xs o,
Paje dels as ai
Ctaa
By properties of the Gaussian distribution, it can be shown that for any 0 < <TST,
((zz)z.) (a x, of,a 1) (Z Zt. x) (b..aa. + cx PI.
In particular,
((2 2e, x) NNbbjju11e + c1/-11x, P 1),
q(2, kT, x) -N(brii + o 1r
and we can use the reparameterization trick to write
2 1 b1 24 Crjt-1 8iie-1 E, E NO,
Z: bry zr + Cr BTy ET, ET ~ ((,
A.2 UNIFORM (OURS)
Our forward process is specified by q(zr)x (x) and ((z.-1 [2, x) for each 6, and closely follows that of the Gaussian diffusion. We set q(zT x) to be the same in the Gaussian case, i.e.
q(zr|x) == ((axx, of
and g(zz-1 [2,, x) to be uniform with the same mean and variance as in the Gaussian case, such that
q(zr- 12z,, x) - V3B 1 cij-1x v3.ggc-1);
+ -
or in other words,
2:-1 = b1it-12e + G + V120 u, ~ 1n((-1/2, 2/222//22/2
In the notation of eq. (4) this corresponds to letting b(t) = beje-11 c(t) = Ctjt-1+ A(t) = -112 follows by algebraic manipulation that
2, = bri 2T + CTP x +
/ 122 Up
(8)
where
u. ~ 2(-1/2, 1/2), u= = + I....I
are independent uniform noise variables, and
y-1
of at
8011 8olv-1
II
SNR(U SNR(e),
1=
14

---

Published as a conference paper at ICLR 2025
where
SNR(s) 0
It can be verified that
E [ 0,
T
Var (wi) =
Σ 0=
82
SNR(() SNR(T)) B7.CI,
or in other words, at any step & our forward-procest "posterior" distribution g(Ze|ZT, x) has the same mean and variance in the Gaussian case.
A.3 CONVERGENCE TO THE GAUSSIAN CASE
We show that both forward processes are equivalent in the continuous-time limit. To allow comparison across different number of steps T. we suppose that as and o, are obtained from continuous- timm schedules ax.. [0,1] + RT and (-)) [0,1] + R+ (which were fixed ahead of time), such that ......,, T, for any choice of T. As in VDM (Kingma et al., 2021), assume that the continuous-tiin time signal-to-ne -oisee ratio snr(.) (a-))/ is strictly monotonically decreasing.
ag = a(t/T) and 0; := a (t/T) for = O....
To obtain the continuous- time limit, hold the "continuous" time p fixed for some E [0, 1), and let T + oo (or equivalently, let the time discretization 1 - 0). We note that the quantities by CTp 37 only depend on p. and are thus well-defined when hold p fixed and let T oo:
of a(1) (p)
a of a(p) d2(1)
(1( a²(1)
(²(p) (p)
(pp)
CT
=2(1)
17.- (P(1))
a1
(p)
x'(p) (1) ap)
(p)
(snr(p) - snr(1)).
(p(
We start by showing that our q(z, |zr converges the corresponding Gaussian distribution in VDM in the continuous-time limit, which in turn implies the convergence of our q(z, x) to the corresponding Gaussian distribution in VDM.
Theorem A.I. For every fixed p== & E 10,1), ((2,222 S N(brie + cree x, x as T
Proof.
Recall the following fact in the forward process of UQDM (see eq. (8)):
2, = bT 2T + C
x
126 N.:
w=t+1
:=
where
u. ~ U(11121
1/2), ++ I....
are independent uniform noise variables, and
r-1
o7 VSNR(v SNN(())
80je == Bolv-1
II
0t
j=++1
where
SNR(s) = a
15

---

Published as a conference paper at ICLR 2025
It therefore suffices to show that wi converges in distribution to (O ( in the continuous- time limit. Since the different coordinates of w/ are independent, focus single coordinate and study the continuous-ting limit of a scalar Ω, given by a sum of scaled uniform variables,
Σ
VI20+(0) a(p)
sur
V 1-11 snr [ T
Us
(10)
T
v=t+1
a
(12
V /snr(p + J 1-1) snr(p + /)
Ug
(11)
a(p)
T
j=l
where U,'s are i..dd u(-1/2, 1/2) variables, and in the last step switched the summation index to j w-
n(T) T and
Define triangular array of variables by
X = (vT
j 1-1, 1-1 snr(p +) Uy. T
V snn(p
a(p)
for j = 1, 1,2.... n and for n E N+. For each n. Axn.11=1...... are independent variables with [[ 0, and can be verified that
72
(2(p) (snr(p) snr(1)).
E E Σ
Var (2te 1r
a²(
To apply the Lindeberg-Felde Feller central limit theorem (Durrett, 2019, Theorem 3.4.10) to Ω4 XR,1 +... Xx,,, remains verify the condition
O, lim
E(X2 n.j 1(xmm > c}} =
D. 12200
Let > O. Since snr(-) is continuous on a compact domain [0, 1]. is also uniformly continuous; then there exists a 6 such that
ca(p) (V120=(0)) Ven, T2, [x1 x22 < 8.
[snr(z1) snr(xe)) <
(12)
Let T (and thus = T t) become sufficiently large such that 1 8. Then, for all such T (and T.
thus n) sufficiently large, and for all j. holds that 1[|xn c) = almost everywhere:
P([Xnn) > E)= = (( (VI20(()
snr(p + j 1-1 sur(p +
10g >E
(13)
a(p)
T
= P U, ca(p)
(14)
v12 Vsur(p + ) snr(p + 4)
by eq. (12)
P(U > 1)
(15)
(16)
since U, ~ U(11222 1/2), and it follows that
2 n.j 1( > c) =
for j for sufficiently large n. We conclude by Lindeberg- theorem that
Ω = X + Xn.. d (O
T Applying the above argument coordinate- wise then proves the original statement.
16

---

Published as a conference paper at ICLR 2025
Corollary A.1.1.
If we assume TT and oT to be constants, then for every i, g(zi(x) - Naxx, of as T - co. that is, our forward model approaches the Gaussian forward process of VDM with an increasing number of diffusion steps.
Proof. As q(<r(z) = Na I does not depend on T. the joint distribution g(ze, ZT(æ) g(z, 2r, x)g((z()) converges in distribution, which in turn implies convergence of (Zz The statement then follows from the identity
of = Nilusbri 2T erpx of (2T T) dar.
B BACKWARD PROCESS DETAILS AND RATE ESTIMATES
B.1 GAUSSIAN (DDPM/VDM)
Kingma et al. (2021) set p(z:-1|z)) (x)) = g(2z-11222 21, = 81) = -N((be-124 1z + Caje-1 &c, 87 11 which yields
T- = KT.(N(br):-1) 24 C x, B3 1)) [1 Zt + G-/-11 8t, 8
2
41-11
|| &211
(SNR(t -1) - SNR(()) ||x - -x:112
= 2 Pt|t-1
2
We have that L, + 0 as + oc, due to the continuity of SNR( /T) = =sn(()=a())/(((())
B.2 UNIFORM (OuRS)
Recall that we choose each coordinate of the reverse-process model p(Zz-1/z:) to have the density
p(z)-1/z)); = gx(2) * ((; A1/2, A./2)
1 24/1/2
1 (Gi(z (o(2++414) GG(z -G) A1/2)). A.
A. 2- (e dz
where G3 and ge are the edf and pdf of distribution with mean fú == bujt-1 13 + Caje- 1 and variance o = (ze)i, == x,, and è = ( t); Using the shorthand f4 = bejc-1 2 + G we can derive the rate associated with the ith coordinate
レ- KL(U(Z;; Pr 4/2 u4 + A1/2) ll (z) * U(( -0:1(z A/21, 4/2.))
D. Im--au
1
pacta0/2
log
I
(G + A122) - Gi(z -11/2) dz
A.
1 1 t
4.
A1f2
log(Gi( + 14 + A1/2) - Gelz + uu -A1/2) dz.
A. L21/2
To gain some intuition for this rate, note that h(2) is lowest when most of the probability mass of Ge concentrated tightly around + μ4, which the case when 1 fdel small. Specifically, if G, in distributional family with standardized cdf Go such that Gz = Go((z - (p)/aa)) then
(G
if 2 + 14 p4 41/2
Gi(+ + me A.22) Gi p4 A122)
if 2 #4 Ag 4./2
else
as -+ 0. Thus, if |p, - Ael < 4e/2, we obtain improved bit-rates for o, that are small (relative A.). On the other hand, as almost certainly 114 fiel O, can't 't choose arbitrarily small
17

---

Published as a conference paper at ICLR 2025
0g because in that case both -t) - and L1-1 + 0g 0. This further motivates the merit of learning the backwards variances as og so(2) s0(2) =
allowing them to adapt lace -ill Conversely, by the mean value theorem, there exists one E (- A1/2) so that
Ge(z+ + pr + A3/2) Go(z A1/2) = Aege(= ue +c) Aege(z + pa)
where the last approximation becomes more accurate for larger dg- we further assume that G, is Gaussian (or sufficiently similar) l(()) becomes approximately quadratic. In that case study
h(2) = (1-
42 h(0) +
2x
D, h( h(-A//2) +
2x A
h(A./2).
A?
A
A
quadratic function that exactly matches h at values E {-Ar/2, 0, A1/2). Finally, this results in
1
2
01f2
1 (h(A:/2) M(+/2)))) / _asps
A1/2
z2 Ach(o))
Lt-1 ~
(h(-A./2) + h(A122) 2A(0))
2² +
A
A7
_
A.
1
(0) + h(--:/2) A(A/2)) N 3 log(2).
=
where the last equality uses h(=) < 0 and b(aA/))) S log(0.25) which follow from the fact that G. is a cdf. Empirically we note that this estimate very accurate as long as o 2 8/)-11 demonstrating that simply matching moments as in VDM will occur a constant overhead for each diffusion step. As number diffusion steps T might still lead to an increase in ELBO. Numerical inteericaliitegggeee integration of L confirms that if of is close to the optimal choice of 0g = 1i4 juel, L+-1 - 0 as T - oc in the Gaussian case.
in Figure 2, this can be partly mitigated with smaller o but increasing the
B.3 FLOW-BASED RECONSTRUCTIONS
Given an intermediate latent Zr, ancestral sampling yields an intermediate lossy reconstruction & p(x(2,) that requires obtaining reconstruction from Zu with the help of p(x[zg). This equivalent to approximately solving reverse SDE (Song et al., 2021c) and introduces additional noise during inference, which can make reconstractions grainy for difffusion models with a small number of steps, as can be seen in Figure 5. Song al. (2021c) further note that an alternative approximate solution the SDE can be obtained by deterministically reversing a "probability-foon -flow" ODE (see also Theis et al. (2022)). Specifically, this involves repeatedly evaluating 2:-1 f(z,,t) where for VDM is defined as
repeatedly sample from the conditional p(z,-1/2,) until finally
a:-1 f(z a2
at-1
01-1
0t-1 σ₃
2i +
0
01
él
2: + [1-1
aq
(17)
a₁
Tt
recovering the same process defined in (Song et al., 2021a). The equivalence of the continuous limit in Corollary A. suggests that the discrete-ting time backward processes of UQDM and VDM are similar enough in the sense that eq. (17) also approximately solves the implied reverse SDE of UQDM. Thus we eq. (17) to obtain flow-based reconstructions for both VDM and UQDM.
18

---

Published as a conference paper at ICLR 2025
Unconditional UQDM samples
T=2 2
T=5
T=10
T= 30
samples
Figure 7: Unconditional samples from UQDM models trained with varying T on the swirl dataset. The sample quality improves with larger T; however the compression performance becomes worse after T 5, as discussed in Section
25
200
VDM
UQDM (VDM weights)
100
UQDM (from scratch)
100
NELBO (bpd)
UQDM (learned rev. var.)
VDM (T 1000)
140
4.4
15
170
00
T
Figure 6: Left: 1000 samples from the toy swirl source. Right: Additional results on swirl data. We examined the compression performance of applying universal quantization to a pre-trained VDM model; conceptually this is equivalent to When using fixed reverse- reverse-process variances, we can directly re-use use weights from a pretrained VDM model (orange), which achieves comparable results training UQDM model from scratch, even for a smaller number of timesteps.
bpp=0.00 00. psnr-
bpp-=0.56,
bpp-2 36,
bpo-7.55 7,55,
bpp=18.00
psAr=20.54
parer 29.42
psnr 42.05
psiy--S 58
UADM bity
187K
UQDW small
bpp-0.00 00. psnrmB 61
bpp-0.56,
hpp-2 36,
bpa
bpp-16. 16, 07,
p5.5=20.54
29 42
psmr=42.00
pS
237M
5.0
NELSO (bits dimaension)
Figure 8: Additional results on ImageNet 64x64 data. Left: Example progressive reconstructions from UQDM trained with T = 4, obtained with flow-based denoising, as in Figure 5. Flow-based reconstructions achieve similar distortion (as meassured with PSNR) than denoised predictions at higher fidelity (as meassured with FID). Right: Ablation of the influence of model size validation loss. Bars are labeled with the number of parameters for each model. Increasing the size of the denoising network allows for smaller bitrates.
19

---

Published as a conference paper at ICLR 2025
C ADDITIONAL EXPERIMENTAL RESULTS
C.I SWIRL DATA
We use the swirl data from the codebase of (Kingma et al., 2021); Figure 6 shows 1000 samples from the toy data source. We use the same denoisng network X as in the official implementation? which consists of 2 hidden layers with 512 units each. Figure 6 highlights the consequence of Corollary 1.1.1: Because VDM and UQDM share the same continuous limit, we can use the weights of pretrained VDM to obtain comparable UQDM results as UQDM model that has been trained from scratch.
C.2 CIFARIO
We scaled-down version of the denoising network from the VDM paper (Kingma al., 2021) for faster experimentation. We use a U-Net of depth 8, consisting of 8 ResNet blocks in the forward direction and ResNet blocks in the reverse direction, with single attention layer and two additional ResNet blocks in the middle. We keep the number of channels constant throughout at 128.
We verified that our UQDM implementation based on tensorf forflloww -owwwwowoo compress achieves file size close the theoretical NELBO. When compressing a single 32x32 CIFAR image, we observe file size overhead S 3% of the theoretical NELBO. terms of computation speed, it takes our model with fixed reverse-proceet process variance less than 1 second to encode or decode CIFAR image, either on CPU or GPU,3 likely because the very few neural-network evaluations required (T = 4). For our model with learned eiverse-process variance, however, it takes about 5 minutes compress or decompress CIFAR image, with nearly all of the compute time spent on a single CPU core. This is because with learned reverse-proced process variance, each latent dimension has different predicted variance, and a separate CDF table needs to be built for each latent dimension during entropy coding; the tensor wwwsorflowwcomm ressi on library builds the CDF table for each coordinate in naive for-loop rather than in parallel. Thus we expect the coding speed to be dramatically faster with a parallel implementation of entropy coding, e.g., using the DietGPU" library.
C.3 IMAGENET 64 x 64
We use the same denoising network as in the VDM paper (Kingma et al., 2021). We use U-Net of depth 64, consisting of 64 ResNet blocks the forward direction and 65 ResNet blocks in the reverse direction, with a single attention layer and two additional ResNet blocks in the middle. We keep the number of channels constant throughout at 256. To investigate the impact of the size of the denoising network, in addition to this configuration with 237M parameters we call UQDM-big also run experiments with three smaller networks with 32 ResNet blocks and 128 channels (UQDM- medium, 122M parameters), 8 ResNet blocks and 64 channels (UQDM-small, 2M parameters), and 1 ResNet block and 32 channels (UQDM-tiny, 127K parameters), respectively. Smaller network are significantly faster and more resource-efficient but will naturally suffer from higher bitrates, as can be seen in Figure 8.
The required number of FLOPS per pixel for encoding and decoding is strongly dominated by the number of neural function evaluations (NFE) of our denoising network which depends on how soon we stop the encoding and decoding process. For lossless compression we have to multiple the FLOPS per NFE with T which is equal to 4 in our case. For lossy compression after t steps, with lossy reconstructions obtained through a denoised prediction, we obtain the required FLOPS for encoding and decoding by multiplying with Ê and + 1 respectively. The FLOPS per NFE depend on the network size, our investigated model size require 389K, 2.3M, 105M, and 204M FLOPS per pixel, in order from smallest to biggest model.
httpps: / twwb.thhbb.c..... www.///oooocoo e- resea httta//eeaeaee////////////aaaaaa httpa///coooabbbooooooooaaaaaaa ipynb
Around 0.6 s for encoding and 0.5 for decoding on Intel (R) Xeon (R) Gold 5218 CPU
2. 30GHz CPU; 0.5 for encoding and 0.3 s for decoding on a single Quadro RTX 8000 GPU.
httpps: / /i thub. com/ fww.bacckkkkeeee researchh dietggu
20

---

Published as a conference paper at ICLR 2025
Figures 9 and 10 show more example reconstructions from several traditional and neural codecs, similar to Figure 1. lower bitrates the artifacts each compression codecs introduces become more visible.
21

---

Published as conference paper at ICLR 2025
original
JPG
BPG
CDC
CDC
CTC
UQDM (ancestral)
UQDM (flow-based)
UQDM (denoise)
(p=0.9)
(p=0)
bpp=1.81
bpp=1.05
bpp=3.63
bpp=3.55 3.58
bpp=1.80
bpp=1.00
bpg=1.90 1.95
bpp=1.77
bpp=2.11
bpp-2.188
b.5-3.51
bpp-2..66
bpp-2.06
bpp=1.94
bpp-2.37
bpp-2.13
bap-3.44 3.46
bpa-3.22 25
bpp 2.344
bpp-24 11p-22.25
boa-1.68
bpp-2.. 2.21
bpp-2.2
bpp-2.34
8op-3.30
bpo-3
hpp-2.28
hpp-2.28
bpo-2.30
bpp-2.27
bpp-2.12
bppm2.19
bpp-3.88
bpa 3.55
bpp-2.12
bpp-2.10
bpo-2.13
bpp-2.11 2.11
bpp-2.15
bpp-2.14
bop-4.16 16
bpo-4.11
bpp=2.15
bpp-2.11
bpg-2.49
bpp-2.08
Figure 9: Additional example reconstructions, chosen roughly similar (high) bitrates.
original
JPG
BPG
CDC (p=0.9)
CDC (p=0)
CTC
UQDM (ancestral)
UQDM (flow-based)
UQDM (denoise)
A
bpp-0.73
bpp-0.59
bap-0.56 0.56
bpa-0.56
bpp-0.58
bpp-0.57
boo-0.48
bpp-0.54
bpp-0.78
hpp-0.78
bop-0.f6
bpg
hpp-0.75
hpp-0.75
bpa=0.53
bpp-0.50 66
bpp-0 68
bpp-0.51
bap-0 51
bpa-0 48
1opp-0.53
bpp-0.50
bpo-0 50
bpp-0.45
bpp=00.
bpp==0.61
0 62
bpp
bpp==0.6
bpp=:0.0 0.60
bpg=0.57
bpp=0.55
bpp-00
58
bpp-
bpp
boo
bpp-0.45 0.47
bop-0.73
bpp-0.63
bap-0.65
bpo-0.77
bpp-0.67
bpp-0.63
bog-0.50
bpo-0.63
Figure 10: Additional example reconstructions. chosen roughly similar (low) bitrates.
22


Reviews:
[{"review_id": "updDoUhthQ", "reviewer": "Reviewer_bjic", "summary": "This paper develops a new form of diffusion model based around universal quantization, which uses a uniform distribution in the reverse process instead of a Gaussian. Previous research made the connection between diffusion and progressive compression via relative entropy coding (REC), but those models used a Gaussian distribution which leads to computational complexity that is exponential in the amount of information being coded / transmitted.\n\nThis paper notes that \"universal quantization\" (Zamir & Feder, 1992) efficiently solves the REC problem in the case of a uniform noise channel. This presents an opportunity for efficient progressive compression with diffusion models if the diffusion process (forward and backward) are reformulated to use uniform noise. This paper does that (Section 3) and the resulting model is evaluated empirically in terms of compression performance using toy data and standard image data sets.", "questions": "I'm surprised that BPG does so well in terms of FID around 2.5 bpp (Fig. 4, right), where it even outperforms VDM (T=1000). Is there something to learn from this observation, or does it say something about the limits of VDM?", "limitations": "", "rating": 8, "confidence": 2, "soundness": 4, "presentation": 4, "contribution": 3, "strengths": "The primary strength of the paper is recognizing that diffusion models can be reformulated to use uniform noise, that this allows them to use universal quantization (UQ), and that UQ is an efficient solution to the REC problem for a uniform noise channel.\n\nThe authors then derive and implement this model, which they call a \"universally quantized diffusion model\" (UQDM), and evaluate it empirically to show that the theoretical benefits translate to practice as shown by the RD curve comparisons in Figures 3 and 4. They evaluate two approaches to intermediate reconstruction for progressive compression: ancestral sampling and denoising (Fig. 5) and show, qualitatively and quantitatively the pro/cons of each approach.\n\nAn additional strength is the clear and concise presentation of the background information (Section 2). While the basics of diffusion models are now probably familiar to most ICLR readers, relative entropy coding, universal quantization, and the connection between the diffusion process and progressive coding probably isn't.", "weaknesses": "Although there are many benefits to the proposed approach, it is not strictly better than previous methods at all bit rates and metrics (PSNR and FID are reported). The comparison is complicated somewhat due to a difference in capabilities, e.g., CDC has better FID than UQDM at low bit rates, but UQDM is progressive while CDC is not. Similarly, UQDM is fairly far behind the (theoretical) quality of VDM, but UQDM provides significant real-world benefits in terms of computational requirements.\n\nIn terms of computation and runtime, the benefit of UQDM over VDM is clear (and that's one of the primary points of the paper), but I suspect the practical decode time is still far too slow compared to standard codecs (JPEG, BPG, etc.) to be usefully deployed. This is an issue for much of the neural compression literature, though, so I don't think it should be a big factor in terms of the ICLR acceptance decision (e.g., CDC uses a diffusion-based decoder so is likely also slow in practice compared to JPEG and BPG).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "KxbUOPRIZp", "reviewer": "Reviewer_bu5E", "summary": "This paper introduces Universally Quantized Diffusion Models, a family of diffusion models utilizing uniform rather than Gaussian noise with the same forward/reverse process tractability (e.g., forward process available in closed form, admit a similar ELBO, etc.) This is achieved by constructing a probabilistic model following the DDIM factorization. The use of uniform noise is motivated in the context of compression, namely to make Relative Entropy Coding (REC) tractable through the use of uniform quantization. This allows the use of progressive coding methods used in prior work exploring compression with diffusion models (Ho et al., Theis et al.) for both lossless and lossy compression. Experiments are presented on a toy dataset of swirl curves and on standard+challenging image benchmarks (CIFAR10, ImageNet 64x64), showing encouraging results compared to standard image compression methods (e.g., jpeg) in the rate-distortion frontier, ditto for other non-neural methods (e.g., bpg) though only at certain rate regimes, and readily outperforming other neural methods such as CTC and CDC.", "questions": "- Is the fundamental benefit of REC through universal quantization that it enables both lossless and lossy compression? Or what other reasons would we want to do this as opposed to directly following prior work where tractability is sufficient to show results in lossy compression for the same image datasets? This could be explained more succinctly in the paper. \n- Are image compression results with rate > 8bpp important to show in any way?\n- For lossless compression (Fig 2), are there explanations or at least intuitions for why more denoising steps can result in worse bpd? This seems like an unexpected result.\n- This is likely a question from my own lack of experience in this area: suppose that we define the model to first apply the standard Gaussian erf on the input image (re-scaled dependent on the noise level). Does this not convert Gaussian noise into uniform noise (given that through the inverse erf, more specifically through the quantile function of the Gaussian, we can convert uniform noise into Gaussian noise)? Why does communicating with Gaussian noise remain intractable?", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "- The paper essentially introduces a formulation of diffusion models using uniform (as opposed to Gaussian) noise, effectively turning diffusion models into a method that through end-to-end optimization directly approximates the quantile function of the data distribution, which can be broadly useful beyond the context of compression.\n- The mathematical background and literature review are well-presented. \n- Even if encoding and decoding uses significantly more FLOPs compared to common non-neural compression schemes, using more compute for less memory can be useful in applications where storage matters more than reads, motivating research on deep-learning-based compression.\n- The method is applicable to both lossless and lossy compression.\n- Experiments include both toy examples (swirl dataset) and real image compression tasks on difficult benchmarks standard in the literature (CIFAR10, ImageNet 64x64)\n- Results on CIFAR10 and ImageNet 64x64 outperform various common lossy image compression schemes like jpeg, jpeg2000, and at certain rates outperform other compression schemes like bpg.\n- Results significantly outperform neural-based methods from prior work such as CTC, CDC. \n- The importance of learning variances is demonstrated experimentally.\n- Further experimental knowledge is introduced, e.g., that learning the noise schedule does not result in significant improvement in log likelihood optimization.", "weaknesses": "The biggest weakness seems to be a lack of comparisons against other diffusion-based lossy compression methods from prior work on the image compression tasks. E.g., DDPM is open sourced so this should be trivial to try. I acknowledge that at least two neural codec baselines (CTC, CDC) are included so this is not a fatal weakness. Still, positive results here would further motivate why we would want to follow the proposed method as opposed to prior ones.\n\nParticularly at low bpp rates, bpg which is efficient even in low-memory settings still outperforms diffusion models. This is not a major weakness, as bigger models could result in lower NELBO across the board. The paper could be strengthened by showing the effects of models of different sizes, even if the largest scales are kept reasonably small.\n\nA similar potential improvement to the paper, especially to discuss the practicality of diffusion-based compression, would be to include quantitative comparisons of FLOPS compared to standard image compression schemes.\nThere are no illustrations of the progressive coding procedure. This would be extremely helpful to include so the method is easier to grasp and follow.\n\nAnother potential improvement would be to include comparisons to other methods that are compatible with Gaussian noise, particularly for lossless compression. For example, it should be possible to compare to bits-back coding, however I acknowledge that this cannot be used to compress individual images (as the paper’s method does) but rather entire datasets, with rate dependent on the number of elements being compressed. Highlighting the benefits of the method, e.g., in a table of desiderata, would be additionally helpful.\n\nShowing more image samples, and showing zoom-ins to help the reader see artifacts across the qualitative comparisons would be very helpful. Expanding the range of rates (bpp) on the samples would also be very welcome.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "j4EBAVm5rh", "reviewer": "Reviewer_ouDv", "summary": "A single diffusion model can be used for data compression at arbitrary bit rates, flexibly trading off rate and distortion, using relative entropy coding (REC). The main drawback is that REC has in general exponential runtime. In this paper, the authors replace Gaussian distributions in the forward process (viewing diffusion models as hierarchical VAEs with a finite number of stochastic layers) with uniform distributions. This allows them to apply universal quantization which leads to cheaper simulation, thereby avoiding the intractability of previous work that was based on Gaussian noise. In addition, the authors show that their diffusion model with uniform noise converges to standard VDM in the continuous-time limit. In practice, this method obtains competitive R-D and R-realism performance, with a single model.", "questions": "- **Appendix C.2 and C.3**: In VDM, the standard architecture typically includes an additional ResNet block in the reverse process (so I would expect 9 blocks here). Could this be a typo?\n\n- **Continuous-Time Equivalence**: Given the equivalence with Gaussian diffusion in continuous time, could this approach be applied to pretrained continuous-time diffusion models by discretizing with uniform distributions? I assume this would be an approximate discretization, as it only matches the mean and variance.", "limitations": "", "rating": 8, "confidence": 3, "soundness": 2, "presentation": 2, "contribution": 3, "strengths": "- This paper presents an interesting idea with relevant contributions for the community. The focus on improving compression with general-purpose diffusion models is exciting and relevant.\n- The theory developed around uniform-noise diffusion, along with its connection to standard Gaussian diffusion in the continuous-time limit, is well-articulated and adds depth to the work.\n- The results seem quite promising.", "weaknesses": "- **Clarity Issues in Background**: The paper could be clearer, especially in the background section. It would help if it were more accessible to readers who aren't already familiar with probabilistic generative models for compression. For example:\n  - It would be useful to give a more intuitive explanation of transmitting a sample from $q$ with close to KL nats, as this is central to the argument about the NELBO corresponding to the lossless coding cost.\n  - Some wording is a bit vague, like \"using roughly $L_{\\mathbf{x}|\\mathbf{z}_0}$ nats.\" This phrasing might make sense to researchers in this specific area, but it could be clearer for others (in particular \"roughly\" in what sense?).\n  - It’s also unclear what exactly the relationship is between REC and universal quantization. My impression is: (1) REC is usually inefficient (with exponential runtime, but exponential in what?), (2) in some specific cases, it may be more efficient, (3) past work has tackled a few of these cases, (4) this paper presents another one of these (which had never been explored before) using a uniform noise channel. I don't know if this is accurate, but in any case the overall picture and context could be a bit more straightforward than it is in the current submission.\n\n- **Structure and Flow**: Wouldn’t it make sense to first define the new diffusion model class with uniform distributions, then explain that this enables universal quantization and more efficient REC? This could better separate the two main ideas: the diffusion model itself (\"substituting Gaussian noise channels for uniform noise channels\" -- line 185) and its application to compression. If that’s not a sensible structure and my interpretation is completely off, then there may be additional clarity issues that need attention. (I don’t doubt that experts in this exact topic would follow along, though.)\n\n- **Exponential Runtime Explanation**: There are a few statements about REC's exponential runtime and how this new approach is more efficient. But it’s not clear what \"exponential\" means here or what it’s exponential with respect to. It would help if the paper included: (1) a clear explanation of what exactly is exponential, (2) the complexity of the proposed algorithm (this doesn’t seem to be covered in the main text or appendix), and (3) solid empirical and quantitative support for this efficiency claim (again, this doesn't seem to be mentioned or investigated?).\n\n- **Algorithm Box**: Including a box or diagram for the main algorithm could help make things clearer. It would give readers a more accessible overview and something concrete to reference.\n\n**Minor**:\n- Typo on line 214 (missing closing parenthesis in the sigmoid).\n- Figure layout could be improved: there’s a lot of whitespace, some figures appear too early relative to where they’re discussed, and the order seems off (e.g., text references Figures 3, 4, 1, 5 in this order, and Figure 2 only appears in the Appendix).\n\n**Overall**: This paper introduces an interesting approach to using diffusion models for flexible data compression, but in my opinion it would significantly benefit from some improvement in clarity and presentation. Making the key concepts easier to understand, reorganizing the content for better flow, and providing detailed support for the claims -- especially regarding complexity -- would be really helpful. Addressing these points will significantly strengthen the paper and enhance its impact.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "07hlpwxJk9", "reviewer": "Reviewer_evNJ", "summary": "### Background:\n\nHo et al [1] proposed an algorithm that compresses a data sample (e.g an image) down to as many bits as the DDPM’s log-likelihood estimate of that sample. This is very interesting since diffusion models are probably the most advanced image priors ever developed. This compression algorithm has the additional advantage of being naturally progressive, that is, the data is communicated as a sequence of progressively less noisy images. However, a key component of this algorithm, called *relative entropy coding*, has an exponential runtime for gaussian diffusion models. This exponential runtime is a key bottleneck to practical implementation.\n\n### Summary:\n\nThis paper introduces a clever solution to a key bottleneck of an otherwise very promising compression algorithm. The authors introduce a new diffusion process which is constructed for computationally efficient relative entropy coding. This solves the exponential runtime problem associated with Gaussian diffusion compression, and allows the authors to implement an actual compression codec, where prior works (Ho et al [1], Theis et al [3]), only calculated the hypothetical performance of this compression algorithm, since it was not practical to implement due to the exponential runtime issue. The authors create new likelihood models based on Kingma’s VDM [2]. The authors evaluate their compression model on three datasets: the toy “swirl” dataset, CIFAR10, and ImageNet64.", "questions": "I see your lossless compression results for the swirl data, but not for CIFAR10 or ImageNet. Would like to see these but they are not critically important.\n\nTheis et al [3] found probability flow based denoising to induce less distortion than ancestral sampling while still preserving realism. Any particular reason you chose ancestral sampling over this flow-based denoising?\n\nIf UQDMs and traditional gaussian diffusion models are equivalent in the continuous-time limit, do you think there’s a point at which you could trivially convert a DDPM into a UQDM without retraining it, just by replacing the gaussian noise at each timestep with uniform noise? Or am I misunderstanding something?\n\n## References:\n\n1. Ho: https://arxiv.org/abs/2006.11239\n2. Kingma: https://arxiv.org/abs/2107.00630\n3. Theis: https://arxiv.org/abs/2206.08889", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 4, "strengths": "Paper is well-written and well-motivated. Topic is theoretically interesting and IMO potentially high-impact. The proposed solution is clever, well-explained, and definitely meets the threshold for novelty.", "weaknesses": "I have only one major concern: While the authors’ modifications to Kingma’s Variational Diffusion Model (VDM) make relative entropy coding computationally tractable, these modifications also appear to significantly degrade VDM's performance. \n\nFor example, the VDM paper included a figure (Figure 3) with non-cherry-picked unconditional samples from their model. IIUC, in Figure 5 of the paper under review, the two bpp=0.00 images generated via ancestral sampling are comparable to the images Kingma et al.’s Figure 3. If this comparison is fair, UQDM has suffered a major degradation in sampling quality compared to VDM.\n\nI am pretty hung up on UQDM’s very low optimal step count. The authors find that their model achieves the best NELBO performance with just 4 or 5 denoising steps. This is counter to the VDM paper which concluded that more steps leads to a lower loss (see Kingma appendix F). \n\nThis seems problematic because a major theoretical underpinning of DDPMs is that the ideal $p(x_{t-1} | x_t)$ is well-approximated by a Gaussian as the step size becomes small. But with such large steps, the ideal $p(x_{t-1} | x_t)$ must surely have some very complicated structure which is poorly captured by the author’s parameterization. This might also explain why the authors see such an advantage from learning the reverse-process variance; you just need a lot more parameters to model this distribution accurately. With say T=1,000, I would guess that the fixed-variance version of UQDM would do a better job of modeling the reverse process. Except for some reason, UQDM’s performance degrades with increasing step size.\n\nMy gut reaction is that this ultra-small optimal step count seems pathological. But I am open to being convinced that I’m mistaken. What do you think is going on here?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Oral)