Compressed Image Generation with Denoising Diffusion Codebook Models
Authors: Guy Ohayon
Hila Manor
Tomer Michaeli
Michael Elad
Venue: ICML 2025 poster (2025)
Keywords: image compression
diffusion models
score-based generative models
compressed real-world image restoration
compressed zero-shot image restoration
compressed posterior sampling
compressed image generation
Primary area: deep_learning->generative_models_and_autoencoders
TL;DR: We propose a novel approach to generate images along with their compact, lossless bit-stream representations. We leverage our method for image compression, as well as other compressed conditional generation tasks such as compressed image restoration.

Abstract: We present a novel generative approach based on Denoising Diffusion Models (DDMs), which produces high-quality image samples *along* with their losslessly compressed bit-stream representations. This is obtained by replacing the standard Gaussian noise sampling in the reverse diffusion with a selection of noise samples from pre-defined codebooks of fixed iid Gaussian vectors. Surprisingly, we find that our method, termed *Denoising Diffusion Codebook Model* (DDCM), retains sample quality and diversity of standard DDMs, even for extremely small codebooks. We leverage DDCM and pick the noises from the codebooks that best match a given image, converting our generative model into a highly effective lossy image codec achieving state-of-the-art perceptual image compression results.
More generally, by setting other noise selections rules, we extend our compression method to any conditional image generation task (e.g., image restoration), where the generated images are produced jointly with their condensed bit-stream representations.
Our work is accompanied by a mathematical interpretation of the proposed compressed conditional generation schemes, establishing a connection with score-based approximations of posterior samplers for the tasks considered.
Code and demo are available on our project's [website](https://ddcm-2025.github.io/).

Compressed Image Generation with Denoising Diffusion Codebook Models
Guy Ohayon Hila Manor 22
Tomer Michaeli?
2 Michael Elad
Abstract
for generating samples from complex signal distributions (e.g,, natural images). Hence, DDMs are commonly lever- aged to facilitate a variety of downstream tasks, such as eext-to-imaee synthesis (Ramesh et al., 2021; Rombach et al., 2022; Saharia et al., 2022), editing (Meng et al., 2022; Snneerman-Spieglglla et al., 2024), compression (Theis et al., 2022; Elata et al., 2024; Körber et al., 2024), and restoration (Kawar al., 2022: Chung et al.. 2023). Com- many of these applications is the reliance on iterative sampling from a continuous Gaussian distribution, yielding an unbounded representation space.
We present a novel generative approach based on Denoising Diffusion Models (DDMs), which pro- duces high-quality image samples along with their losslessly compressed bit-stream representations. This is obtained by replacing the standard Gaus- sian noise sampling in the reverse diffusion with a selection of noise samples from pre-defined code- books of fixed iid Gaussian vectors. Surprisingly, we find that our method, termed Denoising Diffu- sion Codebook Model (DDCM), retains sample quality and diversity of standard DDMs, even for extremely small codebooks. We leverage DDCM and pick the noises from the codebooks that best match given image, converting our generative model into a highly effective lossy image codec achieving statc-of-the-arr perceptual image com- pression results. More generally, by setting other noise selections rules, we extend our compres- sion method to any conditional image generation task (e.g., image restoration), where the gener- ated images are produced jointly with their con- densed bit-stream representations. Our work is accompanied by a mathematical interpretation of the proposed compressed conditional generation schemes, establishing connection with score- based approximations of posterior samplers for the tasks considered. Code and demo are available on our project's 's website.
This work embarks on the hypothesis that such an infi- nite representation space highly redundant. For example, consider any stochastic diffusion generative process with T 1000 sampling steps (e.g.. DDPM (Ho al., 2020)). Suppose that each timestep, the generative process is restricted to choosing between only two fixed noise realiza- tions. Sampling could then lead to 2000 different outputs, an incredibly large number exceeding the estimated amount of atoms in the universe. Thus, in principle, such a process could cover the distribution of natural images densely.
We harness this intuition and propose Denoising Diffu- sion Codebook Models (DDCM), novel DDM genera- tion scheme for continuous signals, leveraging discrete and finite representation space. In particular, we first con- struct a chain of codebooks, where each is a sequence of pre-sampled Gaussian noise vectors. These codebooks are constructed once and remain fixed for the entire lifetime of the model. Then, during the generative process, we simply randomly pick the noises from the codebooks instead of drawing them from Gaussian distribution, as shown in Fig. 2. Since we alter only the sampling process, DDCM can be applied using any pre-trained DDM. Interestingly, we find that our proposed discrete and finite representation space is indeed expressive enough to retain the generative capabilities of standard DDMs, even when using incredibly small codebooks. Since our generative process is entirely governed by the noise indices picked during the generation, an important consequence is that every generated image can be perfectly reconstructed by repeating the process with its corresponding indices.
1. Introduction
Denoising Diffusion Models (DDMs) (Sohl-Dickstein et al., 2015; Ho et al., 2020) have emerged as an effective tool
"Equal contribution 'Faculty of Computer Science, Tech- nion - Israel Institute of Technology. Haifa, Israel 'Faculty of Electrical and Computer Engineering, Technion Is- rael Institute of Technology, Haifa, Israel. Correspon- dence to: Guy Ohayon <guyoep@ gmmi, Hila Manor <hila.manor& campus. acchiion.c.il...
Proceedings of the 4nnd International Conference Machine Learning, Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).
We leverage this property to solve a variety of tasks, using gradient-free noise selection rules to guide the DDCM gener-

---

Compressed Image Generation with Denoising Diffusion Codebook Models
Image Compression
Image Compression
Compressed Image Generation
Compressed Image Restoration
Input
Reconstruction
Eapu
Reconstruction
Input: A mon trovels down a path in the mountains.
Imput
Reconstrictie
1260
900
2960
Figure 1. Our proposed scheme (DDCM) produces visually appealing image samples with high compression ratios (bottom right corners).
ation process, In particular, by choosing the discrete noises to best match a given image, we achieve state-of-the-art per- ceptual compression results. Morcover, using DDCM with different noise selection rules yields versatile framework for other compressed conditional generation tasks, such as compressed image restoration (see examples in Fig. I). Finally, we provide a mathematical interpretation of the proposed schemes based on score-based generative model- ing with SDEs (Song et al., 2020), showing a connection between our generalized selection rules and approximate posterior sampling for compressed conditional generation.
Discrete Generative Modeling for Continuous Data. Recent works have explored discrete generative modeling of continuous data distributions. These employ various discrete representations, such as vector quantized latent to- kens (Wu et al., 2024) or hierarchical modeling schemes that gradually refine each generated sample (Yang, 2025), DDCM offers a new such discrete generative framework, building on the exceptional achievements of DDMs.
Conditional Image Generation with Pre- Pre-Trained DDMs. Pre-trained DDMs are commonly utilized for solving con- ditional image generation tasks, such as image restora- tion (Kawar et al., 2022; Lugmayr et al., 2022; Wang et al., 2023b; Chung et al., 2023; Lin et al., 2024; Song al.,
2. Related Work
Compression. Image compression has seen significant progress in recent years, with the penetration of neural net-
2023; Cohen et al., 2024b; Yue & Loy, 2024; Raphaeli et al.,
2025; Man et al., 2025) and editing (Meng et al., 2022; et al., 2024; Cohen et al., 2024a; Manor & Michaeli, 2024). In this work we address these tasks from the lens of DDCM, where the conditional sam- ples are generated along with their compressed bit-streams.
works to this domain. Neural methods range from construct- ing specialized architectures (Ballé et al., 2017; Zhu al., 2022; Jiang et al., 2023; Jiang & Wang, 2023) to relying on different generative models such as Generative Adver- sarial Networks (GANs) (Mentzer et al., 2020; Muckley et al., 2023; Iwai et al., 2024) and Variational Autoencoders (VAEs) (Theis et al., 2017). Recent compression methods leverage DDMs and offer high perceptual quality results, by training models from scratch (Yang & Mandt, 2023; Ghouse et al., 2023), fine-tuning existing models (Careil et al., 2023; Körber et al., 2024), or using pre-trained DDMs in a zero-shot manner (without further training) (Theis et al., 2022; Elata et al., 2024). Current solutions in the latter category are highly computationally demanding, either due to their communication schemes (e.g., reverse channel cod- ing (Theis& & Ahmed, 2022: Theis et al., 2022)) or their need to perform thousands of denoising operations (Elata et al., 2024). Our work falls into this last category, offering novel and highly effective compression scheme with a fast bit-stream communication method and computational demands that match standard use of DDMs.
Juuberman-Spiegeleee
Compressed Image Generation. The task of compressed image generation (generating images directly in their com- pressed form) has been previously explored. Kang al. (2019) trained an unconditional GAN (Goodfellow et al., 2014) to synthesize JPEG representations. Rajesh et al. (2023) trained a text-conditional GAN in the JPEG domain. In this work, we use DDMs instead of GANs and introduce a novel compressed image representation space different than that of JPEG. Our approach is compatible with any praitre DDM without requiring additional training.
Compressed Conditional Image Generation. Our work also addresses the problem of compressed conditional gen- eration (see Sec. 6), in which the compressed output is generated in accordance with a given input condition, such as a text prompt, a noisy image, or other forms of guid- ance. In this context, Liu et al. (2021) proposed a method based on optimal transport, subject to an informational bot-
2

---

Compressed Image Generation with Denoising Diffusion Codebook Models
x 1
H
+
X
Hi
Generation
(1,25,67....,1000
Choose k
Choose k
XT
Random
(P)
Compression
(7.42,10,......))
Similarity to target imagee
(3)
Compressed Representations
Selection Rules
Fixed Codebook C++1
Fixed Codebook C;
Compressed Representation (kT+1; KT. k4t1 Rss 122)
Figure 2. Method overview. DDCM replaces the standard Gaussian noises in DDPM sampling with a selection of noise samples from pre-defined codebooks of fixed iid Gaussian vectors. This retains the high-quality generative properties of standard DDMs, while producing the results along with their compressed representations. By choosing the discrete noises according to different selection rules, DDCM can perform variety of conditional image generation tasks. Our highly condensed bit-stream representation is especially effective for image compression, leading to state-of-the-art results.
tleneck constraint. Specifically, their approach involves computing an optimal transport map from the distribution of input conditions to the distribution of target signals, while constraining the entropy of an intermediate latent random variable (i.e., the code). This method requires both the in- puts and outputs to reside within the same metric space. In contrast, our approach naturally accommodates more com- plex types of input-output pairs, such as those involving different modalities (c.g.. text and image).
Such a score function (xx), is typically learned via denois- ing score matching (Vincent, 2011; Song & Ermon, 2019; Song et al., 2020; Ho et al., 2020), where model Xoje is trained to predict X, from x; (i.e., a denoiser), and using the well-known equation (Robbins, 1956; Miyasawa et al., 1961; Stein, 1981)
V 01; Xo X;
8; (x;))
(4)
1 a:
where ā; := II == O... This generative process is appli- cable to both pixel space (Dhariwal & Nichol, 2021) and latent space (Rombach et al., 2022) diffusion models, by employing a Vasebased encoder-decoder.
3. Background
Diffusion models (Sohl-Dickstein et al., 2015; Ho et al., 2020; Song al., 2020) generate samples from a data dis- tribution Po by reversing a diffusion process that gradually adds random noise to samples from the data. Specifically, the diffusion process starts with Xg ~ Po and produces the chain No; X1, XT via
4. Denoising Diffusion Codebook Models
Method. Equation (2) depicts the standard DDPM sam- pling approach, where the added noise is sampled from a continuous Gaussian distribution. DDCM instead uses a discrete noise space, by limiting each sampling step to choose from K constant noise realizations, fixed separately for each step. Formally, for each i 2,..., T..,T++ we define codebook of K entries
0,,;-1 -1- 0:%
~ N(O, I), (1)
where 01, aT > 0 are some time-dependent constants. The above dime-iscretizzatin of Variance Presery- ing (VP) SDE (Song et al., 2020). Then, samples from the data distribution Po generated by solving the cor- responding reverse-time VP SDE (Anderson, 1982; Song et al., 2020), i.e., by gradually denoising samples, starting from XT ~ (0 I). In this paper we adopt Denoising Diffusion Probabilistic Models (DDPMs) (Ho et al., 2020), which propose the generative process (i- = =TT...... T,..., 1)
C,
2 (1)
(2)
[3)
(5)
where each C, = (k) is sampled independently from N( I) and remains fixed throughout the model's lifetime. Then, we modify the DDPM sampling process (2), replacing the noise 2, by randomly selected codebook entry,
x = (x;) + oiZi, where
(2)
1
(x(x;)= = (x: +1 (1)0))),,,x))
(3)
x (x:())) CkC)(,s),
(6)
Va:
Z, ~ N - and s (x;) denotes the score Xx, log):(x;) of the probability density function px():).
where the process is initialized with xy = C1+1(kr4a) h; ~ Unif((1...... K)), and sampling step i = does not
3

---

Compressed Image Generation with Denoising Diffusion Codebook Models
Pixel Space (ImageNet)
Latent Space (MS-COCO)
tasks, including image compression and compressed image restoration.
10'
x10'
1.7
1.05
DDPM
DDPM DDCM
5. Image Compression with DDCM
1.00
DDCM
1.6
Since sampling with DDCM yields compact bit- stream representations, a natural endeavor is to harness DDCM for compressing real images. In particular, to com- press image Xo, we leverage the predicted Xojs (Bq. (4)) each timestep i and compute the residual error from the tar- get image, Xo Rooz- Then, we guide the sampling process towards xo by selecting the codebook entry that maximizes the inner product with this residual,
Method.
0.95
1.5
2
8
K 64
16
2
8
16
64
K
Figure 3. Comparing DDPM with DDCM for different code- book sizes K. Interestingly, DDCM with K 64 achieves similar FID to DDPM, suggesting that the continuous representation space of DDPM (DDCM with = 00) is highly redundant. We use a class-conditional ImageNet model (256 256) for pixel space, and the text conditional SD 2.1 model (768 x 768) for latent space, with prompts from MS-COCO. The K axis is in log-scale.
h;
arg max (C,, - -x01t)> kE(1....KK
where the size of the first codebook Cr+1 is K = 1. This process is depicted as the compression branch in Fig. 2, where the resulting set of chosen indices {k:)/22 T+1 is the compressed bit-stream representation of the given image. Section 6 sheds more light on this choice of the noise se- lection from the perspective of score-based generative mod- els (Song et al., 2020). As in Sec. 4, decompression follows standard DDCM sampling (6), re-selecting the stored in- dices instead of picking them randomly. When using latent space DDMs (e.g., SD), we first encode xo into the latent domain, perform all subsequent operations in this domain, and decode the result with the decoder.
involve noise addition. This random selection procedure is depicted in the generation branch in Fig. 2. Importantly, running the generative process (6) with a given sequence of noise vectors {2) always produces the same output image. Thus, as depicted in the bottom part of Fig. 2, the sequence of indices kr.,,..., 22 can be considered a losslessly compressed bit-stream representation of each generated image.
Experiments. While DDPM is equivalent to DDCM with K K= 00, the first question we address is whether DDCM maintains the synthesis capabilities of DDPM for relatively small K values. We compare the performance of DDPM with that of DDCM using K E {2, 8, 16, 64) for sam- pling from pre-trained pixel and latent space models. We compute the Fréchet Inception Distance (FID) (Heusel et al., 2017) to evaluate the generation performance. In App. A we report additional metrics and provide qualitative compar- isons. For pixel space generation, we use a class-conditional DDM trained on ImageNet 256 x 256 (Deng et al., 2009; Dhariwal & Nichol, 2021), and apply classifier guidance (CG) (Dhariwal & Nichol, 2021) with unit scale. We use the 50k validation set of ImageNet as the reference dataset, and sample 10k class labels randomly to generate the images. For latent space, we use Stable Diffusion (SD) 2.1 (Rom- bach et al., 2022) trained on 768 x 768 images and apply classifier-free guidance (CFG) with scale 3 (equivalent to w = 2 in (Ho & Salimans, 2021)). As the reference dataset, we randomly select 10k images from MS-COCO (Lin et al., 2014; Chen et al., 2015) along with one caption per image, and use those captions prompts for sampling.
The bit rate of this approach is determined by the size of the codebooks K, and the number of sampling timesteps T. Specifically, the bit-stream length is given by (T - (T-1))oog log, (K). Therefore, the bit rate can be reduced by simply decreasing the number of codebooks, by using smaller number of timesteps generation, e.g., by skip- ping every other step, or by using specific timestep intervals (see App. B.4). In the approach described so far, the length of the bitstream increases logarithmically with K, making it computationally demanding to increase the bit rate. For instance, even for K = 8192, T = 1000 and 768 x 768 images our BPP is approximately 0.022. Thus, to produce higher bit rates, we propose refine the noise selected timestep by employing matching pursuit (MP) (Mallat & Zhang, 1993). Specifically, at each step i, we construct the chosen noise as a convex combination of M elements from C₁ gathered in a greedy fashion to best correlate with the guiding residual Xg - Xujs (as in Eq. (7)). The resulting convex combination involves M 1 quantized scalar coeffi- cients, chosen from a finite set of C values taken from [0, 1]. Therefore, the resulting length of the bit-stream is given by (T - 1)(log (K)M + C(M - 1)). such that M = 1 is similar to our standard compression scheme, and the length of the bit-stream increases linearly with M and C. We ap- ply this algorithm when the absolute bits number crosses (T-1)- log, (213). Further details are available in App. B.5.
As shown in Fig. 3, DDCM achieves similar FID scores to DDPM at K- = 64, suggesting that the Gaussian representa- tion space of DDPM is redundant. In the next sections we leverage our new representation space to solve a variety of
4

---

Compressed Image Generation with Denoising Diffusion Codebook Models
Original
BPG
PSC-P
PerCo
Ours (K 256)
0.033 BPP
0.049 BPP
0.031 BPP
0.030 BPP
0.035 BPP
0.049 BPP
0.031 BPP
0.030 BPP
Figure Qualitative image compression results. The presented images are taken from Kodak24 (512 512) dataset. Our codec produces highly realistic outputs, while maintaining better fidelity to the original images compared to previous methods.
Experiments. We evaluate compression method Kodak24 (Franzen, 1999), DIV2K (Agustsson & Timofte, 2017), ImageNet 1K 256 x 256 (Deng et al., 2009; Pan et al., 2020), and CLIC2020 (Toderici et al., 2020). For all datasets but ImageNet, we center crop and resize all images 512 x 512. We compare to numerous competing methods, both non-neural and neural, and both zero-shot, fine-tuning based, and training based approaches. For the ImageNet dataset, we use the unconditional pixel space ImageNet 256 x 256 model of Dhariwal & Nichol (2021), and com- pare our results to BPG (Bellard, 2018), HiFiC (Mentzer et al., 2020), IPIC (Xu et al., 2024), and two PSC (Elata et al., 2024) configurations, iestortion-oriinted (PSC-D) and perception-oriented (PSC-P). For all other datasets, we SD 2.1 512 512 (Rombach al., 2022) and com- pare to BPG, PSC-D, PSC-P, ILLM (Muckley al., 2023), PerCo (SD) (Körber et al., 2024; Careil et al., 2023), and twoCRDR (Iwai et al., 2024) configurations, distortion- (CRDR-R) PSC shares the same pre-trained model as ours, while PerCo (SD) requires additional fine-tuning. For our method, we apply SD 2.1 unconditionally, as we saw improvement by adding prompts (see further details in App. B.6). We assess our method for several options of T, K, M, and C' to con- trol the bit rate. See further details in App. B. We evaluate distortion with PSNR and LPIPS (Zhang et al., 2018) and perceptual quality with FID (Birkkowski et al., 2018). For ImageNet, FID is computed against the entire 50k 256 x 256 validation set. For the smaller datasets we follow Mentzer al. (2020) and compute the FID over extracted image patches. Specifically, for DIV2K and CLIC2020 we extract
128 128 sized patches, and for Kodak we use 64 x 64.
As shown on the rate-distortion and rate-perception planes in Fig. 5, our compression scheme dominates previous methods on the rate- tutt-prrception-distotee
tradeoff (Blau & Michaeli, 2019) for lower bit rates, surpassing both the perceptual quality (FID) and distortion (PSNR and LPIPS) of previous methods. For instance, our FID scores are lower than those of all other methods at around 0.1 BPP, while, for the same BPP, our distortion performance is better than the perceptually-oorieneed methods (c.g., PerCo, PSC-P, and IPIC). However, our method under meder-peforrss at the highest bit rates, especially when using SD. When using latent DDM such as SD, we aim to compress the latent encoding of a given image, rather than the image itself. Since encod- ing and decoding image typically leads to distorted reconstruction, the distortion of the compression method is bounded by that of the encoder-decoder This leads to performance ceiling for any latent-space-based compres- sion method (Körber et al., 2024; Elata et al., 2024). We therefore include Fig. 5 the "SD 2.1 Encoder-Decoder bound", which corresponds the distortion resulting from encoding and decoding the original images using the VAE of SD 2.1, without any additional compression. The qualita- tive comparisons in Fig. 4 further demonstrate our superior perceptual quality, where, even for extreme bit rates, our method maintains similarity to the original images in fine details. See App. for more details and results.
oriented (CRDR-R) and perception-orientee
5

---

Compressed Image Generation with Denoising Diffusion Codebook Models
PSC-D
IPIC
HiFIC
ILLM
CRDR-D
SD 2.1 Iacoder-Decoder Bound
PSC-P
BPG
PerCo (SD)
CRDR-R
Ours
ImageNet2r0
Kodak24512
CLIC2020p12
DIV2Ks12
30
30
30
25
PSNR
20
20
20
20
0.50
LPIPS
0.4
0.25
0.25
10a
FID
10a
10a
10a
+
10
10-1
100
10-1
10
10
Bits per pixel
Bits per pixel
Bits per pixel
Bits per pixel
Figure 5. Compression quantitative evaluation. We compare the perceptual quality (FID) and distortion (PSNR, LPIPS) achieved different BPPs. The image sizes each dataset denoted next to name. Our method produces the best perceptual quality at most BPPs. Importantly, this while also attain lower distortion compared perceptually-orientee (SD)). For the three rightmost datasets, note that our approach, PSC-P, and PerCo (SD) use the latent space Stable Diffusion 2.1 model, its encoder-decoder imposes distortion bound. Thus, we report the distortion attained by simply passing the images through this encoder-decoder (dashed line).
methods (e.g., PSC-P and PerCo
6. Compressed Conditional Generation
Compressed Conditional Generation with DDCM. We propose generating a conditional sample by choosing the indices k in Eq. (6) via
We showed that DDCM can be used image codec by following simple index selection rule, guiding the gener- ated image towards target one. Here, we generalize this scheme to any conditional generation task, considering the more broad framework of compressing conditionally gener- ated samples. This particularly valuable framework in scenarios where the input condition y is bit rate intensive, e.g., where y is degraded image and the goal is to produce compressed high-quality reconstruction of it. To the best of our knowledge, this task, which we name compressed conditional generation, has only been thoroughly explored for text input conditions (Rajesh et al., 2023).
k; arg min Lly, xi, Xi, kE(L...,KK
(8)
instead of picking them randomly. Here, L(y, x;, C;,C,. k)) can be any loss function that attains a lower value when (k) directs the generative process towards an image that matches y. For example, for the loss
Cp(y, x,, C, k)) = [IC,(A) - a, (v (9)
obtain the following result (see proof in App. C):
Proposition 6.1. Suppose that image samples generated via process (6), and the indices K; are chosen according to Eq. (8) with C = Lp. Then, when K + co, such a generative process becomes discretization of a probability flow ODE over the posterior distribution po Px(ooly).
naive solution to this task could be simply compress the outputs of any existing conditional generation scheme. Here propose novel end-to-end solution that generates the outputs directly in their compressed form. Importantly, note that our approach in Sec. 4 requires the condition y for decompressing the bit-stream. While this not a strin- gent requirement when the condition is lightweight (e.g., a text prompt), this approach is less suitable when storage of the condition signal itself requires a long bit-stream. The solutions propose in this section enable decoding the bit-stream without access to
In other words, Proposition 6.1 implies that for the loss Lp. increasing K leads more accurate sampling from the posterior (x(xuly) ly). though this results in longer bit-streams. Thus, as long as we have access to Vx; log log,:({ (x;) (or an approximation of it) Lp may serve as a sensible option for solving a compressed conditional generation task with DDCM. Interestingly, we show in App. C.2 that our com- pression scheme from Sec. is special case of the proposed
6

---

Compressed Image Generation with Denoising Diffusion Codebook Models
compressed conditional generation method, with y Xo and L Lp-
6.2. Compressed Real- Face Image Restoration
Real-world face image restoration is the practical task of restoring any degraded face image, without any knowledge of the corruption process it has gone through (Wang et al.,
6.1. Compressed Posterior Sampling for Image Restoration
2021; Gu et al., 2022; Wang et al., 2022; Zhou et al., 2022;
Our compressed conditional sampling approach can be uti- lized for solving inverse problems via posterior sampling. Specifically, we consider inverse problems of the form y Axo, where A is some linear operator. We restrict our attention to unconditional diffusion models and solve the problems in a "zero-shot" manner (similarly to Kawar et al. (2022); Chung et al. (2023); Wang et al. (2023b)). To generate conditional samples, we propose using the loss
Wang et al.. 2023c; Lin et al., 2024; Yue & Loy, 2024;
Chen et al., 2024b; Ohayon et al., 2025). We propose a novel method capable of optimizing any no-reference image quality assessment (NR-IQA) measure at test time (e.g.. NIQE (Mittal et al., 2013)), without relying on gradients.
Specifically, at each timestep i, we start by picking two in- dices - one that promotes high perceptual quality, k.,,, and another that promotes low distortion, As Then, we choose between ks, and k the index that better optimizes a de- sired balance of the prrception-distortion tradeoff (Blau & Michaeli, 2018). Formally, letting r(y) E[xoly] denote the approximate Minimum Dean-Squared-Erron (MMSH) estimator of this task, we pick k..D via
x(y,x,,Co C,, k) jyy (() + (K))I (10)
Note that Eq. (10) attains lower value when (k)) points in the direction that perturbs (x; towards y- Thus, our conditional generative process aims to produce a reconstruction & that satisfies Ax = y, implying that we approximate posterior sampling (Ohayon et al., 2023). No- tably, when assuming that (; (x;) is a multivariate normal distribution centered around Ax; (as in (Jalal al., 2021)), the chosen codebook noise (, approximates the gradient Xx, (y) x.) and Eq. (10) becomes proxy of Eq. (9).
k = arg max (k),, r(y) -x0 (11) kE(l.....K)
Note that this index selection rule is similar to that of our standard compression, replacing Xo in Eq. (7) with r(y). This choice of indices in DDCM would lead to a reconstructed estimate of the MMSE solution y(y). yield- ing blurry results with low distortion (Blau & Michaeli, 2018). In contrast, randomly picking a sequence of in- dices DDCM would produce a high quality sample from the data distribution po- Therefore, we randomly choose k Unif {1, ...,...... K}). Then, use the DDM and com- pute Xo/s-1 for each index & [ks,D. k. separately, de- noting each result accordingly by {(({) xJ--1' The final index is picked optimize the perception-distorins
Following (Chung et al., 2023; Wang et al., 2023b), we implement our method using the unconditional ImageNet 256 x 256 DDM trained by Dhariwal & Nichol (2021). We fix K = 4096 for all codebooks, resulting a compressed bit-stream of approximately 0.183 BPP for each generated image. We compare our method with DPS (Chung et al., 2023) and DDNM (Wang et al., 2023b) on two noiseless tasks: colorization and 4x super-resolution (using the bicu- bic kernel). We evaluate these methods using their official implementations and the same DDM. We additionally com- press the outputs of DPS and DDNM to assess whether such naive approach would yield better results. To do so, we adopt our proposed compression scheme (from Sec. 5), em- ploying the same unconditional ImageNet DDM and using K = 4096 noises per codebook.
tradeoff via
k; = arg min MSE (IS((())) ( o0k-1) k.k,k,
)) + +XQ (x)0))))- (12)
where Q(-) can be any NR-IQA measure, even a non- differentiable one. In App. C.4 we explain our choice to set r(y) as an MMSE estimator.
We assess our approach choosing r(y) as the FFHQ (Karras et al., 2019) 512 x 512 approximate MMSE model trained by Yue & Loy (2024). We set X = 1 and optimize three different (-)) measures: NIQE, CLIP-IQA* (Wang et al., 2023a), and TOPIQ (Chen et al., 2024a) adapted for face im- ages by PyIQA (Chen & Mo, 2022). We utilize the FFHQ 512 x 512 DDM of Yue & Loy (2024) with T 1000 sampling steps and K 4096 for all codebooks. We compare our approach against the state-of-the-an methods PMRF (Ohayon et al., 2025). DifFace (Yue & Loy, 2024), and BFRffusion (Chen et al., 2024b), using the standard evaluation datasets CelebA-Tess (Karras et al., 2018; Wang et al., 2021), LFW-Test (Huang et al., 2008), WebPhoto- Test (Wang et al., 2021), )), and WIDER-TesT (Zhou al.,
Qualitative and quantitative results are reported in Fig. 6. trade-
As expected, due to the cate-perception-distore off (Blau& & Michacli, 2019). we observe that compressing the outputs of DPS and DDNM harms either their percep- tual quality (FID), or their distortion (PSNR), or both. This is while our method achieves superior perceptual quality compared to both DPS and DDNM, including their com- pressed versions. While our method achieves slightly worse PSNR, this is expected due to the perception-distortoon -istortiin trade- off (Blau& & Michaeli, 2018). See App. C.3 for more details.
7

---

Compressed Image Generation with Denoising Diffusion Codebook Models
Original
Degraded
DPS
Compr.
DDNM
Compr. DDNM
Colorization Suppe-rosoluttinn
Super-resolution
Colorization
Compreased DPS
Compressed DDNM
PENT
DDNM
24.4
14.3
Ours
22.5
8.0
42.7
46.0
49.2
32.4
47.4
48.9
86.1
87.0
FID
FID
Figure 6. Comparison of zero-shof posterior sampling image restoration methods. Our approach achieves better perceptual quality compared previous methods, while maintaining competitive PSNR and automatically producing compressed bit-stream representations for each restored image.
2022). We use PSNR to measure the distortion of the out- puts produced for the CelebA-Test dataset, and measure the ProxPSNR (Ohayon et al., 2025; Man et al., 2025) for the other datasets, which lack the clean original images. Per- ceptual quality is measured by NIQE, CLIP-IQA* TOPIQ- FACE, and additionally DDDINOV (Stein et al., 2023) to assess our generalization performance to a common quality measure which we do not directly optimize. Finally, as in Sec. 6.1, compress each evaluated method using our standard compression scheme, adopting the same FFHQ DDM with K 4096 and T 1000.
results. Besides image restoration, our compressed condi- tional generation framework can be used for any type of diffusion guidance, e.g., for text-conditional generation. We demonstrate this option in Apps. C.5 and C.6, introducing new classifier- based and classier-free guidance methods that do not use y for decompression. Moreover, we present in App. C.7 preliminary results for compressed image editing using DDCM, by decompressing an image using desired edit text prompt.
While our empirical results are encouraging, our work does not explain theoretically why DDCM sampling and our simple index selection strategies work so effectively. We encourage future works to investigate the principles behind the success of our methods. Moreover, when operating in latent space, our codec's performance is bounded by the underlying VAE, particularly at higher bit rates. Our results at higher bit rates could also be improved through better ap- proaches than our current matching pursuit inspired solution. Additionally, our compression efficiency could be improved through entropy coding of the selected indices, potentially reducing bit rates without sacrificing quality. Lastly, all DDCM-based solutions can be improved further by optimiz- ing the codebooks, e.g., through dictionary learning.
The results for the WIDER- Test dataset are reported in Fig. 7 (see App. C.4 for the other datasets). Our approach clearly optimizes each quality measure effectively and gen- eralizes well according to the FDDINO<2 scores. This is also confirmed visually, where all of our solutions produce high-quality images with less artifacts compared to previ- ous methods. While our approach shows slightly worse distortion, this once again expected due to the perception- distortion tradeoff (Blau & Michaeli, 2018).
7. Discussion
We introduced DDCM, novel generative approach for DDMs that produces high-quality image samples jointly with their lossless compressed bit-stream representations. We found that DDCM achieves comparable generative per- formance to DDPM, even when the codebooks are extremely small. We leveraged DDCM solve several compressed image generation tasks, including image compression and compressed restoration, where achieved fate-of-the-art
While DDCM produces compressed image representations in the form of indices, it is worth noting that the codebook entries can be interpreted as discrete image tokens, which are chosen autoregressively. This implies that DDCM can be interpreted an image tokenization method, forming "language" for images. Such perspective opens the door to intriguing possibilities, such using the tokenized
8

---

Compressed Image Generation with Denoising Diffusion Codebook Models
Input (real-world)
PMRF
DifFace
BFRffusion
Ours (TOPIQ)
Ours (NIQE)
Ours (CLIP-IQAT )
29.6
29.6
29.8
29.6
ProxfsS
27.4
27.4
27.4
27.4
26.3
26.3
26.3
26.3
0.68
0.80
3.30
4.46
5.04
0.63
0.68
572.64
619.50
666.37
713.23
TOPIQ
NIQB
CLIP-IQA
FDDINov2
Compresed
Compresed DifFace
Compresse BF Bifuussion
Ours(NIQR)
Qurs(TOPIQ)
(LL:CCII))))
PMRF
BF BFRffusion
Figure Comparing real-world face image restoration methods on the WIDER-Test dataset. successfully optimize the NR-IQA measures and produce appealing output perceptual quality with less artifacts compared to previous methods.
image representation as natural (ext-conditioning mech- anism for unconditional diffusion models. For instance, one could train transformer to predict the sequence of codebook indices corresponding to an image given its text description. Such a transformer can then be used to generate text-conditional images, by predicting the corresponding sequence DDCM codebook indices.
References
Agustsson, E. and Timofte, R. NTIRE 2017 challenge single image super-resolution: Dataset and study. In Proceedings of the IEEE conference on computer vision and pattern recognition workshops, pp. 126-135, 5
To conclude, our work demonstrates promising results across numerous tasks while providing opportunities for both theoretical analysis and practical improvements.
Anderson, B. D. Reverse-time diffusion equation models. Stochastic Processes and their Applica- tions, 12(3):313-326, 1982. ISSN 0304-4149, doi: http://ww.oo/////000000044111919 URL https://ww.scieneedeee pecenee/articee//////44419999999 3, 32
Acknowledgements
This research was partially supported by the Israel Science Foundation (ISF) under Grants 2318/22, 951/24 and 409/24, and by the Council for Higher Education Planning and Budgeting Committee. We thank Noam Elata and Matan Kleiner for assisting with our figures and experiments.
Ballé, J., Laparra, V., and Simoncelli, E. P. End-to-end optimized image compression. In 5th International Con- ference Learning Representations, ICLR 2017, Toulon, France, April 24-26, 2017, Conference Track Proceedings. OpenReview.net, 2017. URL https: : openrevien.eeee 2
Impact Statement
This paper presents work whose goal advance the field of Machine Learning. There are many potential societal consequences of our work, none which feel must be specifically highlighted here.
Bellard, F. BPG image format, 2018. URL https:// oallargdorgg/bg///
Binkowski,, M., Sutherland, D. J., Arbel, M., and Gretton, A. Demystifying MMD GANs. In International Conference on Learning Representations, 2018. URL https: // openreview net wtt.:/ewwee//oounteeeee forum?id=r1 1002000 5,

---

Compressed Image Generation with Denoising Diffusion Codebook Models
Blau, Y. and Michaeli, T. The perception-distortion tradeoff. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2018. 7, 8
Cohen, N., Manor, H., Bahat, Y., and Michaeli, T. From pos- terior sampling meaningful diversity in image restora- tion. In The Twelfth International Conference on Learn- ing Representations, 2024b. 2
Blau, Y. and Michaeli, T. Rethinking lossy compres- sion: The rate-distortion-pereee perceepionnn tradeoff. In Chaud- huri, K. and Salakhutdinov, R. (eds.), Proceedings of the 36th International Conference on Machine Learn- ing, volume 97 of Proceedings of Machine Learn- ing Research, pp. 675-685. PMLR, 09-15 Jun 2019. URL hwwph:ttp//.... http://oocedengg// wwweeeeiinnnnmiieeeee mlr...r..... wtt.ssm.......rrrrreeeeem press/v97/ blaul 1a. html. 5, 7, 44
Deng, J., Dong, W. Socher, R., Li, L.-J., Li, K., and Fei-Fei, L. Imagenet: A large-scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition, pp. 248-255, Ieee, 2009. 4, 4,5,4 5.40
Dhariwal, P. and Nichol, A. Diffusion models beat GANs on image synthesis. Advances neural information processing systems, 34:8780-8794, 2021. 3, 4, 40, 42
Careil, M., Muckley, M. J., Verbeek, J., and Lathuilière, S. Towards image compression with perfect realism at ultra- low bitrates. In The Twelfth International Conference on Learning Representations, 2023. 2,5
Elata, N., Michaeli, T., and Elad, M. Zero-shot image com- pression with diffusion-bart based posterior sampling. arXiv preprint arXiv:2407.0900 0.9999,, 2024. 1, 2, 5, 18
Chen, C. and Mo, J. IQA-Py'Torch: Pytorch toolbox for im- age quality assessment. [Online]. Available: https: /ttps:// //
Franzen, R. Kodak lossless true color image suite. source: http://rOk. us/graphics/kodak, 1999. 5
gi github..o. com/ http://ahoacccccccecoooooo IQA Sttpse//AAAAPPPoooooe PyTorch, 2022.
Freirich, D., Michaeli, T., and Meir, R. A theory of the distortionn-errcetionn tradeoff in wasserstein space. In Beygelzimer, A., Dauphin, Y.. Liang, P., and Vaughan, J. W. (eds.), Advances in Neural Information Processing Systems, 2021. URL https: : / oppnrevieww .et.eeeieeweeee/eeeeeeee forum?id=gee FNKC. 37
Chen, C., Mo, J., Hou, J., Wu, H., Liao, L., Sun, W., Yan, Q., and Lin, W. Topiq: A top-down approach from semantics to distortions for image quality assessment. IEEE Trans- actions on Image Processing. 33:2404-244 2418, 2024a. doi: 10.1 00.1009/TP.2222333334466
Chen, X., Fang, H., Lin, T.-Y., Vedantam, R., Gupta, S., Dollar, P., and Zitnick, C. L. Microsoft COCO captions: Data collection and evaluation server. arXiv, 2015. URL https: / arxiv. org/
Ganz, R., Kawar, B., and Elad, M. Do perceptually aligned gradients imply robustness? In Proceedings of 40th In- ternational Conference on Machine Learning, ICML'23. JMLR.org. 2023. 40
org/abs//////eoooeeo..goo..eesse abs 1504 00325. 4
Chen, X., Tan, J., Wang, T., Zhang, K., Luo, W., and Cao, X. Towards real-world blind face restoration with gen- erative diffusion prior. IEEE Transactions on Circuits and Systems for Video Technology, pp. 1-1, 2024b. doi: 10.11097CCSVT.2244.388838
Ghouse, N. F., Petersen, J., Wiggers, A., Xu, T., and Sautiere, G. residual diffusion model for high per- ceptual quality codec augmentation. arXiv preprint arXiv:2301.0544 2023. 2
7
Goodfellow, I., Pouget-Abadie J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., and Bengio, Y. Generative adversarial nets. In Ghahramani, Z., Welling, M., Cortes, C., Lawrence, N., and Wein- berger, K. (eds.), Advances in Neural Information Processing Systems, volume 27. Curran Associates, Inc.,
Chung, H., Kim, J., Mccann, M. T., Klasky, M. L., and Ye, J. C. Diffusion posterior sampling for general noisy in- verse problems. In The Eleventh International Conference on Learning Representations, 2023. URL https://///. // opennevieev.ee///orrmne htt.://oooruuuum-doo000000000000 1,
7,
2014. URL pttpe://prooeeediggs neurips. cc cc/paper cc/paper_files/ http:///eeeeaaae/e////////////// caaeebbl2261.eet00ssscchhaaaaaa2 pdf.2
Cohen, N., Kulikov, V., Kleiner, M., Spiieelln I., and Michaeli, T. Slicedit: Zero-shot video edit- ing with text-to-mmage diffusion models using spatio- temporal slices. In Salakhutdinov, R., Kolter, Z., Heller, K., Weller, A., Oliver, N., Scarlett, J., and Berkenkamp, F. (eds.), Proceedings of the 41st International Conference on Machine Learning, volume 235 of Proceedings of Machine Learning Research, pp. 9109-9119 9137. PMLR, 21-
Gu, Y., Wang, X., Xie, L., Dong, C., Li, G., Shan, Y.. and Cheng, M.-M. Vqfr: Blind face restoration with vector-quantized dictionary and parallel decoder. In Computer Vision ECCV 2022: 17th European Con- ference, Tel Aviv, Israel, October 23-27, 2022, Pro- ceedings, Part XVIII, pp. 126-143, Berlin, Heidelberg, 2022. Springer-Verlle Verlag. ISBN
27 Jul 2024a. URL https: htpp//poeedinng// mlr. press/ press/v235/ v235// 4 html.
doi:
10

---

Compressed Image Generation with Denoising Diffusion Codebook Models
10. 10.1007/978-3333119999----3 URL https: //doii /doi. org/10.1007 0t/5://0.100///77----011--993339
A., Dauphin, Y., Liang, P., and Vaughan, J. W. (eds.), Advances in Neural Information Processing Systems, volume 34, pp. 14938-14954, Curran Associates, Inc., 2021. URL htwps: /ttppe/e////occeengggs... neurips. cc wetpappr_fillllspppre//22222222
Hertz, A., Aberman, K., and Cohen-Or, D. Delta denoising score. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 45
2023a.
Paper.
pdf.
Hertz, A., Mokady, R., Tenenbaum, J., Aberman, K., Pritch, Y., and Cohen-or, D. Prompt-to-prone proo-rrrmpptttt image edit- ing with cross-attention control. In The Eleventh In- ternational Conference on Learning Representations, 2023b. URL htt.ttp:://pppeeeteeee wwwenrreiee.nee/Eooeee mettEoorm?? id=_CDi www.ciwww...b........ 45
Jiang, W. and Wang, R. MLIC++: Linear complex- ity multi-reference entropy modeling for learned image compression. In ICML 2023 Workshop Neural Com- pression: From Information Theory to Applications, id-hxIpeSoz2t. 2
2023. URL https: /pwpo/peeeeeeeeeeeeeeeeeeeeeeeee net/ forum?
Hessel, J., Holtzman, A., Forbes, M., Bras, R. L.,, and Choi,
Jiang, W. Yang. J., Zhai, Y., Ning, P., Gao, F., and Wang, R. MLIC: Multi-reference entropy model for learned image compression. In Proceedings of the 31st ACM International Conference on Multimedia, pp. 7618-7627, 2023. doi: 10.1 15/355817883.366 1691699.7 2
Y. CLIPScore: reference-free evaluation metric for image captioning. In EMNLP, 2021. 43
Heusel, M., Ramsauer, H., Unterthiner, T., Nessler, B., and Hochreiter, S. Gans trained by two time-scale update rule converge to local nash equilibrium. Advances in neural information processing systems, 30, 2017. 4
Kang, B., Tripathi, S., and Nguyen, T. Q. Toward joint im- age generation and compression using generative adver- sarial networks. arXiv, 2019. URL https://///// //arxiv.. org/ abs///////// / 1101999111111 07838.
Ho, J. and Salimans, T. Classifier-free diffusion guidance. In NeurIPS 2021 Workshop on Deep Generative Models and Downstream Applications, 2021. 4, 40, 42
Karras, T., Aila, T., Laine, S., and Lehtinen, J. Progres- sive growing of GANs for improved quality, stability, and variation. In International Conference on Learning Representations, 2018. URL https: /ownneveee...eeeee net/ forum? id= 92CeAb. 7
Ho, J., Jain, A., and Abbeel, P. Denoising diffusion proba- bilistic models. In Larochelle, H., Ranzato, M., Hadsell, R., Balcan, M., and Lin, H. (eds.), Advances in Neural Information Processing Systems, volume 33, pp. 6840- 6851. Curran Associates, Inc., 2020. 1, 3, 32,
Karras, T., Laine, S., and Aila, T. A style-based generator architecture for generative adversarial networks. In 2019 IEEF/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 4396-4405, 2019. doi: 10.1109) PPRR20019.00453.0
Huang, G. B., Mattar, M., Berg, T., and Learned-Mille E. Labeled Faces in the Wild: A Database forStudy- ing Face Recognition in Unconstrained Environments. In Workshop on Faces in 'Rfel-Life' Images: Detec- tion, Alignment, and Recognition, Marseille, France, October 2008. Erik Mouned-Millee and Andras Fer- encz and Frédéric Jurie. URL https: inria hal. science/ http://nrraa0001922333
Kawar, B., Elad, M., Ermon, S., and Song, J. Denoising diffusion restoration models. In Koyejo, S., Mohamed, S., Agarwal, A., Belgrave, D., Cho, K., and Oh, A. (eds.), Advances in Neural Information Processing Systems, vol- ume 35, pp. 23593-23606. Curran Associates, Inc., 2022. 1, 1, 7
Sobberman-SSiegelggle I., Kulikov, V., and Michaeli, T. An edit friendly DDPM noise space: Inversion and manip- ulations. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 12469- 12478, 2024. 2, 45
Körber, N., Kromer, E., Siebert, A., Hauke, S., Mueller- Gritschneder, D., and Schuller, B. PerCo (SD): Open perceptual compression. In Workshop on Machine Learn- ing and Compression, NewrlPS 2024, 2024. 2.
Iwai, S., Miyazaki, T., and Omachi, S. Controlling rate, distortion, and realism: Towards a single comprehensive neural image compression model. In Proceedings of the IEEE/CVF Winter Conference on Applications of Com-
Li, J., Li, D.,, Savarese, S., and Hoi, Blip-2: Bootstrapping language- pre-training with frozen image encoders and large language models. In International conference machine learning, pp. 19730-19742. PMLR, 2023. 19
puter Vision, pp. 2900-2909, 2024. 2,
Jalal, A., Arvinte, M., Daras, G., Price, E., Dimakis, A. G., and Tamir, J. Robust compressed sensing MRI with deep generative priors. In Ranzato, M., Beygelzimer,
Lin, T.-Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., and Zitnick, C. L. Microsoft

---

7,8 Compressed Image Generation with Denoising Diffusion Codebook Models
COCO: Common objects in context. In Computer Vision- ECCV 2014: 13th European Conference, Zurich, Switzer- land, September 6-12, 2014, Proceedings, Part V 13, pp. 740-755. Springer. 2014. 4
Miyasawa, K. al. An empirical bayes estimator of the mean of a normal population. Bull. Inst. Internat. Statist, 38(181-188)112, 1961. 3
Muckley, M. J., El-Nouby, A.. Ullrich, K., Jégou, H., and Verbeek, J. Improving statistical fidelity for neural im- age compression with implicit local likelihood models. In International Conference on Machine Learning, pp. 25426-25443. PMLR, 2023. 2.5
Lin, X., He, J., Chen, Z., Lyu, Z., Dai, B., Yu, F. Qiao, Y., Ouyang. W., and Dong. Difibir: Toward blind image restoration with generative diffusion prior. In European Conference on Computer Vision, pp. 430-448. Springer, 2024. 2,7
Obukhov, A., Seitzer, M., Wu, P.-W., Zhydenko, S., Kyl, J., and Lin, E. Y.J. High-fidelity performance metrics for generative models in pytorch, 2020. URL https: / //github. cww. www.c/ot./tttsscc sion: 0.3.0, DOI: 1..228//zeoodo.495573... 15, 18
Liu, H., Zhang, G., Chen, J., and Khisti, A. J. Lossy com- pression with distribution shift as entropy constrained optimal transport. In International Conference on Learn- ing Representations, 2021. 2
Ver-
Lugmayr, A., Danelljan, M., Romero, A., Yu, F,, Timofte, R., and Van Gool, L. Repaint: Inpainting using denoising diffusion probabilistic models. In Proceedings of the IEEE/CVF conference on computer vision and paltern recognition, pp. 11461-11471 2022. 2
Ohayon, G., Adrai, J., Elad, M., and Michaeli, T. Rea- sons for the superiority of stochastic estimators over deterministic ones: Robustness, consistency and per- ceptual quality. In Krause, A., Brunskill, E., Cho, K., Engelhardt, B., Sabato, S., and Scarlett, J. (eds.), Pro- ceedings of the 40th International Conference on Mu- chine Learning, volume 202 of Proceedings of Machine Learning Research, pp. 26474-26494, PMLR, 23-29 Jul 2023. URL.ttps:///prt httpp/ppoceeen...eeee/ee/ mlr press/ v202 chayon223aaa html.
Mallat, S. G. and Zhang, Z. Matching pursuits with time- frequency dictionaries. IEEE Transactions on signal processing, 41(12):3397-3111 1993. 4
Man, S., Ohayon, G., Raphaeli, R., and Elad, M. Proxies for distortion and consistency with applications for real- world image restoration. arXiv, 2025. URL https: //arxiv..or// abs 20t.. 12102. 2
Ohayon, G., Michaeli, T., and Elad, M. Posterior-mean recti- fied flow: Towards minimum MSE photo-realistic image restoration. In The Thirteenth International Conference on Learning Representations, 2025. URL https:// openreview net/ / wowrmmidd-hot.yyye
Manor, H. and Michaeli, T. Zero-shot unsupervised and text-based audio editing using DDPM inversion. In Salakhutdinov, R., Kolter, 7., Heller, K., Weller, A., Oliver, N., Scarlett, J., and Berkenkamp, F. (eds.), Pro- ceedings of the 41st International Conference on Ma- chine Learning, volume 235 Proceedings of Machine Learning Research, pp. 14603-34629 PMLR, 21-27 Jul 2024. URL http:///pooeeeigeeeeeeeeee ngs mlr. prress/ v235 01/55maaoos htm/mmano22aa. htm...1..................... 2, 45
Oquab, M., Darcet, T., Moutakanni, T., Vo, H. V., Szafraniec, M., Khalidov, V., Fernandez, P., HAZIZA, D., Massa, F,, El-Nouby, A., Assran, M., Ballas, N., Galuba, W., Howes, R., Huang, P.-Y., Li, S.- Misra, I., Rabbat, M., Sharma, V., Synnaeve, G., Xu, H., Je- gou. H., Mairal, J., Labatut, P., Joulin, A., and Bo- janowski, P. DINOv2: Learning robust visual features without supervision. Trunsuctions on Machine Learn- ing Research, 2024. ISSN 2835-8856. URL https: openreview net / forum?id=a68SUe 6zFt. Fea- tured Certification. 15
Meng, C., He, Y., Song, Y., Song, J., Wu, J., Zhu, J.- Y., and Ermon, S. SDEdit: Guided image synthesis and editing with stochastic differential equations. In International Conference on Learning Representations, 2022. URL https:ttp........t. www.oopeevveeeeee id= IddaBssCjPu_Lt t3. 1,2, 45
Pan, X., Zhan, X., Dai, B., Lin, D.,, Loy, C. C., and Luo, P. Exploiting deep generative prior for versatile image restoration and manipulation. In European Conference Computer Vision (ECCV), 2020. 5
Mentzer, F., Toderici, G. D., Tschannen, M., and Agustsson, E. High-fidelity generative image compression. Advances in Neural Information Processing Systems, 33:11913- 11924, 2020. 2.5
Radford, A., Kim, J. W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., Krueger, G., and Sutskever, I. Learning transferable visual models from natural language su- pervision. In Meila, M. and Zhang, T. (eds.), Pro- ceedings of the 38th International Conference on Ma- chine Learning, volume 139 of Proceedings of Machine
Mittal, A., Soundararajan, R., and Bovik, A. C. Making a "completely blind" image quality analyzer. IEEE Signal Processing Letters, 20(3):209-212 2013, doi: 10./ LSP.2012.2227226.
12

---

Compressed Image Generation with Denoising Diffusion Codebook Models
Learning Research, pp. 8748-8763. PMLR, 18-24 Jul 2021. URL http:://///oeeeeeeee ing..ngs......... mlr press// 1.399radford2la.ttmm 43
Song, Y. and Ermon, S. Generative modeling by estimating gradients of the data distribution. In Wallach, H., Larochelle, H., Beygelzimer, A., d'Alché-Buc, F., Fox, E., and Garnett, R. (eds.), Advances in Neural Information Processing Systems, volume 32. Curran Associates, Inc., 2019. URL https:: cc cc/ssppet_fileleppppr///////1111 http://pproeeddnngg.eeee...... 201le225747766333744566666661112 pdf.
Rajesh, B., Dusa, N., Javed, M., Dubey, R., and Nagab- hushan, P. T2ci-gan: Text compressed image genera- tion using generative adversarial network. In Gupta, D.,
Bhurchandi, K., Murala, S., Raman, B., and Kumar, S. (eds.), Computer Vision and Image Processing, pp. 292- 307, Cham, 2023. Springer Nature Switzerland. ISBN 978-3-031-31417-9) 2,6
Song, Y., Sohl-Dickstein, J., Kingma, D. P., Kumar, A., Er- mon, S., and Poole, B. Score-based generative modeling through stochastic differential equations. In International Conference Learning Representations, 2020. 2, 4, 32, 33
Ramesh, A., Pavlov, M., Goh, G., Gray, S., Voss, C., Rad- ford, A., Chen, M., and Sutskever, I. Zero-shot text-to- image generation. In International conference on ma- chine learning, pp. 8821-8831. Pmlr, 2021.
Stein, C. M. Estimation of the mean of multivariate normal distribution. The Annals of Statistics, 135-1135-1151, 1981. 3
Raphaeli, R., Man, S., and Elad, M. SILO: Solving in- verse problems with latent operators. arXiv, 2025. URL ht https:: /tatpp://axiiiv.i aaaii.iiiive//.. http://aaxxxvvx../////. wttpoo/g/////ooo 11746..
Stein, G., Cresswell, J. C., Hosseinzadeh, R., Sui, Y., Ross, L., Villecroze, V., Liu, Z., Caterini, A. L., Taylor, E., and Loaiza-Ganem, G. Exposing flaws of generative model evaluation metrics and their unfair treatment of dif- fusion models. In Thirth-seventh Conference on Neural Information Processing Systems, 2023. URL https: openreview net. forum? f 8,
Robbins, H. An empirical bayes approach to statistics. In
Proceedings of the Third Berkeley Symposium on Mathe- matical Statistics and Probability, 1954-1955, volume 1, pp. 157-163. Berkeley and Los Angeles: University of California Press, 1956. 3
Rombach, R., Blattmann, A., Lorenz, D., Esser, P., and Ommer, B. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF con- ference on computer vision and pattern recognition, pp. 10684-10695, 2022. 3, 4,
Szegedy, C., Zaremba, W., Sutskever, I., Bruna, J., Erhan, D., Goodfellow, I. J., and Fergus, R. Intriguing prop- erties neural networks. In Bengio, Y. and LeCun, Y. (eds.), 2nd International Conference on Learning Rep- resentations, ICLR 2014, Banff. AB, Canada, April 14- 16. 2014, Conference Track Proceedings, 2014. URL http:/ /tt///axxxa /tt.//rxivv///////a/ .rgg abs 1312. 6199. 40
Saharia, C., Chan, W., Saxena, S., Li. L.. Whang, J., Denton, E. L.. Ghasemipour, K., Gontijo Lopes, R., Karagol Ayan, B., Salimans. T., et al. Photorealistic text-to-ing image dif- fusion models with deep language understanding. Ad- vances in neural information processing systems, 35: 36479-36494 2022.
Theis, L. and Ahmed, N. Y. Algorithms for the communi- cation of samples. In Chaudhuri, K., Jegelka, S., Song, I.., Szepesverii, C., Niu, G., and Sabato, S. (eds.), Pro- ceedings of the 39th International Conference chine Learning. volume 162 of Proceedings of Machine Learning Research, pp. 21308-21328, PMLR, 17-23 Jul 2022. URL http:///ooeeeeeeeee/ee mlr. .tt.inngssm...rrrss/..... v1 wtt22thhis22a.... html.
Sohl-Dickstein, J., Weiss, E., Maheswaranathan, N., and Ganguli, S. Deep unsupervised learning using nonequi- librium thermodynamics. International conference on machine learning, pp. 2256-2265 PMLR, 2015. 1,3
Ma-
Song, J., Meng, C., and Ermon, S. Denoising diffusion implicit models. In International Conference on Learning Representationss 2021. URL https: openreview. net/
Theis, L., Shi, W., Cunningham, A., and Huszár, F. Lossy image compression with compressive autoencoders. In International Conference on Learning Representations, /ttt:s:://peerevviee wwtpr/veeneeeeeeeeeeeeeeeeeeeeep net/ forum?
45
2017. URL httph:ttp///cc
Song. J., Vahdat, A., Mardani, M., and Kautz, J. secudoinverse-guidee diffusion models for inverse prob- lems. In International Conference on Learning Represen- tations, 2023. URL https:/ http///ppppeeeeeee////e/// / forum? d= ==9_gsMMBMRKKG 2
i id-rJiNwv9gg. 2
Theis, L., Salimans, T., Hoffman, M. D., and Mentzer, F. Lossy compression with gaussian diffusion. arXiy preprint arXiv:2206. 08888, 2022. 1.2
13

---

Compressed Image Generation with Denoising Diffusion Codebook Models
Toderici, G., Shi, W., Timofte, R., Theis, L.,, Balle, J., Agustsson, E., Johnston, N., and Mentzer, F. Workshop and challenge on learned image com- pression (CLIC2020), 2020. URL http://www. compression www.ssiiincccooooo cc
Wu, C. H. and De la Torre, F. A latent space of stochastic diffusion models for zero-shot image editing and guid- ance. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 7378-7387, 2023. 45
Tsipras, D., Santurkar, S., Engstrom, L., Tumner A., and Madry, A. Robustness may be at odds with accuracy. In International Conference on Learning Representations, 2019. URL https:/ wwwpoeeeewe.eeeeoe.e..ee id=SyxAb30cY7, 40
Wu, X., Hu, J., and Wei, X. RDPM: Solve diffusion prob- abilistic models via recurrent token prediction. arXiv preprint arXiv:2412. 18390, 2024. 2
Xu, T., Zhu, Z., He, D., Li, Y., Guo, L., Wang, Y., Wang, Z.. Qin, H.. Wang, Y., Liu, J., and Zhang, Y.-Q. Idempo- tence and perceptual image compression. In The Twelfth International Conference on Learning Representations, 2024. URL https:/ /ttp///peeee.eeeeeeeeeee net forum?
Tumanyan, N., Geyer, M., Bagon. S., and Dekel, T. Plug- and-play diffusion features for text-driven image-to- image translation. In Proceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition, pp- 1921-1930, 2023. 45
id=Cy5v64DqEF. 5
Yang. L. Discrete distribution networks. In The Thirteenth International Conference on Learning Representations, 2025. URL https:/ /pwpneveeewewweeeeoe/ee/eeroreee net forum?
Vincent, P. A connection between score matching and de- noising autoencoders. Neural Computation, 23(7):1661- 1674, 2011. doi: 10.1162/NECO a a.0014.. 3
id=xNaIfzleff. 2
Yang. R. and Mandt, S. Lossy image compression with conditional diffusion models. In Oh, A., Naumann, T., Globerson, A., Saenko, K., Hardt, M., and Levine, S. (eds.), Advances in Neural Information Processing Sys- tems, volume 36, pp. 64971-64995. Curran Associates, Inc., 2023. 2
Wallace, B., Gokul, A., and Naik, N. Edict: Exact diffusion inversion via coupled transformations. In Proceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recognition, pp. 22532-22541, 2023. 45
Wang, J., Chan, K. C.,, and Loy, C. C. Exploring clip for assessing the look and feel of images. Proceedings of the AAAI Conference on Artificial Intelligence, 37 (2):2555-2563, Jun. 2023a. doi: 10.1609/aaii.v27 v3772. 25353. URL https://ojsss.oo. aaai. org/ index. 7
Yue, Z. and Loy, C. Difface: Blind face restoration with diffused error contraction. IEEE Transactions on Pattern Analysis and Machine Intelligence, pp. 1-15, 2024. doi: 10. 109/19999IIIMMM 20w..M....24.4433323 2.
Zhang, R., Isola, P., Efros, A. A., Shechtman, E., and Wang, O. The unreasonable effectiveness of deep features as a perceptual metric. In CVPR, 2018. 5
Wang, X., Li, Y., Zhang, H., and Shan, Y. Towards real- world blind face restoration with generative facial prior. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2021.7 7
Zhou, S., Chan, K. C., Li, C., and Loy, C. C. Towards robust blind face restoration with codebook lookup transformer. In NeurlPS, 2022.
Wang, Y., Yu, J., and Zhang, J. Zero-shot image restora- tion using denoising diffusion null-space model. In The Eleventh International Conference on Learning Represen- tations, 2023b. URL https:/ // owwnreview..c. nett forum? i d=mR i eQgMtNTQ. 2, 7,
Zhu, X., Song, J., Gao, L., Zheng, F., and Shen, H. T. Uni- fied multivariate gaussian mixture for efficient neural image compression. In Proceedings of the IEEF/CVF Conference on Computer Vision and Pattern Recognition, pp. 17612-17621, 2022.
Wang, Z., Zhang, J., Chen, R., Wang, W., and Luo, P. Re- storeformer: High-quality blind face restoration from undegraded key-value pairs. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 17521,, 2022. 7
Wang, Z., Zhang, J., Chen, T., Wang, W., and Luo, P. Re- storeformer++: Towards real-world blind face restoration from undegraded kal-value pairs. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2023c.
14

---

Compressed Image Generation with Denoising Diffusion Codebook Models
A. Additional DDCM Evaluation
Figure provides additional quantitative comparisons between DDPM and DDCM, using different K values. Specifically, we compute the Kernel Inception Distance (KID) (Binkowski et al., 2018). as well as the Fréchet Distance and Kernel Distance evaluated in the feature space of DINOv2 (Stein et al., 2023; Oquab et al., 2024). These quantitative results remain consistent with the ones presented in Sec. 4, showing that DDCM with small K values is comparable with DDPM. Figures 9 and 10 show numerous outputs from both DDPM and DDCM with small values of K, demonstrating the sample quality and diversity produced by the latter for such K values. We use Torch Fidelity (Obukhov et al., 2020) to compute the perceptual quality measures.
Pixel Space
Latent Space (MS-COCO)
(ImageNett)
x s02
x10
+KDD >DDDINO2
2.2
2.1
1.7
x10
100
3.25
3.00
2.75
x10
x10
J. KDDINNv2
1.2
1.0
4
16
2
8
16
64
K
DDPM
- DDCM
Figure 8. Comparing DDPM with DDCM for different codebook sizes K. As in Sec. 4, DDPM and DDCM with K = 64 (sometimes K 16) achieve similar generative performance, suggesting that the continuous representation space of DDPM (DDCM with K = (o) is highly redundant. We use class-conditions ImageNet model (256 x 256) for pixel space, and the text-conditional SD 2.1 model (768 768) for latent space. The K axis is in log-scale.
15

---

Compressed Image Generation with Denoising Diffusion Codebook Models
cat sitting under an umbrella near yellow box.
DDCM (K'
DDCM (K 4)
9
DDPM
smiling male tennis player swinging racket
DDCM (X
DDCM K 4)
DDPM
Figure Qualitative comparison of sample quality and diversity between DDCM and DDPM. We generate multiple samples for each prompt, using the 768 x 768 SD 2.1 model.
16

---

Compressed Image Generation with Denoising Diffusion Codebook Models
Lorikeet
DDCM (IC =
DDCM RK 4)
Monarch butterfly
DDCM (K 2)
DDCNA (K
DDPM
Figure 10. Qualitative comparison of sample quality and diversity between DDCM and DDPM. We generate multiple samples for each class, using the conditional 256 x 256 ImageNet model.
17

---

Compressed Image Generation with Denoising Diffusion Codebook Models
B. Image Compression Supplementary
We compute all distortion and perceptual quality measures using Torch Metries (which relies on Torch Fidelity (Obukhov al., 2020)).
B.1. Experiment Configurations
We specify here the different configurations used for the compression experiments in Sec. 5.
In 256 256 experiments use the 256x256 di diffusion wwwusidonuunond...mm. pt checkpoint from the official GitHub repository. In our 768 768 and 512 512 experiments we use Stable Diffusion 2.1 with the s wtwpttttaatttaalddleiieeee checkpoints from Hugging Face, respectively. The different K, M, C and T values for each of the 512 x 512 and the 768 x 768 experiments plotted in Figs. and are summarized in Tab. 1.
and stabilityals
official
Table Image compression experiments contigurations.
Model
Image Resolution
T
K
M
C
1000
64, 128, 256, 4096 2048
1000
2, 3, 5
Pixel Space DDM
256
500
128, 512
300
16, 32, 128, 512
1000
256,, 1024, 8192
512x512
1000
2048
2, 3, 6
3
1000
16, 32, 64, 256, 1024, 8192
Latent Space DDM
1000
2048
2, 3,
x68x768
500
16, 32, 64, 256, 1024, 8192
Adapted 500 (App. B.4)
16, 32, 64, 256, 1024, 8192
For PSC-D and PSC-R we use the same pre-trained ImageNet 256 256 model as ours the 256 x 256 experiments, and the same Stable Diffusion 2.1 model in the 512 512 experiments. We adopt the default hyper-parameters of the method described by the authors (Elata et al., 2024), setting the number of measurements to 12 2 for i 0, 8.
For IPIC we adopt the official implementation using the ELIC codec with five bit rates, combined with DPS sampling for decoding with T = 1000 steps and Ç {0.3, 0.6, 0.6, 1.2, 1.6), recommended by the authors.
For BPG we considered quality factors g {51, 50, 48, 46, 42, 40, 38, 36, 34, 32, 30).
For HiFiC we test the low, medium and high quality regimes, using the checkpoints available in the official GitHub repository.
PerCo (SD) is tested using the three publicly available Stable Diffusion 2.1 fine-tuned checkpoints from their Official GitHub repository, using the default haper-parameters.
For ILLM we use the MS-ILLM pre-trained models available in the official GitHub implementation. For the 512 x 512 image size experiments we use ms www.qualimty....... x
For the 768 768 image size experiments we use 2,3 3 and ms imm.llm.a.. quality vloY, 1,2.
ms 1m quali ty
CRDR-D and CRDR-R are evaluated using quality factors of 2,,1, 4), where CRDR-D uses and CRDR-R / 3.84, recommended in the paper.
B.2. Additional Evaluations
In Figs. 11, 12, 13 and 14 we provide additional qualitative comparisons on the Kodak24 (512 x 512) and ImageNet (256 x 256) datasets. We additionally compare our method on images size 768 x 768, and present the results in Fig. 15. Our method is implemented as before, while using a Stable Diffusion 2.1 model trained on the appropriate image size (see App. B.
18

---

Compressed Image Generation with Denoising Diffusion Codebook Models
B.3. Numerical Results
Tabs. 2, 3, and include the numerical results that appear in Fig. 5. Additionally, Tabs, and 7 include the numerical results for Fig. 15.
B.4. Decreasing the Bit Rate via Timestep Sub-Sampling
As mentioned in Sec. 5, decreasing the bit rate of our compression scheme can be accomplished in two ways. The first option is reduce K, which sets the number of bits required to represent each communicated codebook index. The second option relates to the number of generation timesteps, which sets the total number of communicated indices. Specifically, DDMs trained for T = 1000 steps can still be used to generate samples with fewer steps, by skipping alternating timesteps and modifying the variance in Eq. (2). Thus, we leverage such timestep sub-sampling in DDCM to shorten the compressed bit-stream. We find that pixel space DDMs yields good results with this approach, while the latent space models struggle to produce satisfying perceptual quality.
Thus, for latent space models we propose a slightly different timestep sub-sampling scheme. Specifically, we keep T 1000 sampling steps inference and set different K values for different subsets of timesteps. We choose K for subset of L sampling steps, and K > 1 for the rest T - L steps. Thus, our compression scheme only optimizes T L steps and necessitates transmitting only T - L indices. The rest L indices correspond to codebooks that contain only one vector, and thus do not affect the bit rate.
We use T = 1000, set the same codebook size K > for every timestep E {899.... 400), and use K for all other steps. We compare our proposed method against the aforementioned naive timestep skipping approach with 500 sampling steps and the same K > which attains the same bit rate our proposed alternative.
Quantitative results are shown in Fig. 15, where our timestep adapted method is denoted by Ours Adapted. Our adapted approach achieves better perceptual quality compared to the naive one, at the expense of slightly hindered PSNR.
B.5. Inereasing the Bit Rate via Matching Pursuit
In Sec. 5 we briefly cxplain how to achieve higher bit rates with our method, by refining each selected noise via matching pursuit inspired solution. Formally, at each timestep i we iteratively refine the selected noise by linearly combining it with
M - 1 other noises from the codebook. We start by picking the first noise index k; according to Eq. (7). Then, we set
R(1) k, 7/ 1, and x(4) (ka), and pick the next indices and coefficients (m 1,..., M 1) via
1mm+1), (m++1)
ke(II....K,, 7ET (7 + + -1-18 +11-7) xo -X01s)
Pi
arg max
(13)
The noise vector 2! =(m+1) 1) is then updated via
(m+ &- 7/4
(m++)) z(m)) + +(1+++1111 (11- Ci (n)m+t))).
(14)
z(m++1) 1)
(m+1)
(15)
std s(m++1))
where std(2) is the empirical standard deviation of the vector 2. We the resulting vector 2! (M) the noise in Eq. (6) to produce the next x,-1, and repeat the above process iteratively. Note that setting M = 1 is equivalent to our standard compression scheme.
In our experiments, the set of coefficients I is a subset of (0. 1]. containing C= = [T] values that are evenly spaced in this range. We pick C' = and assess M (2,3,6) for the latent space model experiments, and M e (2, 4,5 for the pixel space model experiments.
B.6. Assessing the Effectiveness of Text Prompts in Compression using Text-to-Image Latent Space DDMs
Stable Diffusion 2.1 is a text-to-imen -mage generative model, which both PerCo (SD) and PSC leverage for their compression approach. Specifically, both of these methods start by generating a textual caption for every target image using BLIP-2 (Li
19

---

Compressed Image Generation with Denoising Diffusion Codebook Models
al., 2023), and feed the captions prompts the SD model. In case, find that using such prompts hinders the compression quality. Specifically, follow the same automatic captioning procedure in PerCo (SD) and PSC, using the Sa Salesforce "bbllppp plipp opt--pt----- 2. 7w--w---- coco checkpoint of BLIP-2 from Hugging Face. We then continue with our standard compression approach, where the denoiser is used with standard classifier-free guidance (CFG). Note that using text prompts requires transmitting additional bits that serve as a compressed version of the text. Specifically, we use BIIP-2 with maximum = 32 word tokens, each picked from a dictionary containing a total 30,524 words. Thus, at most 32 (lgol(30222 (30524) R 480 bits are added to the bit-stream in our method.
We assess this text-conditional approach on the 512 x 512 SD 2.1 DDM, using CFG scales of 3, We compare the performance of this conditional approach with that of the unconditional one used in Sec. 5. The results in Fig. 16 show disadvantage for using text-prompts for compression with our method.
20

---

Compressed Image Generation with Denoising Diffusion Codebook Models
Original
BPG
PSC-P
PerCo
Ours 256)
0.035 BPP
0.049 BPP
0.031 BPP
0.030
0.016 BPP
0.049 BPP
0.031 BPP
0.030
0.036 BPP
0.049 BPP
0.031 BPP
0.030 BPP
MID
0.084 BPP
D 0499 BPP
0,031 BPP
0.030 BPP
0.049 BPP
0.031 BPP
0.030 BPP
0.044 BPP
O. 0.049 BPP
O. 0.031 BPP
0.030 BPP
0.033 BPP
(.049 BPP
0.031 HPP
0.030 HPP
Figure 11. Qualitative extreme image compression results. The presented images taken from the Kodak24 dataset, cropped
512 x 512 pixels. Our compression scheme produces highly realistic decompressed outputs, while maintaining better fidelity to the original images compared previous methods.
21

---

Compressed Image Generation with Denoising Diffusion Codebook Models
Original
PSC-P
ILLM
PerCo
CRDR-R
Ours (K 2018,
AJ095 BPP
0.085 BPP
OUT125 BPP
0.102 BPP
0.095 BPP
0.095 BPP
0.085 BPP
0.125 BPP
0:118 BPP
0.095 BPP
0.095 BPP
0.085 BPP
0.125 BPP
0.094 BPP
0.095 BPP
0.095 BPP
0.085 BPP
0.125 BPP
0.070 BPP
0.095 BPP
Figure 12. Qualitative image compression results. The presented images are taken from the Kodak24 dataset, cropped to 512 x 512 pixels. Our compression scheme produces highly realistic decompressed outputs, while maintaining better fidelity the original images compared to previous methods.
22

---

Compressed Image Generation with Denoising Diffusion Codebook Models
Original
PSC-D
PSC-P
IPIC
Ours (K 16)
0.077 BPP
0.077 BPP
0.134 BPP
0.061 BPP
0.077 BPP
0.077 BPP
0.180 BPP
0.061 BPP
m
I It!!!!
0.077 BPP
0.077 BPP
0.120 BPP
01061 BPP.
0.077 BPP
0.077 BPP
0:102 BPP
0.061 BPP
0.077 BPP
0.077 BPP
0.053 BPP
0.061 BPP
Figure 13. Qualitative image compression results. the presented images are taken from the ImageNet 256 x 256 dataset. Compared to previous methods, compression scheme produces higher perceptual quality and better fidelity to the original images.
23

---

Compressed Image Generation with Denoising Diffusion Codebook Models
Original
PSC-D
PSC-P
IPIC
HFFI
Ours (K 512)
0.155 BPP
0.155 BPP
0.373 BPP
0.264 BPP
0.137 BPP
0.155 BPP
0.155 BPP
0.451 BPP
0.324 BPP
0.137 BPP
0.155 BPP
0.155 BPP
0.276 BPP
0.263 BPP
0.133 BPP
0.555 BPP
0.155 BPP
2,227 HPP
260 HPP
0.7 HPP
0.155 BPP
0.155 BPP
0.118 BPP
0.150 BPP
0.137 BPP
Figure 14. Qualitative image compression results. the presented images taken from the ImageNet 256 x 256 dataset. Compared previous methods, our compression scheme produces higher perceptual quality and better fidelity to the original images.
24

---

Compressed Image Generation with Denoising Diffusion Codebook Models
PerCo (SD)
Eacoder-Deodee
HIFIC
Our 2018,
Adapled,
CLIC2020rGs
D1V2K768
30
PSNR
10
10-3
10
10
-LIPS
0.2
10-2
10-2
10a
FID
10a
10-2
Bits per pixel
Bits pixel
Figure 15. Quantitative image compression results 768 768 sized images. At higher bit rates, method achieves the lowest
(best) FID scores in both datasets while maintaining better distortion metrics compared to PerCo (SD). At extremely low bit rates, while PerCo (SD) shows marginally better FID scores, method attains superior PSNR performance.
Without Prompt
BLIP, CFG=3.0
BLIP, CFG=6.0
Kodaks12
CLICs12
DIV2Ks12
23
20.5
PSNR
23
20.0
22
19.5
22
3 10
x 10-2
x 10-2
3 10-2
4 10-2
5x 10-2
3 10-2
4 10-2
5x 10-2
LPIPS
0.25
0.20
0.25
0.20
0.15
0.20
10-2
10
5 10
10
10
10-2
10-2
10
5 10-2
KIDe4
0.0025
0.0050
0.004
0.0025
0.0015
0.002
3 x 10
4 10-2 x 10
3 10
10
5x 10
10
x 10
5 x 10
Bits per pixe!
Bits pixel
Bits per pixel
Figure 16. Evaluating the effectiveness of using text prompts in image compression. We evaluate unconditional compression method with the text-conditional one, while using the text captions generated by BLIP-2, We find that using such text prompts hinders our compression results, both in terms of perceptual quality and distortion.
25

---

Compressed Image Generation with Denoising Diffusion Codebook Models
Table Compression quantitative evaluations, for the ImageNet 256 x 256 dataset.
Method
BPP
FID
LPIPS
PSNR
0.056
144.322
0,460
23.771
0.118
111.821
0.330
26.085
BPG
0.248
81.555
0.218
28.329
0.503
60.512
0.127
30.864
0.941
42.714
0.064
33.602
0.220
42.610
0.055
27.713
HiFiC
0.395
31.567
0.033
29.849
0.584
25.125
0.021
31.622
0.095
45.367
0.211
24.470
0.148
38.413
0.152
26.011
IPIC
0.228
36.301
0.132
27.664
0.550
27.075
0.074
31.178
0.945
20.689
0.044
33,800
0,019
147.317
0.591
21.648
0.039
127.275
0.496
23.572
PSC-D
0.077
105.225
0.404
25.605
0.155
86.787
0.316
27.711
0.019
67.457
0.431
18.575
0.039
59.119
0.353
20.178
PSC-P
0.077
51.464
0.268
22.649
0.155
44.589
0.194
25.107
0.018
68.339
0.419
19.624
0,023
63.731
0.365
20.566
0.032
57.027
0.296
21.877
0.041
52.157
0.251
22.795
0.053
47.834
0.220
23.401
0.069
43.972
0.185
24.199
Ours
0.091
40.787
0.165
24.643
0.122
36.890
0.134
25.587
0.183
31.977
0.102
26.765
0.381
28.247
0.076
27.949
0.594
25.705
0.063
28.694
0.808
24.165
0.055
29.211
1.021
22.938
0.050
29.596
26

---

Compressed Image Generation with Denoising Diffusion Codebook Models
Table 3. Compression quantitative evaluations, for the Kodak24
512
512 dataset.
Method
BPP
FID
LPIPS
PSNR
0.037
148.306
0.540
24.211
0.044
141.575
0.513
24.638
0.062
128.095
0.461
25.539
BPG
0.088
114.256
0.407
26.492
0.121
104.289
0.357
27.375
0.167
92.423
0.307
28.346
0.227
82.184
0.260
29.359
0.114
94.475
0.257
27.988
CRDR-D
0,210
76.015
0.167
30.069
0.114
35.676
0.094
27.315
CRDR-R
0.210
28.391
0.057
29.373
0.191
31.713
0,068
27.376
HiFiC
0.363
25.734
0.042
29.466
0.085
36.080
0.110
25.683
ILLM
0.159
28.556
0,072
27.296
0.304
24.448
0.044
29.395
0.025
47.334
0.473
18.516
PSC-P
0.049
40.691
0.371
20.080
0.095
35.314
0.266
21.683
0.033
37.019
0.307
19.017
PerCo (SD)
0.127
26.418
0.145
22.325
0.030
32.031
0.222
22.066
0.038
29.117
0.190
22.551
0.050
25.647
0.161
23.013
Ours
0.095
24.215
0.138
23.606
0.149
23.199
0.124
24.069
0.309
22.260
0.108
24.665
SD 2.1
Encoder- Decoder
0.071
26.428
Bound
27

---

Compressed Image Generation with Denoising Diffusion Codebook Models
Table 4. Compression quantitative evaluations, CLIC2020 dataset.
Method
BPP
FID
LPIPS
PSNR
0.040
112.453
0.451
24.777
0.046
106.936
0.428
25,213
0.064
96.809
0.380
26. 14.149
BPG
0.088
86.578
0.331
27.138
0.119
77.203
0.285
28.072
0.160
68.827
0.242
29.049
0.214
60.534
0.204
30.063
0.106
52.459
0.193
28.934
CRDR-D
0.188
40.228
0.124
31.062
0.355
27.793
0.070
33.705
0.106
14.292
0.074
28.352
CRDR-R
0.188
10.022
0.045
30.452
0.355
6.225
0,025
33,028
0.173
14.000
0.053
28.878
HiFiC
0.322
10.157
0.032
31.040
0.081
12.666
0.087
26.898
0.146
9,923
0.056
28.662
ILLM
0.271
8.246
0.034
30.840
0.391
7.222
0.024
32.233
0.025
15,007
0.435
18.153
PSC-P
0.049
13.527
0.335
20.110
0.095
11.879
0.229
22.276
0.033
13.896
0.287
18.111
PerCo (SD)
0.127
7.888
0.128
22.453
0.030
9.459
0.186
22.630
0.038
8.386
0.160
23.171
0.050
7.755
0.137
23.748
Ours
0.095
7.340
0.116
24.472
0.149
7.164
0.103
25,008
0.309
6,825
0.088
25.782
SD 2.1
Encoder-Decoder
0.056
27.901
Bound
28

---

Compressed Image Generation with Denoising Diffusion Codebook Models
Table 5. Compression quantitative evaluations, for the DIV2K
512
512 dataset.
Method
BPP
FID
LPIPS
PSNR
0.058
152.623
0.490
22.793
0.068
146.219
0.463
23.217
BPG
0.096
134.029
0.407
24.124
0.134
122.895
0.348
25.095
0.327
87.879
0.192
28.049
0.152
89.766
0.203
26.618
CRDR-D
0.274
69.466
0.118
28.762
0.152
39.291
0.081
26.110
CRDR-R
0.274
31.422
0.048
28.275
HiFIC
0.226
39.832
0.066
26.138
0.104
42.591
0.107
24.579
ILLM
0.187
34.103
0.069
26.206
0.332
29.267
0.043
28.229
0.025
48.350
0.474
15.869
PSC-P
0.049
44.267
0.386
17.470
0.095
38.406
0.281
19.220
0.033
40.944
0.325
16.518
PerCo (SD)
0.127
29.516
0.155
20.483
0.030
36,283
0.238
19.769
0.038
32.917
0.206
20.319
0.050
30.724
0.177
20.881
Ours
0.095
29.322
0.150
21.566
0.149
28.011
0.132
22.107
0.309
26.756
0.114
22.867
SD 2.1
Encoder-Decodee
0.076
24.878
Bound
29

---

Compressed Image Generation with Denoising Diffusion Codebook Models
Table 6, Compression quantitative evaluations, for the CLIC2020
768
768 dataset.
Method
BPP
FID
LPIPS
PSNR
BPG
0.032
115.009
0.451
25.547
0.038
109.587
0.429
25.984
0.054
99.482
0.385
26.925
0.075
89.013
0.338
27.915
0.102
79.456
0.296
28.840
0.138
71.107
0.255
29.811
0.185
62.617
0.219
30.812
HiFiC
0.154
9.359
0.052
29.745
ILLM
0.006
62.327
0.447
21.304
0.009
41.479
0.340
22.703
0.072
7.852
0.084
27.808
0.134
5.966
0.055
29.567
PerCo (SD)
0.003
30.409
0.517
15.339
0.032
12.869
0.269
19.018
0.126
5,419
0.122
23.387
Ours
0.007
23,862
0.404
19.672
0.008
19.521
0.354
20.532
0.010
15.559
0.314
21.207
0,014
11.362
0.262
22.116
0.017
8.722
0.227
22.722
0.022
6.753
0.192
23.366
0.042
5.051
0.156
24.136
0.066
4.549
0.133
24.739
0.137
4.132
0.108
25.650
SD 2.1
Encoder-Decoder
0.055
28.850
Bound
30

---

Compressed Image Generation with Denoising Diffusion Codebook Models
Table 7. Compression quantitative evaluations, for the DIV2K 768 x 768 dataset.
Method
BPP
FID
LPIPS
PSNR
BPG
0.049
139.204
0.485
23.504
0.058
132.393
0.460
23.928
0.081
120.682
0.406
24.841
0.114
107.472
0.350
25.826
HiFiC
0.205
23.077
0.063
27.096
ILLM
0.007
92.242
0.485
19.234
0.011
71.777
0.369
20.522
0.093
24.253
0.102
25.446
0.169
18.386
0.067
27.085
PerCo (SD)
0.003
51.915
0.519
14.436
0.032
31,378
0.311
17.364
0.126
16.487
0.148
21.317
Ours
0.007
48.315
0.469
17.194
0.008
42.932
0.422
17.858
0.010
37.680
0.383
18.413
0.014
31.038
0.328
19.246
0.017
26.599
0.286
19,832
0.022
22.073
0.247
20,432
0.042
18.330
0.201
21.167
0.066
16.396
0.172
21.780
0.137
14.662
0.138
22.707
SD 2.1
Encoder-Decoder
0.071
25.919
Bound
31

---

Compressed Image Generation with Denoising Diffusion Codebook Models
C. Compressed Conditional Generation Supplementary
C.1. Background and Proof of Proposition 6.1
We will prove that Proposition 6. holds for any score- score-based diffusion model (Song et al., 2020). For completeness, we first provide the necessary mathematical background and then proceed to the proof of the proposition.
C.1.1. BACKGROUND
Score-Based Generative Models. Score-based generative models (Song al., 2020) define diffusion process (x(1) E [0, T),, where Po and pr denote the data distribution and the prior distribution, respectively, and denotes the distribution of zt).. Such a diffusion process can generally be modeled as the stochastic differential equation (SDE)
dx f(x, t)dt g({)dw,
(16)
where t() t) is called the drift coefficient, gt)) is called the diffusion coefficient, w(t) is standard Wiener process, and dt denotes an infinitesimal timestep. Samples from the data distribution po can be generated by solving the reverse-time SDE (Anderson, 1982).
dx = [xx, (x)) di + g(O)dw,
(17)
starting from samples of x()). Here, (x(())) = V (() log (x(()) is the score of Pt. w(t) denotes standard Wiener process where time flows backwards, and dt is an infinitesimal negative timestep. Samples from the data distribution can also be generated by solving the probability flow ODE,
[(x,,) t) - - (t)s dt.
dx
(18)
The reverse-time SDE Eq. (17) can be solved with any numerical SDE solver (e.g., Maler-Maryyama), which corresponds to some time discretization of the forward and reverse stochastic dynamics. For the sake of our proof, we adopt the simple solver proposed by Song al. (2020),
Solving The Reverse-Time SDE.
x;-1 = - (x,) + 9,24, 2, ~ (O,,
(19)
where = T..... and f, and g, are the time-discretized versions of f and g. respectively. Note that DDPMs (Ho et al., 2020) are acore-based diffusion models that solve a reverse-time Variance Preserving (VP) SDE, where f(x((), 1) = (()(()) and g(t) /3(t) for some function B.
C. 1.2 PROOF OF PROPOSITION 6.
Given any general score-based diffusion model, we can write the DDCM compressed conditional generation process as
= = fe(x;) + gx,; + (kC,
(20)
where k; are picked according to Eq. (8). Choosing L - Lp, we have
h; arg min [[C|(k) 9 Vx, (y|x)( kE(
(21)
Since each C, contains K independent samples drawn from a normal distribution (O(, we have
R"
(22)
K-+00
where n denotes the dimensionality of each vector in C, and (C,(1), (I),.... (e.......KKK)) C,(K) is the set comprised of all the elements in the C₁ (without repetition). Since gi xx, log (x.) E R have
min [[C,(k) g. Vx, (y|:)( kE(l...,K)
0. K-000
(23)
32

---

Compressed Image Generation with Denoising Diffusion Codebook Models
Thus,
(i(k)) - 9i Xx, logx:
(24)
K-000
Plugging Eq. (24) into Eq. (20), we get
X4-1
+ X; fi(x,)
+ Vx, log ly
(25)
K
=X f((x)) v logp, (x:) 9f Vx, logp:
(26)
=X f((x) 9 Vx, log pi (x:) + Vx, (y|x;
(27)
=X; - f((x) + 9 vx log p: (x:
(28)
where in Eq. (28) we used Bayes rule and the fact that Vx, log p(y) = 0. Note that Eq. (28) resembles time discretization of probability flow ODE (Eq. (18)) over the posterior distribution po (xoly), with f(: t) and V2g(t) being the drift and diffusion coefficients in continuous time, respectively. Thus, when K oc, our compressed conditional generation process becomes sampler from po (xn
C.2. Image Compression
Private Case of Compressed Conditional Generation
We show that our standard image compression scheme from Sec. is private case of our compressed conditional generation scheme from Sec. 6, where y Xy and L Lp. When y
Xo we have
Vx. log (ygx;()(x,) Vx, log p; (x,]x;)
Xx, log p; (x, (xo) Vx, (x.).
(29)
= V. log p. (x; xx)
s;(x))
(30)
where Xx; (x; can be computed in closed-form via the forward diffusion process in Eq. (l). In particular, we have (x))) = (xx;; /a,x0, (1 T) (Ho et al., 2020), and thus
/x
Vx x; log lo: = Vx;
(31)
2(1 āi)
X - Vāxx0
(32)
1 ai
Vā,X0 Xi
(33)
1 -ē
Moreover, it is well known that (Ho al., 2020; Song et al., 2020)
Vā, X; a;
s (x,)
(34)
where Sujs is the (time-aware) Minimum Mean-Squuared-Error (MMSE) estimator of xo given x, Plugging Eqs. (33)
and (34) into (30), we get
V Vx, log (xol(x)) = Vas (Xo xo/1)) 1 - ā;
(35)
Thus, we have
k, arg min ke(I....K) Cr(y,x;, C,,
(36)
2
Va;
arg min RE(I.....K)
Ci(k) 0i ai
(xo x01+)
(37)
Vai
Von
arg min kE(I....K)
[[C(())]12 2(C((K), a;
(xo
{ol1)) +
0; a.
(xo 2011)
(38)
ai
V
= arg min kE(l....K)
[[C(+)]] 2(C)(K), ai
(xo
)01()/-
(39)
33

---

Compressed Image Generation with Denoising Diffusion Codebook Models
Below, we show that [[C(( is roughly equal for every k. Thus, it holds that
h; arg min [CC(())]] (k)) oi Voa k)
(xo
k0/1))
(40)
ai
Vii
arg min const (K), o3 ke K)
(xo &)
(41)
Vā: (xo- - -oo1;)
arg max (C:(k), 0, kE(l...))
(42)
1 ai
Vā; (C,(k), Xo - Soo1o)
arg max Ji ke(l...,K)
(43)
-1 ā
arg max (C;(k)) Xo x
(44)
KE kE()...,K)
Note that the noise selection strategy in Eq. (44) is similar to that of our standard compression scheme, namely Eq. (7). Thus, our compression method is a private case of our compressed conditional generation approach. practice, we used Eq. (7) instead of Eq. (36) since the former worked slightly better.
To show that [+]]2 is roughly constant for every k. note that (k) sample from a n-dimensional multivariate normal distribution (0, r). Thus, [k)]2 is sample from chi-squared distribution with n degrees of freedom. It is well known that samples from this distribution strongly concentrate around its mean for large values of n. Namely, [|C] is highly likely to be close to n, especially for relatively small values of K.
34

---

Compressed Image Generation with Denoising Diffusion Codebook Models
Original
Degraded
DPS
Compr. DPS
DDNM
Compr. DDNM
Ours
Figure 17. Qualitative comparison of zero-shot image super-resolution methods (posterior sampling). Our approach clearly produces better output perceptual quality compared to previous methods.
C.3. Compressed Posterior Sampling for Image Restoration
DPS and DDNM are implemented with the official settings recommended by the authors (Chung et al., 2023; Wang et al., 2023b). Specifically, DPS uses DDPM with = 1000 sampling steps, and DDNM uses DDIM with 7) = 0.85 and T 100 sampling steps. We also tried T 1000 for DDNM and found that 100 works slightly better for the tasks considered. The additional qualitative comparisons in Figs. 17 and 18 further demonstrate that our method produces better output perceptual quality compared to DPS and DDNM.
35

---

Compressed Image Generation with Denoising Diffusion Codebook Models
Original
Degraded
DPS
Compr. DPS
DDNM
Compr. DDNM
Ours
Figure 18. Qualitative comparison of zero-shot image colorization methods (posterior sampling). Our approach clearly produces better output perceptual quality compared to previous methods.
36

---

Compressed Image Generation with Denoising Diffusion Codebook Models
C.4. Compressed Real-World Face Image Restoration
C.4.1. EXPLAINING THE CHOICE OP r(y)
To explain our choice of r(y) E E[xoly], first note that the MSE of any estimator Xo of x given y can be written (Freirich et al., 2021)
MSB(xo, Xn) MSE(X,, r(y)) MSE(((y)) xo)
MMM(((()) xx) +
(45)
where c, the MMSE, does depend Xg- In theory, our goal is optimize the tradeoff between the MSE of Xo and its output perceptual quality according to some quality measure Q. This can be accomplished by solving
min MSE(Xo, Xn) + XQ((s),
(46)
X
where À is some access to the original image Xg. By plugging Eq. (45) into Eq. (46) we obtain
that controls the perception-diisortion tradeoff. At test time, however, we do not have
min MSE xo) + XQ(xx) = min MSE(r (y), So) +c+ AQ(xo)
%
min MSE(r(y)) St)) + XQ(xt))
(47)
%0
Namely, as long as r(y) is good approximation of the true MMSE estimator, rely on it for optimizing tradeoff (46) without having access to Xo. This resembles our approach in Sec. 6.2, where greedily optimize Eq. (47) throughout the trajectory of the DDCM sampling process.
C.4.2. ADDITIONAL DETAILS AND EXPERIMENTS
We use the PyIQA package to compute all perceptual quality measures, with nige for NIQE, topiq face for TOPIQ, clipiqa+ for CLIP-IQA* and fid daddinovv2 for FDDINOw2, adopting the default settings for each measure. Additional qualitative and quantitative comparisons are shown in Figs. 19, 20, 21 and 22.
37

---

Compressed Image Generation with Denoising Diffusion Codebook Models
Original
WehPlustoTest
Lew
Laput (ecal.
PMRP
Compe PMRT
DiiFaos
Compr. Defifcc
Compr. BFTURuuio
HT
Ourst (TOPIQ)
(iiQe)
Ours (urrs CCLBP-CPPOOO (QA)
Figure Qualitative comparison of real-world face image restoration methods. Our method produces high perceptual quality results with less artifacts compared to previous methods, especially for challenging datasets such as WIDER-Test.
38

---

Compressed Image Generation with Denoising Diffusion Codebook Models
26.5
26.0
26.3
26.0
25.7
25.7
25.7
25.7
PSNR
25.0
25.0
24.2
0.75
0.79
0.84
0.88
3.40
3.93
4.45
4.98
0.67
0.69
0.72
0.74
747.26
781.62
815.96
850.34
TOPIQ
NIQE
CLIP-IQA
FDDINOv2
Compressed
PMRF
Compresed DifFace
Compresed BF Rifuussoon BF BPRfussion
Ours(NIQR)
(urs(TOPIQ)
Ours(CLIP-IQA
PMRF
DifFace
Figure 20. Quantitative comparison of real-world face image restoration methods, evaluated the CelchA-Test dataset. We successfully optimize each NR-IQA measure, surpassing the scores of previous methods. Here, only our NIQE-based solution generalizes well FDDNNO terms of perceptual quality.
33.0
ProxPSNN
30.7
30.7
30.7
30.7
28.4
28.4
28.4
28.4
26.1
26.1
26.1
26.1
0.73
0.78
0.63
0.89
3.05
3.70
4.35
4.99
0.68
0.70
0.72
0.75
571.86
625.54
679.21
732.89
TOPIQ
NIQE
CLIP-IQA*
FDDINOv2
Compressed
PMR
Compresse DifFace
Compressdd BFRffusion
BF BFRflusion
Ours(NIQE)
Ours(TOPIQ)
(LLs(CLIP-)QA
DifFace
Figure 21. Quantitative comparison of real-world face image restoration methods, evaluated on the LFW-Test dataset. We successfully optimize each NR-IQA measure, surpassing scores previous methods. All our solutions achieve impressive FDpINo scores, while our NIQE-based solution surpasses all methods according to this measure.
ProxPSNN
23.1
0.76
2.90
1140.63
TOPIQ
NIQE
CLIP-IQA
FDpiNov2
Compressed
Compresedd
Compressed BFRiuusion
Ours(NIQE)
Ours(TOPIQ)
(LLCCCII)
PMRE
DifFace
BFRifusion
Figure 22, Quantitative comparison of real-world face image restoration methods, evaluated on the WebPhoto- dataset. successfully optimize each NR-IQA measure, surpassing the scores of previous methods. Our TOPIQ-based solution achieves the best FDDINO<2 scores compared to all methods..
39

---

Compressed Image Generation with Denoising Diffusion Codebook Models
C.5. Compressed Classifier Guidance
Consider the case where y represents the class of an image Xo. An unconditional score-based generative model can be guided to generate samples the posterior Po (xoly), by perturbing the generated samples according to the gradient of time-dependent trained classifier co co x,, i) s p)(y)x)) (Dhariwal & Nichol, 2021). This approach is known as classifier
guidance (CG). Such guidance method can be interpreted as an attempt to confuse the classifier by perturbing its input adversarially (Ho & Salimans, 2021). However, trained classifiers are typically not robust to adversarial perturbations, making their gradients largely unreliable and unaligned with human perception (Szegedy et al., 2014; Tsipras et al., 2019; Ganz et al., 2023). Thus, the standard CG approach has not seen major success (Ho & Salimans, 2021).
We propose an alternative to this method, circumventing the reliance on the classifier's gradient. Specifically, we set L in Eq. (8)
fly, C,, log co (y: (x())) + (kC,
(48)
Thus, C(y, X,y Ci, k) attains lower value when (A) points some direction that maximizes the probability of the class y. Note that since the codebooks remain fixed, choosing k; (out of 1...., K) to minimize Eq. (8) would always lead the same generated sample for every y. Thus, we promote sample diversity by first randomly selecting subset of K < K indices ka.1..... K Unif([1..... K}), and then choosing
k = arg min kE .
Cly, x C,,
(49)
We coin our method Compressed CG (CCG).
We compare our proposed CCG with the standard CG using the same unconditional diffusion model and time-dependent classifier trained on ImageNet 256 x 256 (Deng al., 2009; Dhariwal & Nichol, 2021). We compare the methods "on the same grounds" by using the same standard DDPM noise schedule and T = 1000 diffusion steps. Our method is assessed with K" = 256 and K = 2, while the standard CG is assessed with CG scales E {1, 10, 20). The quantitative comparison in Tab. 8 shows that CCG achieves better (lower) FID and FDDINOV2 scores. A visual comparison is provided in Fig. 23. Note that while using DDCM with standard CG does still produces compressed output images, decoding the produced bit-streams requires access to y. Using DDCM with CCG instead sidesteps this limitation, as y is not needed for decompression.
Table Quantitative comparison of compressed classifier guidance (CCG) and standard classifier guidance (CG) for ImageNet 256 256 conditional image generation, using an unconditional DDM and a classifier for guidance. Our proposed CCG not only outperforms CG in terms of generation performance, but also automatically produces compressed image representations.
Compressed CG (Ours)
Standard CG
K 256, K- =
1 10 20
FID
13.669
31.548|
14.481
14.921
FDDINO12
204.693
459.42
255.41
248.34
40

---

Compressed Image Generation with Denoising Diffusion Codebook Models
Figure 23. Qualitative comparison of CCG (left) with CG (right). CCG achieves superior image quality compared to CG while avoiding the use classifier gradients. Additionally, CCG enables decompression without requiring access to the original class labels.

---

Compressed Image Generation with Denoising Diffusion Codebook Models
male black shirt cellphone bridge
DDPM+C fo #9
DDCM+ DDCM+ CCFG
(k = 64, 2)
(KC = 84, = 6)
Figure 24. Qualitative comparison of CCFG with CFG. CCFG achieves comparable image quality and diversity to CFG, while enabling decompression without requiring the original inputs.
C.6. Compressed Classifier-Free Guidance
The task of text-conditional image generation can be solved using conditional diffusson model, which, theoretically speaking, learns to sample from the posterior distribution (x)l)).. In practice, however, using conditional model directly typically yields low fidelity to the inputs. To address this limitation, CG can be used improve this fidelity the expense of sample quality and diversity (Dhariwal & Nichol, 2021). Classifier-Free Guidance (CFG) is used more often in practice, as it achieves the same tradeoff by mixing the conditional and unconditional scores during sampling (Ho & Salimans, 2021), thus eliminating the need for a classifier. Particularly, assuming we have access to both the conditional score s, (x.,)) = Vx, log., (x4l) and the unconditional one s, (x;), CFG proposes to modify the conditional score by
(x.,) = (1 + (x,. (x;(x;)
(50)
where the CFG scale, is hyper-parameter controlling the tradeoff between sample quality and diversity.
Here, we introduce a new CFG method that allows generating compressed conditional samples using any pair of conditional and unconditional diffusion models, while controlling the tradeoff between generation quality and the fidelity the inputs.
42

---

Compressed Image Generation with Denoising Diffusion Codebook Models
The flamingos oulaide in the lugether.
DDPNe cre Ioo 00222 23
DDPMHO (w = 8)
DDCMAN CCFG
[X a4 -
DDCM+CCFG (& 64, K = 6)
Figure 25. Qualitative comparison of CCFG with CFG. CCFG achieves comparable image quality and diversity to CFG, while enabling decompression without requiring the original inputs.
Specifically, since Xx, log (y)xx())x,) = (x)l)) - (x;)) we simply use
C(y, C, k)) - (C, (x - (xx)))
(51)
Note that optimizing Eg. (51) is roughly equivalent to optimizing Lp when is high dimensional (see App. C.2). As in App. C.5, we promote sample diversity by choosing k; from a randomly sampled subset of K < K indices at each step during the generation. We coin our method Compressed CFG (CCFG).
We implement our method using SD 2.1 trained on 768 x 768 images, adopting a DDPM noise schedule with = 1000 diffusion steps, K 64 fixed vectors in each codebook and K E (2,3, 4, 6, 9}. We compare against the same diffusion model with standard DDPM sampling, using T 1000 steps and CFG scales w E {2, 5, 8, 11). The generative performance of both methods is assessed by computing the FID between 10k generated samples and MS-COCO, similarly to Sec. 4. Additionally, we evaluate the alignment between the outputs and the input text prompts using the CLIP score (Hessel et al., 2021) with the OpenAl CLIP ViT-L/14 model (Radford et al., 2021).
Figure 26 shows that our CCFG method is on par with CFG in terms of FID, while CFG produces higher CLIP scores. This suggests that the outputs of CFG better align with the input text prompts compared to CCFG. Yet, the qualitative comparisons Figs. 24 and 25 show that there no significant difference between the methods. Importantly, decoding the
43

---

Compressed Image Generation with Denoising Diffusion Codebook Models
13 14
15
16
18
+- FID
CFG
CCPG (Ours)
Figure 26. Quantitative evaluation of CCFG and CFG. CCFG achieves comparable FID scores CFG while achieving slightly lower fidelity the input prompts. However, unlike CFG, CCFG enables decompression without access to original conditioning inputs.
bit-streams produced by CCFG does involve accessing the original input and so our loss in CLIP scores are expected due
to the wwtp-pppeeeeeddisieeeeeee distortionn tradeoff (Blau & Michaeli, 2019) (here, we achieve 1000-logg (64) x 0.01 BPP). Note that 7682
using CCFG in DDCM is fundamentally different than using CFG (Sec. 4), since the latter requires access to
44

---

Compressed Image Generation with Denoising Diffusion Codebook Models
C.7. Compressed Text-Based Image Editing
Image editing has seen significant progress in recent years, particularly in solutions that rely on pre-trained DDMs. Generally speaking, most methods rely inversion Suuberman-SSiegelll
et al., 2024; Manor & Michaeli, 2024; Wu & De la Torre, 2023; Wallace et al., 2023; Hertz et al., 2023a), noising and denoising the original input (Meng et al., 2022), manipulating the DDM's s attention layers (Hertz et al., 2023b; Tumanyan et al., 2023). We demonstrate simple approach for editing images using DDCM with a text-conditional DDM.
Specifically, we compress an image while feeding the DDM with a source prompt Gre describing the image. To edit the image, we start by decoding up to timestep Tet while feeding the DDM with the original prompt Gae- We then continue with the decoding from timestep Tedi,, while feeding the DDM with the target prompt Cas- Intuitively, this approach should preserve the low frequency contents in the original image (e.g., objects), while guiding the image towards the described edit.
We compare our approach to DDPM inversion (Huberman-Spiegen Spiegelglas et al., 2024) and DDIM inversion (Song et al., 2021). For DDPM inversion we set T = 100, Taki 36, as recommended by the authors. For DDIM inversion we set T = 100, and follow Subberman-Spiegelggee et al. (2024) to additionally apply DDIM inversion may, using ski = 40. For our approach, we set T = 1000 and Tedi = 600. Note that this value of Tdi equivalent to Takp = 40 if T = 100 sampling steps were used. For all methods we use the pre-trained Stable Diffusion 1.4 checkpoint from Hugging Face. Images are taken from the modified- modifie-IImageNetee ImageNetR-T121 dataset (Huberman-Spiegeelllan et al., 2024; Tumanyan al., 2023).
Preliminary qualitative results are shown Fig. 27. Our approach is less structure preserving than DDPM inversion, while offering more semantic object preservation than DDIM inversion. We encourage future works to investigate DDCM-based image editing methods.
45

---

Compressed Image Generation with Denoising Diffusion Codebook Models
Original
DDIM Inversion
DDIM Inv. Mid-Way
DDPM Inversion
Ours (K = 8192)
a photo of foxes
a photo of cats
graffiti of cat sculpture of cat
sketch of a
an origami of bear
painting of a husky
sculpture of husky
Figure 27. Qualitative comparison of image editing methods. Our approach preserves less image structure compared to DDPM inversion, while offering more semantic object preservation than DDIM inversion. This can be quite useful in scenarios where the editing prompt requires major structural changes, such as transforming a sketch cat into an origami a bear.
46


Reviews:
[{"review_id": "FPhmCnIpnb", "reviewer": "Reviewer_EqE9", "summary": "This paper presents a novel approach, DDCM, to represent an image by DDPM procedure using a set of gaussian noise (codebook) indices. In other words, this paper shows that it is possible to \"discretize\" the \"z\" at every step of DDPM to approximate a high-quality diffusion procedure. DDCM can also be leveraged to solve various tasks, like text-to-image and image-to-image problems, which shed a light for codebook-based diffusion sampling procedure.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "# Fig. 3 FID\nYes, the FID scale in Fig. 3-L is $10^1$. The FID of DDPM is 9.21 (black dashed line). Indeed, some prior works with the same DDM reported lower FID on ImageNet 256x256, when using 50k generated samples [R2]. However, we use only **10k generated samples** (see L171-L) to reduce computation time, similarly to [R3,R4]. Decreasing the number of samples is known to increase the FID [R1]. To validate this, we computed the FID between the same 50k reference images and two random subsets from the ImageNet 256x256 training set, consisting of 10k and 50k images. These subsets obtained FID=4.934 and FID=1.968, respectively.\n\nNote that this experiment only aims to demonstrate DDCM's competitiveness with DDPM, specifically for small codebooks. Indeed, the modest decrease in FID from 10.5 to 9.5 (when increasing $K$) is a strength of our method rather than a limitation: even for $K=2$, DDCM achieves FID$\\approx$10.5 (competitive with DDPM). This supports our hypothesis (L016-R & L182-L) that the Gaussian representation space used by standard DDMs is redundant. \n# Fig. 5 & VAE Bound\nThe FID scores for the compression results in Fig. 5 are in log scale.\n\nRegarding the “VAE bound,” we apologize for the misunderstanding, this term was not clarified in the paper. In some experiments we employ a latent DDM (specifically, SD 2.1), so we are compressing the latent VAE encoding of a given image, rather than the image itself. Since encoding-decoding an image using a VAE typically does not yield perfect image reconstruction, the distortion of any VAE-latent-space-based compression method (e.g., ours, PerCo, PSC) is bounded by that of the VAE encoder-decoder. The “VAE bound” in the figure corresponds to the distortion resulting from encoding and decoding the original images using only the VAE, **without any additional compression**. This bound is important to present since it allows to distinguish the distortion caused by the VAE from that caused by the compression scheme.\n\nThanks for these important points. We will clarify them in the revised paper and replace the term “VAE-bound” with “SD 2.1 Encoder-Decoder bound.”\n# Eq. (7) Efficiency\nPlease note that Eq. (7) does not involve any gradient computation, iterative optimization, or use of a neural network. It involves a straightforward matrix multiplication followed by picking the maximum out of K scalars. Empirically, on an L40S GPU with $K=8192$ and 256x256 images, this operation takes 0.357ms on average (over 100 trials). This is negligible compared to the DDM’s forward pass, which takes 57.2ms on average.\n# Tokenization Methods\nWe appreciate this suggestion. Viewing DDCM-based compression as an image tokenization method with random noises acting as the tokens is interesting. We will discuss this in our paper revision.\n# Numerical Results\nThanks for this valuable suggestion. We will add numerical comparisons to the appendix.\n# Applying to Flow Matching Models\nAlthough flow-matching models typically generate samples by solving an ODE, several works (e.g., [R5,R6]) have shown they can also generate samples by solving an SDE, which involves adding noise at each generation step. Thus, we believe that such flow-matching SDEs may be discretized as well, similarly to DDCM.\n# Resolution Generalization\nDDCM is easily generalizable to any resolution. Instead of pre-sampling the codebooks with a particular resolution, one can rely on a shared-seed random generator to dynamically generate codebooks for different resolutions. Specifically, one can store/transmit the resolution as part of the bitstream and, before compressing/decompressing an image, randomly sample the codebooks for the given resolution. Using the same seed for compression and decompression ensures that both would rely on the same codebooks.\n\nIf no shared-seed random generator is available, one can pre-sample the codebooks for a maximal allowed resolution, store/transmit the resolution of the image, and dynamically slice smaller codebooks from the pre-sampled ones.\n\nTherefore, the generalizability of DDCM to different resolutions is determined only by that of the pre-trained DDM.\n# Learnable Codebooks\nLearning the codebooks may indeed enhance DDCM’s performance (as noted in L433-R), particularly in compression. We investigate this in our future work.\n\n## References\n[R1] Mikołaj Bińkowski et al. Demystifying MMD GANs. ICLR 2018.\n\n[R2] Cheng Lu et al. DPM-solver: A fast ode solver for diffusion probabilistic model sampling in around 10 steps. NeurIPS 2022.\n\n[R3] Cheng Lu et al. DPM-Solver++: Fast solver for guided sampling of diffusion probabilistic models. arXiv:2211.01095.\n\n[R4] Kaiwen Zheng et al. DPM-Solver-v3: Improved diffusion ODE solver with empirical model statistics. NeurIPS 2023.\n\n[R5] Jeongsol Kim et al. FlowDPS: Flow-driven posterior sampling for inverse problems. arXiv:2503.08136.\n\n[R6] Litu Rout et al. Semantic image inversion and editing using rectified stochastic differential equations. ICLR 2025.", "claims_and_evidence": "Claims in this submission are fully supported by clear and convincing evidence.", "methods_and_evaluation_criteria": "The proposed method, DDCM, is suitable for the visual generative problem, especially for compressed image generation. The evaluation criteria, computing several metrics on validation set like ImageNet, makes sense for visual generation. There are some minor points and see experiment design parts.", "theoretical_claims": "I am not an expert on mathematical proof on diffusion theory but proofs in C.2 seem sound and correct to me.", "experimental_designs_or_analyses": "The experimental designs and analyses are sound but some points and result presentation seem confusing.\n\n1. In Figure 3 left, is the FID scale 10^1? Then it means that FID only decreases from 10.5 to 9.5 when increasing K from 2 to 64, which is not significant. Additionally, what is the exact FID value for DDPM in this figure? In my view, an FID of approximately 9.0 on ImageNet for DDPM or DDCM is not that strong. However, the visualizations do show high-quality samples. I'd like to see the authors' insights on this.\n\n2. Relevant to P.1, what is the scale in Figure 5 FID, is it linear scale or log scale? Why show VAE bound instead of DDPM bound in Figure 5?\n\n3. Can authors include the computing time of Equ (7)? Is the method efficient?", "relation_to_broader_scientific_literature": "I think that the key contribution of this paper is discretization of the z noise in DDPM procedure, which is indeed novel and unexplored before.", "essential_references_not_discussed": "I have not found essential references not discussed in this submission. However, I feel it is necessary to include discrete visual tokenization methods in the discussion since the proposed DDCM essentially does the discretization on images using diffusion models. The authors can discuss DDCM and discrete tokenization methods in a paragraph but not experiments needed.", "other_strengths_and_weaknesses": "Overall, this submission is a good and novel paper on discretizing the z noise in diffusion steps and representing images by compressed noise indices. For other minor concerns, please see Questions For Authors.", "other_comments_or_suggestions": "Suggestions: the authors can provide absolute metric values in *numeric forms* in figure or table presentation, which could make results of DDCM more clear.", "questions_for_authors": "I'd like to see insights for several open-end questions listed below from authors. No experiments needed.\n\n1. Flow matching/rectified flow generative models are gaining popularity in the research community and have demonstrated competitive results. However, there is no random noise in ODE process behind FM except for initial point. Given this, can DDCM be applied to recent flow matching models like FLUX?\n\n2. Is DDCM limited to the fixed-resolution generation process, given that the codebook fixed after intialization? Can noise codebook generalize to other resolutions with minimal efforts?\n\n3. For better performance, is it doable to make noise map codebook learnable?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "9eQQFIJkkz", "reviewer": "Reviewer_bEwf", "summary": "This paper introduces Denoising Diffusion Codebook Models (DDCM), a approach that replaces standard Gaussian noise sampling in Denoising Diffusion Models (DDMs) with selections from fixed codebooks of i.i.d. Gaussian vectors. Despite using a discrete and finite noise representation, DDCM preserves the sample quality and diversity of standard DDMs. The method enables state-of-the-art perceptual image compression by selecting optimal noise samples for a given image and generalizes to other conditional generation tasks, such as image restoration. Additionally, the paper provides a mathematical interpretation linking DDCM to score-based posterior sampling.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "# DDCM Bitrate and File Size Clarification\nPlease note that the reported bitrate is precisely that of the compressed file size, as in traditional compression methods. Indeed, the mentioned logarithmic relationship $T\\cdot log_{2}(K)$ between the codebook size $K$ and the number of sampling steps $T$ directly calculates the file size (in bits). This holds since each DDCM-generated image corresponds to a sequence of $T$ integers, each in $[0,K-1]$, that represent the indices of the codebook entries chosen during the generation (as depicted in Fig. 2). This sequence is then represented in a binary format and saved as a file. E.g., for $T=3,K=4$, the indices sequence [1,3,0] translates to the bitstream “011100”. We compare our method against previous compression approaches (including traditional ones) according to their file size (using BPP).\n# Experimental Setup and Results Clarification\nWe apologize for any confusion about our experimental setup and results. Our paper contains multiple different experimental settings (starting at lines 157(R), 188(R), 309(R), and 368(R)), involving image generation, compression, and restoration, following common practices in each of these fields. Let us clarify.\n### Section 4 in the paper\nThis section shows that DDCM maintains competitive generation capabilities relative to DDPM, even with small codebooks (lines 158(R)). Specifically, we show that DDCM achieves comparable performance to DDPM as measured by standard generative metrics like FID (Fig. 3, main text), KID (Fig. 8, App. A), and through visual inspection (Figs. 9 & 10, App. A) across two distinct datasets: ImageNet and MS-COCO. This experiment is intended to validate our hypothesis regarding the redundancy of the continuous Gaussian representation space utilized in traditional DDMs (lines 016(R) & 182(L)).\n### Section 5 in the paper\nThis section introduces our perceptual image compression method, based on DDCM. We compare our method to prior approaches using standard rate-perception-distortion evaluations [R2], assessing perceptual quality (FID) and distortion (LPIPS, PSNR) across multiple bitrates (BPP). These quantitative comparisons are provided in Fig. 5, complemented by qualitative assessments in Fig. 4, following common evaluation practices. Both our quantitative and qualitative results demonstrate the superiority of our proposed method over existing techniques (lines 243-258(L)). In Appendix B, we also include additional qualitative results, as well as additional quantitative results for higher-resolution images (Figs. 11-16).\n### Section 6 in the paper\nHere, we propose extending our compression framework to handle more general compressed conditional generation tasks, such as compressed image restoration. Our experiments in the main text cover two tasks: zero-shot posterior sampling (Sec. 6.1) and blind face image restoration (Sec. 6.2). We compare our method with previous state-of-the-art approaches, assessing both distortion (PSNR) and perceptual quality (FID), thus adhering to established evaluation protocols in image restoration [R1]. The qualitative and quantitative results, presented in Figs. 6 & 7, show that our method achieves superior perceptual quality compared to previous methods (lines 353-357(L) & 421-428(L)). Apps. C.3 & C.4 provide additional results (Figs. 17-22).\n\nWe hope this clearly addresses the concerns raised regarding the clarity and comprehensiveness of our experimental setup and results.\n# Superior Perceptual Compression Performance\nKindly note that our image codec is designed for **perceptual** image compression, meaning that we aim to achieve best output perceptual quality. Due to the rate-perception-distortion tradeoff [R2], this is expected to compromise the distortion (e.g., PSNR, LPIPS) of our method. Indeed, since our method achieves the lowest (best) FID (perceptual quality) for almost all bitrates and datasets (Figure 5), the rate-perception-distortion tradeoff explains why our method does not always achieve the best distortion. However, note that we do achieve better distortion compared to previous perceptual image compression methods such as PerCo (SD), while also achieving better perceptual quality.\n# Generation Evaluation Using No-Reference Metrics\nWe kindly note that it is not common practice to evaluate image generation methods with no-reference quality measures (e.g., NIQE [R3]), since the goal in image generation is to sample from a data distribution. No-reference quality measures only assess the quality of the generated images and not their diversity, whereas metrics such as FID and KID (which we report in the paper) assess quality and diversity simultaneously.\n## References\n[R1] Yochai Blau & Tomer Michaeli. The perception-distortion tradeoff. CVPR 2018.\n\n[R2] Yochai Blau & Tomer Michaeli. Rethinking lossy compression: The rate-distortion-perception tradeoff. ICML 2019.\n\n[R3] Anish Mittal et al. Making a ‘completely blind’ image quality analyzer. ACCSC 2012.", "claims_and_evidence": "Without rigorous inspection, it can be considered that the claims in this paper are all supported by evidence.", "methods_and_evaluation_criteria": "The paper defines bit rate based on the codebook size and the number of sampling timesteps, with the bit-stream length determined by their logarithmic relationship. This approach is closer to VQGAN-style methods rather than traditional compression metrics based on actual file size. While this provides a useful measure within the proposed framework, it would be helpful to clarify how this definition translates to real-world storage and transmission costs, and how it compares to standard rate-distortion metrics used in compression research.", "theoretical_claims": "The theoretical part of this paper is fine.", "experimental_designs_or_analyses": "Just from the text expounded in this article, the experimental designs and analysis are somewhat reasonable.", "relation_to_broader_scientific_literature": "The paper is situated within the broader literature on diffusion models, generative modeling, and neural compression.", "essential_references_not_discussed": "No essential references are not discussed.", "other_strengths_and_weaknesses": "The idea of using discrete noise representations in diffusion models is intriguing, but the experimental setup and results are somewhat unclear. While the approach offers a novel take on generative modeling, its compression performance does not appear particularly strong when compared to traditional methods. As a generation method, it would benefit from more comprehensive evaluation using non-reference metrics, which could provide a clearer picture of its capabilities. The strength lies in the concept, but the execution in terms of compression and performance metrics needs further refinement.", "other_comments_or_suggestions": "See above.", "questions_for_authors": "See above.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "HjU68dJSKE", "reviewer": "Reviewer_r3bs", "summary": "This paper introduces a novel approach called the Denoising Diffusion Codebook Model, which replaces the standard Gaussian noise sampling in the reverse diffusion process with a codeword from a predefined codebook. This method enables the development of new lossy image codecs and, more broadly, compressed image restoration schemes.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "# Essential Reference\nWe thank the reviewer for highlighting the insightful work by Huan Liu et al. (ICLR 2022). It is compelling to address tasks such as compressed image restoration by formalizing them as a distribution shift (via optimal transport) with an informational bottleneck. We will include a discussion of this important reference in the related work section of our revised manuscript.\n\n# Codebook Size Adjustment in Different Timesteps\nDynamically adjusting the codebook size across timesteps may indeed offer potential benefits. A positive indication of this can be seen in Appendix B.3, where using different (fixed, not dynamically adjusted) codebook sizes for different timesteps leads to improved performance. However, identifying the “optimal” codebook size for each step can be challenging, as modifying the codebook size at some timestep can affect the outcomes of all subsequent timesteps. We will include this in the discussion as an interesting option for future research.", "claims_and_evidence": "Variants of the proposed Denoising Diffusion Codebook Model have a wide range of applications. This paper demonstrates them through image compression, compressed image restoration, and compressed text-based image editing, among others. Overall, the experimental results are quite convincing.", "methods_and_evaluation_criteria": "Yes, the proposed method, which replaces continuous Gaussian samples with discrete codewords, is intuitively clear.", "theoretical_claims": "I have reviewed the theoretical analyses in the supplementary material. The analyses are relatively straightforward, and I have no concerns about their correctness.", "experimental_designs_or_analyses": "The experimental designs are well aligned with the proposed theoretical framework.", "relation_to_broader_scientific_literature": "This work is related to several areas, including generative models, image compression, and image restoration.", "essential_references_not_discussed": "When it comes to compressed image restoration, the following paper should be cited and discussed.\n\nH Liu, G Zhang, J Chen, AJ Khisti, Lossy compression with distribution shift as entropy constrained optimal transport, International Conference on Learning Representations, 2021", "other_strengths_and_weaknesses": "Overall, the proposed method is highly promising and has numerous potential applications. However, as noted at the end of the paper, several aspects of the method can be further improved. Additionally, there is a lack of theoretical understanding regarding its effectiveness.", "other_comments_or_suggestions": "The current version is acceptable. However, the paper would be stronger with a deeper analysis of a specific topic, such as image compression, rather than providing a uniform treatment of multiple applications.", "questions_for_authors": "In image compression, is there any benefit to adjust the codebook size K over the timesteps?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: This paper has received all acceptances in the final recommendations. The authors present a novel approach, DDCM, to represent an image by the DDPM procedure using a set of Gaussian noise indices. Although the reviewers expressed concerns about the unclear experimental setup and suboptimal compression performance, the authors' rebuttal effectively addressed several of these issues. This led to a consensus among the reviewers in favor of acceptance. Consequently, the Area Chair has decided to accept this paper.
Decision: Accept (poster)