Lossy Compression with Pretrained Diffusion Models
Authors: Jeremy Vonderfecht
Feng Liu
Venue: ICLR 2025 Poster (2025)
Keywords: Compression
Stable Diffusion
Flux
DDPM
DiffC
Primary area: generative models
TL;DR: We implement a principled algorithm for compressing images using pretrained diffusion models (e.g. Stable Diffusion and Flux)

Abstract: We apply Theis et al. (2022)'s DiffC algorithm to Stable Diffusion 1.5, 2.1, XL, and and Flux-dev, and demonstrate that these pretrained models are remarkably capable lossy image compressors. A principled algorithm for compression using pretrained diffusion models has been understood since at least 2020 (Ho et al.), but challenges in reverse-channel coding have prevented such algorithms from ever being fully implemented. We introduce simple workarounds that lead to the first complete implementation of DiffC, which is capable of compressing and decompressing images using Stable Diffusion in under 10 seconds. Despite requiring no additional training, our method is competitive with other state-of-the-art generative compression methods at low ultra-low bitrates.

Published as a conference paper at ICLR 2025
LOSSY COMPRESSION WITH PRETRAINED DIFFUSION MODELS
Jeremy Vonderfecht & Feng Liu Department of Computer Science Portland State University
{vonder2, fliu f @pdx. edu
ABSTRACT
We apply the DiffC algorithm (Theis et al. 2022 to Stable Diffusion 1.5, 2.1, XI., and Flux-dev, and demonstrate that these pretrained models are remarkably capable lossy image compressors. A principled algorithm for lossy compression using pretrained diffusion models has been understood since at least (Ho et al. 2020), but challenges in reverse-channel coding have prevented such algorithms from ever being fully implemented. We introduce simple workarounds that lead to the first complete implementation of DiffC, which is capable of compressing and decompressing images using Stable Diffusion in under 10 seconds. Despite requiring no additional training, our method is competitive with other state-of-the- art generative compression methods at low ultra-low bitrates.
INTRODUCTION
Today's AI labs are spending millions of dollars training state-of-the-art diffusion models. The original Stable Diffusion cost around $600,000 to train, while Stable Diffusion 3 rumored have cost around $10 million (Wikipedia) 2024). The sophistication and expense of these models are only likely to increase as generative modelling expands into the video domain.
While diffusion models are used primarily for poompt-based image generation, they are also highly powerful and flexible image priors. They provide many other affordances, such as casy multiplica- tion with other distributions (i.e., posterior sampling), and log-likelihood estimation (Sohl-Dickstein jet al., 2015). Researchers have discovered many remarkable applications of pretrained diffusion models, including image restoration (Chung al. 2022), optical illusions (Geng et al., 2024b and generative 3D modeling (Poole et 2022).
Ho al. 2020) proposed algorithm by which pretrained diffusion models can be leveraged for data compression. Using this algorithm, an image can be compressed to number of bits approach- ing the model's negative log- -iikelihhoddd estimate of the data. Leveraging this simple algorithm, could today's flagship diffusion models also serve as powerful image compressors?
Image compression remains a relatively underexplored application of diffusion models. Theis al. (202) produced the primary paper on this compression algorithm and christened it DiffC. Theis et al. (2022) cited the challenges associated with reverse-channel coding as significant bottleneck to further adoption of the DiffC algorithm. For this reason, they do not offer complete implementation of DiffC; they only evaluate its hypothetical performance, assuming perfect solution to the reverse- channel coding problem. However, by introducing some simple workarounds, we find that reverse- channel coding induces only minor overhead costs in computation and bitrate. Our contributions are crucial for making DiffC a practical, usable algorithm instead of hypothetical subject of study.
To the best of our knowledge, we are the first to fully implement the DiffC algorithm and the first to apply this algorithm to flagship open-source diffusion models, namely Stable Diffusion 1.5, 2, XL, and Flux-dev. We offer the first publicly available implementation of a DiffC compression protocol GitHut! Additionally, schedule, which both improves our rate-distortion curve and reduces encoding time to less than
propose a principled algorithm for optimizing the denoising
https://githhb.ce wtt.://///eeemm//////./.i.t.me

---

Published as a conference paper at ICLR 2025
Original
Ours, 0.0123 hpp
Text- Sketchh. 0,0281 hpp
VTM. 0.0242 bpp
PerCo. 0,0032 hpp
Figure 1: Kodak images compressed using our method on Stable Diffusion 1.5, Text-Sketch- PICS (Lei et al.) 2023), VTM (vtm), and PerCo Careil et al. 2024). Text=StetchVTM/PeeCC images are taken from Careil et al. (2024)
10 seconds with Stable Diffusion 1.5, Our algorithm works "zero-shot", requiring no additional training, and is naturally progressive, easily allowing images to be encoded to any desired bitrate. Remarkably, despite the fact that these diffusion models were not intended for image compression, we obtain competitive rate-distortion curves against other purpose-buill state-of-the- generative compression methods including HiFiC (Mentzer al. 2020), MS-IILMM (Muckley et al. 2023).
DiffEIC (Li et al. 2024), and PerCo (Careil et al. 2024.
BACKGROUND & RELATED WORK
Our work is founded on the DiffC algorithm introduced by Theis et al. (2022). We contribute several practical advancements to implementing DiffC. While Theis al. applied DiffC to a variant of VDM (Kingma et al. 202T trained on 16x16 ImageNet patches, and only analyzed theoretical compression rate lower bounds, we adapt the algorithm to run efficiently on Stable Diffusion and Flux.
2.1 LOSSY COMPRESSION WITH GENERATIVE MODELS
Methods leveraging generative models have managed to compress images to unprecedented bitrates while maintaining high perceptual quality. The dominant paradigm in this field encodes images by quantizing the latent representations of variational autoencoders. For example, see Minnen et al.) 2018). HiFiC (Mentzer al. 2020), MS-ILLM (Muckley al., 2023). and GLC Jia et al. (2024). Since diffusion models can be regarded as special case of VAEs (Kingma et al., 2021). DiffC can also be considered member of this class of compression techniques. But while other VAE-based approaches build sophisticated entropy models for their latent representations, DiffC has an elegant and principled entropy model "built-in" the algorithm.
Diffusion models represent a powerful and flexible class of generative models for images. Nat- urally, researchers have sought to leverage them for image compression. Elata et al. (2024) pro- pose Posterior-Samplinn based Compression (PSC), which encodes an image using quantized linear measurements of that image, and uses those measurements for posterior sampling from a diffusion model for lossy reconstruction. Like DiffC, PSC can be applied to pretrained diffusion models with no additional training. However, it is also more computationally intensive, as it requires posterior sampling from diffusion model many times encode single image. Xu et al. (2024) use DDPM inversion to generate reconstructions with the same compressed encoding as the original image.
The dominant paradigm in diffusion-bart based compression is and to encode the target image as a compressed representation of the conditioning information. Tkethh (Lei al. 2023) uses ControlNet (Zhang al. 2023) reconstruct image from
conditional diffusion model,
2

---

Published as a conference paper at ICLR 2025
Algorithm 1 Sending Xo (Ho et al., 2020)
Algorithm 2 Receiving
1: Send Xy rv ((xr xx) using p(xx)
1: Receive xy using p(xr)
2: for = T 1, 1,...,,, ,, do
2: for L T L..... 1,0 do
3: Send x ((x) xx+1, xo) using pa (xx(xx+11)
Receive x, using pe po(xi xx41)
4: end for
5: Send Xu using pe (x)
5: return Xo
caption and a highly compressible Canny edge detection-based "sketch" of the original image. PerCo (Careil et al. 2024) and CDC (Yang & Mandt 2024) train conditional diffusion models to reconstruct images from quantized embeddings. DifiC more general, that can be applied to any diffusion model, with or without additional conditioning. In fact, DiffC can easily be combined with these conditional diffusion approaches, to more precisely guide the denoising process towards the ground truth. For example, our prompt-conditioned method could be considered an extension of Text+Sketch's s PIC method. While we do not combine DiffC with other conditional diffusion approaches, like PerCo CDC, regard this as interesting future research direction.
2.2 DIFFUSION DENOISING PROBABILISTIC MODELS
This section provides a brief review of the DDPM forward and reverse processes. For more comprehensive introduction to these concepts, see Sick-Sickstein et al. (2015) and et al. 2020).
DDPMs learn reverse stochastic forward process q. Let (xx, Xi,... Xx) be a joint probability distribution over samples X2 E Rd. Let the marginal distribution ((x)) be the training data distribu- tion. Subsequent samples 11.--.. , Fr sampled from random walk starting Xg and evolving according to:
((x := N( VI B,
where 3, is the "beta schedule" which determines how much random noise to add at each step. The beta schedule is chosen so that ((x)) Rs (O, I). This defines a random process by which samples from training distribution ((x)) are gradually transformed into samples from standard normal distribution g.xr).
To generate samples from the training distribution g(x,). DDPMs construct a reverse process; start- ing from a Gaussian noise sample Xy and then generating a sequence of successively denoised samples Xy-11... Xp. To do so, they must approximate (xx+-1(x,). Happily, as Bg becomes small, g(x;-1 becomes Gaussian. DDPMs parameterize this distribution as a neural network (x(x)_((xx). and train the network to minimize the KL divergence D ( |x.) [[ppp((x (x:-1 (x:).
Importantly, this training objective is equivalent to optimizing the variational bound on the log like- pe pe
lihood: [(xx)= S lgolog
(xo)], which the exact objective we want for data compression. In the next section, we will explain how to communicate sample xo using close to - -((x)) bits.
2.3 DDPM-BASED COMPRESSION
DDPM-basee based image compression works similarly to DDPM-based sampling: we start from random Gaussian sample XT and progressively denoise it produce a sequence of samples XT-1, XT-2,, 2........ Xo. To sample a random Xg. we draw these successive denoised samples from po(x--1xx,) ~ qx;-1 ).) But to compress a specific target image Xg. we instead wish to draw our denoised samples from q(x,11 (x,, Xo). Fortunately, this is tractable Gaussian distribution.
At high level, our compression algorithm sends the minimal amount of information needed to "steer" the denoising process sample from g(x,-1|x, xo) instead of p l The receiver doesn't 't have access to Xp. but can compute po (o(x-11/x) which conveniently optimized to min- imize the expected value of DKz(g(XL
x.) Ie (x--1 )x)).. Using reverse-channel coding (RCC), it is possible to send a random sample from using a shared distribution using close Dar. (gllp) bits. Algorithms and detail the full compression protocol.
3

---

Published as a conference paper at ICLR 2025
While Algorithms and communicate Xy losslessly, they permit simple modification for lossy compression; simply stop after any time to communicate X,, which is noisy version Xg. Then denoising algorithm to reconstruct X Xp. Theis et al. (2022) propose using flow-based DDPM sampling to perform the final denoising step, as this yields naturalistic images with a higher PSNR than standard ancestral sampling. For more detailed explanation of the DifíC algorithm,
Theis et al. (0022).
2.4
REVERSE-CHANNEL CODING
The central operation in Algorithms and 2 wish to communicate random sample from distribution ((x)) using shared distribution p(x)), using roughly Du((lp) lp) bits, and shared random generator. This is a basic problem in information theory called reverse-chamel coding (RCC). We the Poisson Functional Representation (PFR) algorithm 4 from Theis & Ahmed (2022) for reverse channel coding, as this algorithm achieves perfect sampling from the target distribution only slightly more than (ql(p) bits.
is to "send x ~ ((x) using p(æ)". In other words, we
METHOD
As mentioned Section 2.3 the DiffC algorithm operates two stages:
1. First, communicate a noisy sample the data z, using Algorithms [and] Then, denoise this sample using probabiltiy-flo--bee DDPM denoising.
The challenge of implementing this algorithm lies in making RCC efficient. this section, we will describe this challenge and our solution it.
In a theoretically idealized version of DiffC, communicating x using would be possible using exactly DxI(gllp) bits. Under these idealised conditions, communicating noisy sample x, requires
Σ
-((x))
bits of information. This is the appeal using large pretrained diffusion models for compression: the models increase in scale, expect L(x,) to asymptotically approach the ideal "true" log- likelihood of x₁.
Unfortunately, there is known algorithm for doing this efficiently. Instead, one of the best-known algorithms for RCC is PFR (4), which allows us to us to send x ~ 9 using using roughly
Des(glle) + (Dx +
bits. As Dx (gl(p) becomes large, this bitrate overhead becomes neglegible. Unfortunately. PFR's time complexity is exponential in D DxL(0 (x<(gllp)) So when D (xz(gll)) is too large, the encoding time is impractical. But when D (4lp)) is too small, PFR is bitrate-ineffiiient. So there is a "sweet spot" of Drr (gllp) values for which both the encoding time and bitrate overheads are tolerable. With our fast CUDA implementation of PFR, we can RCC 16 bits at a time with negligible computational overhead.
In practice, (xz(g(x;-1xx,, xo) Pe (xx- is impractically small when t is close to T' (as little as 0.25 bits per step), and impractically large when is close 0 (thousands of bits per step). Fortunately, there are workarounds for both cases:
When DrL per denoising step is too small, can take larger steps. For example, instead of sending Zggy ~ (Xyy9) X1000, X²) using Po P x1000)), We can send Ta90 ~ |Xxy90 X1000, Xy) using pe
Skipping denoising steps during DDPM sampling is already common practice for image generation, and DiffC faces similar compute/likelihood tradeoffs. Section 4.3 examines this tradeoff in more detail.
4

---

Published as a conference paper at ICLR 2025
When DKZ is too large, independent distributions, q = Q1 92 ... 9n and p = P1 P2 ... Pn. We select n so that Dx (q: (g:l)p)) is near sweet spot of 16 bits for all i [1. This is possible because q and p high- dimensional anisotropic Gaussian distributions, so each dimension can be sampled (and RCC'd) independently. And long the dimension of is much greater than n, the law of large numbers ensures that:
randomly split up the dimensions of q and p into statistically
(xi p:) ~ - DKI lI
Ei [1,n]
With careful choices of hyperparameters, we have managed to produce fast implementation of DiffC for Stable Diffusion that incurs < 30% bitrate overhead to achieve the same PSNR as the idealized method (see Figure 2).
3.1 CUDA-BASHR REVERSE- REVERSE-CHANNEL CODING
Unoptimized implementations of reverse-channel coding may still be prohibitively slow 16 bits per chunk. The TensorFlow implementation of reverse-channel coding from MIRACLE (Havasi et al. 2019) takes about 140 per 16-bit chunk of our data with an A40 GPU. For kilobyte image encoding using our Stable Diffusion 1.5 compression method, this implementation of RCC takes 80% of the encoding time; requiring about 80 seconds per image produce an encoding.
The key issue is that standard PyTorch/ Tensorflow implementations reverse-channel coding must materialize large arrays of random samples in order take advantage of GPU paralellism. Our custom CUDA kernel can avoid these memory requirements. For more details, see Appendix A.S
Our implementation of reverse-channel coding is approximately 64x faster than what we could achieve with Py Torch or Tensorflow alone. Encoding a 16-bit chunk takes less than 3 millisec- onds; RCC becomes neglegible contribution the overall runtime this compression algorithm. This allows to encode image in less than 10 seconds.
3.2 GREEDY OPTIMIZATION OF DIFFC'S HYPERPARAMETERS
The DiffC algorithm communicates a sequence of successively denoised samples of the target image. These noisy samples have convenient property: each x, send is statistically indistinguishable from a random sample of q(x, xx)). As long as the samples we communicate obey this property, any choices we make about how obtain x, have further side effects. Each subsequent denoising step can be greedily optimized to obtain 2; using as little information as possible. We use this fact optimize the timestep schedule of our denoising process as follows:
To communicate x, efficiently, we wish to choose a timestep schedule S S li, .j... dinal which minimizes the expected number of bits needed communicate Xtems Let C(i,j) be the coding "cost" (e.g. required bits) of sending x, given that x, has already been received. For example, using Algorithm| C(. j) I + log(I) + 5, where entire timestep schedule S is:
(qxx, x,, x.) ||((x,))))))))))))) x))) So the cost of an
1 C(S) Σ (C()), E=1
Fortunately, j(i,j) does not depend on how x; was communicated. By the nature of our compres- sion algorithm, each x; we send is statistically indistinguishable from a random sample of q(x;(xx), X Therefore, optimizing the hyperparameters of Algorithm [ is a problem with optimal substructure. Let Copli, [i, j] be the minimum cost (however defined) sending a noisy sample Xt, ~ q(xi, Xt, Xo) given a shared sample Xx; g(xx, Then:
Copll [i min; Clú, klopt Clk,
Therefore, we can use greedy algorithm to find the timestep schedule which will require the min- imum expected number of bits communicate x,. We optimize timestep schedule for set of
5

---

Published as a conference paper at ICLR 2025
Algorithm 3 Optimal DiffC Timestep Schedule
Require:
p: reverse process specified by the diffusion model
q: diffusion forward process
X: dataset of images to encode (c.g. Kodak 24)
C(Dk)): coding cost function, where (' p)) is the cost of sending using
Iimal timestep of the last noisy sample to communicate
Ensure: S: Minimum average cost timestep schedule (T, (imal)
1: D[x,i, + oo for all x, i,
2: for Xo E X do
3: Sample XT ~ (Xr Xu) using p(xr)
4: for T,T 1.... .......... Tinall + do
5: for j = 1, 2.... final do D[x, i, &- DRI(1(x, Xi X xx)l((xxx)))
6:
7: end for
8: end for
9: end for
10: Cli,j] < mean mean(C(Dixx i,5l) for X)
11: S + ShortestPath(C, T, final)
Find shortest path from T to Finall in adjacency matrix C
12: return
images X: for the Kodak dataset, X consists of all 24 images. For Div2K we choose random sample of 30 images. See Algorithm 3
Note that Algorithm| requires | forward passes through the diffusion model, only need to perform one forward pass for each new X,. Given the model's noise prediction 6, there is simple closed-form equation to compute p(xy) x;) for any j.
3.3 DENOISING X;
To evaluate DifIC at different bitrates, we generate lossy reconstructions from noisy samples x, at t E [900, 800,8...... ... 100, .0.....0)) 10).
Once we have communicated the noisy sample x,, we have several options as to how to denoise it. Theis al. (2022) proposed "DiffC-A" (ancestral sampling) and "DiffC-F" (probability flow based sampling). Ancestral sampling simply continues the stochastic DDPM denoising process to arrive fully denoised image. DiffC-F denoises X, by following the probability flow ODE. As far as we can tell, DiffC-F is strictly superior, in that requires fewer inference steps, and results in lower distortion than DiffC-A. Therefore, adopt DiffC-F as our main denoising method. For simplicity we follow the probability flow with standard 50-step DDIM scheduler (Song et al.) 2020).
In Appendix A.2 we consider third denoising method, which simply predicts Xo from X, with -aaaa
single forward pass through the diffusion model. While there are a-priori reasons for this method to seem interesting, we did find it valuable in practice.
3.4
ADAPTING DIFFC FOR FLUX
was originally formulated for Diffusion Denoising Probabilistic Models (DDPMs) (Ho et al. but Flux-dev is trained with Optimal Transport Flow Matching (OT flow) (Limman et al. Fortunately, there is a simple transformation that converts OT flow probability paths into DDPM probability paths, allowing us treat Flux-dev exactly like DDPM. For details, see Ap-
DiffC
2020
2022
pendix A.3
3.5
FURTHER IMPLEMENTATION DETAILS
Fixed Bitrate Per Step: To efficiently communicate random sample from q using p. the sender and receiver must pre-establish ( (gllp). But Dx (9l()) varies with the contents of the encoded image. To establish complete compression protocol, the sender and receiver must establish the

---

Published as a conference paper at ICLR 2025
Table 1: Parameter count and average encoding and decoding times (in seconds) on an A40 GPU for Kodak/Div2k- 2024 images. DiffC encoding/decoding times depend on the number of denoising steps taken, which can vary from roughly 10- 150 for encoding, and from 50-150 for decoding, de- pending on the bitrate and reconstruction quality. Encoding is slightly slower per step than decoding due to the asymmetry reverse-channel coding.
Method
Params
Encoding (s)
Decoding (s)
MS-ILLM (Kodak)
181M
1.6
2.9
MS-ILLM (Div2K)
181M
2.3
4.8
HiFiC (Div2k)
181M
1.5
3.8
PerCo (Kodak)
950M
0.1
3.6
DINEIC (Kodak)
950M
0.2
6.6
DifIC (SD 1.5) (Kodak)
943M
0.6-9.4
2.9-8.5
DifIC (SD 2.1) (Kodak)
950M
0.6-8.8
2.8-8.2
DiffC (SDXL) (Div2k)
2.6B
3,2-46.2
15.3-36.6
DifCC (Flux) (Div2k)
12B
21.0-321.4
98.5-215.0
KI divergence of the distribution being reverse-channel coded in each step. In principle, you could develop an entropy coding model for this side-information. But practice, we have found that just hard-coding sequence of expected Dx values into the protocol based their averages does not affect the performance our method too much. In fact, we were surprised at the extent to which the R-D curves robust variations in the protocol's DEL and timestep schedules. See Appendix for further results.
SDXL Base Refiner: SDXL consists of two models, base model and "refiner". which is spe- cialized for the last 200 of the 1,000 denoising steps (Podell et al. 2023). We found no meaningful difference between the compression rates achieved by the base and refiner models of SDXI. in the high signal-to- est improvement in CLIP and quality scores. See Figure for comparison. Therefore we elected to use the refiner for the last 20% of the denoising process as Podell et al. (2023) recommends.
ratio domain. However, using the refiner during reconstruction leads to a mod-
4 RESULTS
4.1 LATENT DIFFUSION LIMITS FIDELITY
Algorithms T and allow for exact communication of a sample 2g. However, with latent diffusion models, Xo does not represent the original image, but its latent representation. Therefore, the fidelity of our compression algorithm is limited by the reconstruction fidelity of these models' variational autoencoders. On the Kodak dataset, SD1 AD1.5/2.1 1's variational autoencoder achieves an average PSNR of 25.7 dB, while SDXL's VAE obtains 28.8 dB. Therefore. Stable Diffusion- primarily useful in the ultra-low bitrate regime, where average PSNRs under 25 dB are competitive with the state of the art. However, Flux's 's VAE is much higher fidelity, obtaining 32.4 dB PSNR on the Kodak dataset. Therefore Flux is capable of much higher fidelity compression, and remains competitive with HiFiC 0.46 bits per pixel. Figure didsplays distortion results in reference this VAE bound.
compression
4.2 RATE-DrEL SITEE-DITTRTION CURVES
Here, we evaluate the rate-distortion curves for our DiffC algorithm under a variety of settings. Fig-
ure 2 shows our primary results, evaluating the performance of our methods on Kodak and DIV2K images with Stable Diffusion 1.5, 2.1, XL. and Flux-dev. We compare against HiFiC (Mentzer et al.)
2020), MS-ILLM (Muckley et al. 2023), DiffEIC (Li et al. 2024). PerCo (Careil et al. | (024), and
VVC-intra (Wieckowski et al.). Distortion metrics used are PSNR, LPIPS (Zhang al. 2018). 2021 Q-Align (Wu et al. 2023 is no-reference image quality metric which believe is meaningful than Frechet Inception Distance (Heusel et al. 2017
and CLIP scores (Hessel et al.
given our relatively small datasets. Figure 3 gives a visual comparison of DiffC to other generative compression methods.

---

Published as a conference paper at ICLR 2025
SD 1.5
5D 1,5 prompppp
502.1 prompt
KS ILLM
WCC SD1. VAEE
SD 1.5 typothetical
2:1
PerCo
PSNR
LPIPS
CLIP Score
Q-Align
100
27.5
25.0
0.6
Kodak
22.5
0.4
70
0.002
0.01 0.02
0.002
0.03
0.02
2.05
0.002
0.01 0.02
0.01 0.02
0.05 0.1
Bits
Bits per pise!
Bits per pixel
SDXL
SDXL hypothetical
Flox hypothetical
HIRE WC
SDXL
SDXL (no refiner)
Flux
MS-ILLM
Flux VAE
PSNR
LPIPS
CLIP Score
Q-Align
30
0.6
DIV2K-1024
25
0.4
15
0.002
1.0
0.002
0.01
Bits per pixel
Bits per pixe!
Bits per pixel
Bits per pixel
Figure 2: Rate-distortion curves for generative compression methods across three sets of images. "Hypothetical" refers hypothetically-optinal RD curves assuming ideal reverse-channel coding. Best viewed zoomed in.
From these and other data make the following qualitative observations:
The differences between the RD curves of different flagship diffusion models is relatively modest: SD1.5 and 2.1 have comparable performance on the Kodak dataset, while SDXL and Flux have comparable performance on the Div2K dataset. The primary advantage of Flux appears to be that it is capable of much higher bitrate compression, and higher reconstruction quality. We speculate that larger and more sophisticated diffusion models may be asymptotically approaching the "true" log-likelihood of the data, so that larger diffusion models will yield in increasingly small bitrate improvements.
Hypothetically ideal reverse-channel coding yields better R-D curves than practical imple- mentation. Improvements to our reverse-channel coding scheme might help realize these models' full compressive potential.
We do not find prompts to be worth their weight in bits. We tried both BLIP captions (Li et al., 2022) and Hard Prompts Made Easy (Wen et al., 2024). We found that prompt guidance scale near was optimal for communicating the noisy latent, and denoising with guidance scale around 5 was optimal for maximizing CLIP scores. For both methods, we found that you would be better off just allocating those bits directly to DiffC without prompt conditioning. While Figure 2 only shows the effect of prompt conditioning for Stable Diffusion 1.5, the effect for 2.1 was almost identical. For Flux and SDXL, prompt conditioning offered no improvement to distortion metrics, but did significantly improve reconstruction quality at lower bitrates (see Appendix Figure 7
DiffC achieves competitive rate-distortion curves against other state-of-the- generative com- pression methods, and has place of the atte/distortion/percepiii Pareto frontier. It obtains lower distortion than PerCo or DiffEIC, and achieves higher perceptual quality than HiFiC and MS- ILLM. Of course its other advantage is that DiffC works zero-shot and is naturally progressive, while all other methods compared here require specialized training for each new bitrate.
4.3
VS. NUMBER OF INFERENCE STEPS
We analyze the tradeoff between the number of inference steps in our denoising schedule vs. the rate- distortion curve. To determine the best denoising timestep schedule, we use greedy optimization
8

---

Published as a conference paper at ICLR 2025
PerCoo (SD v2.1)
DifCC (SD v2.1)
Ground Truth
0.0029 hpp
0.031 bpp
0.0030 bpp
0.032 bpp
HiFC 0.14 bpp
MS-ILIM 0.14 bpp
DiRC (SDXL.)
DifIC (Flux-dev)
Ground
0.12 bpp
0.14 bpp
Figure 3: Visual comparison of generative compression methods.
163 stops; sec.
54 stops: 3.4 Sbc.
20 steps, 1.1
12 steps: 0,7 sec.
PSNR 1
LPIPS
CLIP Score
Q-Align
0.6
0.5
90
Kodak
20
0.3
18
70
0.1
per pixel
per pixel
per pixel
per
Figure 4: RD curves Kodak dataset number of RCC steps. Legend shows encoding time per image in seconds.
technique that minimizes the total number of bits needed reach target timestep For more details, see Algorithm 3 Figure rate-distortion curves for SD1.5 on the Kodak dataset using between 12 and 163 total steps. (We do indicates that that 163 steps is bitrate-optimal, given the reverse-channel coding overhead.) Notice that the number of steps can be reduced to 54 with only minor degradation quality. Table T shows the number of model parameters and the encoding/decoding times for each compression method.
more steps, since our greedy optimization technique
5 CONCLUSION
By solving several challenges surrounding reverse-channel coding, we introduce the first practical implementation of DiffC. We evaluate our implementation using Stable Diffusion 1.5, 2. XL, and Flux-dev. Our method is competitive with other generative compression methods on rate/distor- tion/perception curves. Our method outperforms PerCo and DiffEIC on rate-distortion curves, and exceeds HiFiC and MS-ILLM perceptual image quality. Compared to other generative compres- sion methods, which typically must be trained for each new bitrate, DiffC can be applied zero-shot

---

Published as a conference paper at ICLR 2025
to pretrained diffusion models to compress images any bitrate. However, the slowness of diffusion inference, and the large number required denoising steps, remains a significant bottleneck.
6 FUTURE WORK
We believe there is a wide frontier of future work to be done on DiffC applications. We will conclude by highlighting four potential directions: inference speed, Rectified Flow models, flexible image sizes, and conditional diffusion models.
Inference time: Despite our alleviation of the RCC bottleneck, these compression algorithms are still quite slow. The DifiC algorithm requires least tens of inference steps, and large models such as Flux
require multiple seconds per inference step, even powerful GPUs. For Flux, this can result in an encoding time more than five minutes per image. However, we remain optimistic that there many potential avenues faster compression. As example, diffusion activation caching (Moura et al. 2019 Ma et al. 2024) could eliminate redundant computation between denoising steps.
Rectified Flow models: DiffC is designed for DDPM-baaed diffusion models, and can be easily adapted to Optimal Transport Flow models (see Appendix (A.3). (Lipman et al. 2002).. However, Sfate-of-the-ae diffusion models such as Stable Diffusion (Esser al.) and Flux moving towards rectified flow (Liu et al.] 2022). It is not obvious how to adapt DiffC to leverage rectified flow diffusion models. But for this class of compression algorithms remain relevant the most state-of sfate-of-the-are the-art diffusion models, this limitation must somehow be overcome.
Flexible image sizes: Our compression method inherits Stable Diffusion's 's image size limitations: performance degrades quickly for image sizes outside the training distribution. Extending the capabilities of pretrained diffusion models to image sizes outside their training distribution is an active area of research (Haji-Ali et al. 2024). and progress in generation may be applicable to compression well.
Conditional Diffusion Models: Another clear avenue for future work is to combine DiffC with information condition the diffusion process. Following Lei et al. (2023). we have tried condi- tioning on prompts generated by Hard Prompts Made Easy (Wen et al. 2024). But this could be taken much further. For example, one could try using an entropy model to directly compress an
image's ' CLIP embedding. As mentioned in Section 2. it could also be interesting to apply DiffC on top of other conditional diffusion compression methods, such as PerCo (Careil al. 2024) or CDC (Yang & Mandt 2024).
REFERENCES
VTM reference software for VVC. URL https::/vegiit hhi. fraunhofer wereoofeeedddeeeeeee jvet/ VVCSoftware VTM
Marlene Careil, Matthew J. Muckley, Jakob Verbeek, and Stéphane Lathuilière. Towards image compression with perfect realism at ultra-low bitrates. In The Twelfth International Confer- ence Learning Representations, 2024. URL https: /ttp//pppevvveeeeeeeeeeeeeeeeeee net. forum?id
htttp:://operrviiie
dtc9UU9JBc
Hyungjin Chung, Jeongsol Kim, Michael T Mecann, Mare L Klasky, and Jong Chul Ye. Diffusion posterior sampling for general noisy inverse problems. arXiv preprint arXiv:2209. 2022.
Noam Elata, Tomer Michaeli, and Michael Elad. Zero-shot image compression with diffusion- posterior sampling, 2024. URL. htttps://arxis org/ http:/bb//2200 09896
Patrick Esser, Sumith Kulal, Andreas Blattmann, Rahim Entezari, Jonas Müller, Harry Saini, Yam Levi, Dominik Lorenz, Axel Sauer, Frederic Boesel, et al. Scaling rectified flow transformers for high-resolution image synthesis, march 2024. URL http://rxiv. org/abs/2403.08 000oo///220003066..
Daniel Geng, Inbum Park, and Andrew Owens. Factorized diffusion: Perceptual illusions by noise decomposition. In European Conference on Computer Vision (ECCV), 2024a. URL hetps:
//arxiv..rr//ab//2222 11615
10

---

Published as a conference paper at ICLR 2025
Daniel Geng, Inbum Park, and Andrew Owens. Visual anagrams: Generating multi-view optical illusions with diffusion models. In Conference on Computer Vision and Pattern Recognition (CVPR), 2024b. BtLLhttss///axtt org. abs//311- 17919
Moayed Haji-Ali, Guha Balakrishnan, and Vicente Ordonez. Elasticdiffusion: Training- free ar- bitrary size image generation through global-local content separation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 6603-6612, 2024.
Marton Havasi, Robert Peharz, and José Miguel Hermandez-Lobato Minimal random code learning: Getting bits back from compressed model parameters. 2019.
Jack Hessel, Ari Holtzman, Maxwell Forbes, Ronan Le Bras, and Yejin Choi. Clipscore: A reference- evaluation metric for image captioning. arXiv preprint arXiv:2104.08718, 2021.
Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochre- iter. Gans trained by two time-scale update rule converge to a local nash equilibrium. In I. Guyon, U. Von Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Gamett (eds.), Advances in Neural Information Processing Systems, volume 30. Curran Associates, Inc., 2017. URL https: http:///oooeeeeeeeeeoc www.eeeriie. cttpapprrcppppppcceppeeeeeeee fileff /ftee/////e////// http:///....aa..eeeeeoooeeeeee12
Jonathan Ho, Ajay Jain, and Pieter Abheel. Denoising diffusion probabilistic models. In H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin (eds.), Advances in Neural Information Processing Systems, volume 33, pp. 6840-6851. Curran Asso-
ciates, Inc., 2020. URL https://procedinng sitroceedingg.neuripss... httppp/cccca..oetttsts.cc/eeppap #c5bcfec8584
ab101 /9ca4b------ Papeaaaaaaaapppppppp pdf
Paper.
Zhaoyang Jia, Jiahao Li, Bin Li, Houqiang Li, and Yan Lu. Generative latent coding for ultra-low bitrate image compression. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 26088-20222 260888, June 2024.
Gu Jinjin, Cai Haoming, Chen Haoyu, Ye Xiaoxing, Jimmy S Ren, and Dong Chao. Pipal: large- scale image quality assessment dataset for perceptual image restoration. In Computer Vision- ECCV 2020; 16th European Conference, Glasgow, UK, August 23-28. 2020, Proceedings, Part XI 16. pp. 633-651. Springer, 2020.
Diederik Kingma, Tim Salimans, Ben Poole, and Jonathan Ho. Variational diffusion models. Ad- vances in neural information processing systems, 14:21696-21707 2021.
Eric Lei, Yigit Berkay Uslu, Hamed Hassani, and Shirin Saeedi Bidokhti. Text+ sketch: Image compression at ultra low rates. arXiv preprini arXiv:2307.0194. 2023.
Junnan Li, Dongxu Li, Caiming Xiong, and Steven Hoi. Blip: Bootstrapping language-image pre-training for unified vision-language understanding and generation, 2022. URL https:
//arxiv.org abs /alp//2201.2 12086
Zhiyuan Li, Yanhui Zhou, Hao Wei, Chenyang Ge, and Jingwen Jiang. Towards extreme image compression with latent feature guidance and diffusion prior. IEEE Transactions on Circuits and Systems for Video Technology, 2024. doi: 10.1109TCSV..302.345555777
Yaron Lipman, Ricky TQ Chen, Heli Ben-Hamu, Maximilian Nickel, and Matt Le. Flow matching for generative modeling. arXiv preprint 17X:0:2210.022777, 2022.
Xingchao Liu, Chengyue Gong, and Qiang Liu. Flow straight and fast: Learning generate and transfer data with rectified flow. arXiv preprint arXiv:2209.0300 2022.
Xinyin Ma, Gongfan Fang, and Xinchao Wang. Deepcache: Accelerating diffusion models for free. In Proceedings of the IEFE/CVF Conference on Computer Vision and Pattern Recognition, pp- 15762-15772. 2024.
Fabian Mentzer, George Toderici, Michael Tschannen, and Eirikur Agustsson. High-fidelity gener- ative image compression. arXiv preprint 2oXiv:2006.09955, 2020.

---

Published as a conference paper at ICLR 2025
David Minnen, Johannes Ballé, and George D Toderici. Joint autoregressive and hierarchical priors for learned image compression. Advances in neural information processing systems, 31, 2018.
Giovane CM Moura, John Heidemann, Ricardo de O Schmidt, and Wes Hardaker. Cache me if you can: Effects of dns time-to-live. In Proceedings of the Internet Measurement Conference, pp. 101-115, 2019.
Matthew Muckley, Alaaeldin El-Nouby, Karen Ullrich, Hervé Jégou, and Jakob Verbeek. Im- proving statistical fidelity for neural image compression with implicit local likelihood models. In International Conference Machine Learning, pp. 25426-25443. PMLR, 2023.
Dustin Podell, Zion English, Kyle Lacey, Andreas Blattmann, Tim Dockhorn, Jonas Müller, Joe Penna, and Robin Rombach. Sdxl: Improving latent diffusion models for high-resolution image synthesis. arXiv preprint arXiv:2307.01952. 2023.
Ben Poole, Ajay Jain, Jonathan T. Barron, and Ben Mildenhall. Dreamfusion: Text-to-3d using 2d diffusion. arXiv, 2022.
Jascha Sohl-Dickstein, Eric Weiss, Niru Maheswaranathan, and Surya Ganguli. Deep unsupervised learning using nonequilibrium thermodynamics. In International conference on machine learn- ing, pp. 2256-2265. PMLR, 2015.
Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. arXiv preprint arXiv:2010.02502 2020.
L. Theis, T. Salimans, M. Hoffman, and Mentzer. Lossy compression with gaussian diffusion. arXiv:2206.08889, 2022.
/20r0g//ab///2..... 2206. 08889
Lucas Theis and Noureldin Y Ahmed. Algorithms for the communication of samples. In Interna- tional Conference on Machine Learning, pp. 21308-21328. PMLR, 2022.
Yuxin Wen, Neel Jain, John Kirchenbauer, Micah Goldblum, Jonas Geiping, and Tom Goldstein. Hard prompts made easy: Gradient-based discrete optimization for prompt tuning and discovery. Advances in Neural Information Processing Systems, 36, 2024.
Adam Wieckowski, Jens Brandenburg, Tobias Hinz, Christian Bartnik, Valeri George, Gabriel Hege, Christian Helmrich, Anastasia Henkel, Christian Lehmann, Christian Stoffers, Ivan Zupancic, Benjamin Bross, and Detlev Marpe. Vvene: An open and optimized encoder implementation. In Proc. IEEE International Conference on Multimedia Expo Workshops (ICMEW), pp. 1-2. doi: 10. 20L1109/1CMEW55226222000... 94559444.
Wikipedia. Stable diffussio - Wikipedia, the free encyclopedia, 2024. URL https: //en.
wi wikipedia wwtp/o/gggw/ii.oo
Diffusion [Online; accessed 18 (cptember-2024]
Haoning Wu, Zicheng Zhang, Weixia Zhang, Chaofeng Chen, Chunyi Li, Liang Liao, Annan Wang, Erli Zhang, Wenxiu Sun, Qiong Yan, Xiongkuo Min, Guangtai Zhai, and Weisi Lin. Q-align: Teaching Imms for visual scoring via discrete text-defined levels. arXiv preprint arXiv:2312.1 12312.170990 17090, 2023. Equal Contribution by Wu, Haoning and Zhang, Zicheng. Project Lead by Wu, Haoning. Corresponding Authors: Zhai, Guangtai and Lin, Weisi.
Tongda Xu, Ziran Zhu, Dailan He, Yanghao Li, Lina Guo, Yuanyuan Wang. Zhe Wang, Hong- wei Qin, Yan Wang, Jingjing Liu, and Ya-Qin Zhang. Idempotence and perceptual image com- pression. In The Twelfth International Conference on Learning Representations, 2024, URL
net /t
Ruihan Yang and Stephan Mandt. Lossy image compression with conditional diffusion models. Advances in Neural Information Processing Systems, 36, 2024.
Lvmin Zhang, Anyi Rao, and Maneesh Agrawala. Adding conditional control text-to-image diffusion models, 2023.
Richard Zhang, Phillip Isola, Alexei A Efros, Eli Shechtman, and Oliver Wang. The unreasonable effectiveness deep features as perceptual metric. In CVPR, 2018.
12

---

Published as a conference paper at ICLR 2025
Metric
SRCC 1
PSNR
0,406
SDL.5 VAE
0.574
LPIPS
0,584
SDXL VAE
0.600
Flux VAE
0.656
Table SRCC against user mean opinion scores on the PIPAL dataset.
A APPENDIX
A.I THE POISSON FUNCTIONAL REPRESENTATION (PFR) REVERSE- CHANREL CODING ALGORITHM.
Algorithm 4 PFR, Theis & Ahmed (2022)
Require: 9, Wmin
1: n, 0,1,00
2: repeat
3:
f- si simulate(n,p)
Candidate generation b Poisson process
4: +- + expon(m, 1)
5:
t. p(2)/q(z)
P Candidate's 's score
6:
if < S then
Accept/reject candidate
7: 8", n + 8, n
8: end if
9:
10:
until
S* S t Wmin
11:
return n
this algorithm, simulate shared pseudorandom generator that takes a random seed n and distribution p and draws a pseudorandom sample p.
A.2 DIFCCMMSE
We originally considered an additional denoising method: "DiffC-MSE". This is a 1-step denoiser which simply predicts Ry from x, with single forward-pass through the diffusion model. Xo minimizes the expected distortion against the true Xo. as measured by euclidean distance in the diffusion model's latent space.
This is an interesting property because we find that euclidean distance in the latent spaces of SD 1.5/2.1, XL. and Flux correlate surprisingly well with perceptual similarity. We used the PIPAL dataset (Jinjin et al., 2020) to evaluate the correlation of VAE latent distance with users' perceptual similarity rankings using Spearman's 's rank correlation coefficient (SRCC). shows the results.
Therefore, it stands to reason that DiffC-MSE may yield reconstructions with lower perceptual dis- tortion than either DiffC-A DiffC-F,, at the expense of realism. However, in our experiments, we found that "DiffC-MSEE was almost universally inferior to DiffC-F along all distortion metrics. Hence its relegation to the appendix.
A.3 TRANSFORMING OPTIMAL TRANSPORT FLOW INTO STANDARD GAUSSIAN DIFFUSION
Flux-dev is trained with Optimal Transport Flow (OT flow) Lipmmn et al. 2022). To apply the DiffC algorithm, we must first transform it into a standard DDPM. Fortunately, DDPM and OT flow probability paths equivalent under simple transformation.
13

---

Published as a conference paper at ICLR 2025
Both DDPM and OT flow objectives can be formulated as trying predict the expected original signal &o given a noisy sample x = 8x80 + n,c, where e ~ (O,,).
For OT flow models:
X4 = (1 - -0,)20 + ore
For DDPMs:
X; = Vāxxo vI āye
where a₁ 11- - 3r and ar II 0a.
Therefore, to translate between DDPM and OT flow model inputs, we just need to:
1. Translate the timestep: Convert the DDPM timestep tDDPM into the corresponding OT flow timestep for (or vice-versa) with the same signal-to-noise ratio:
(1-
VOLDDPM
too
V1
ŌtDDPM
Rescale the input: Rescale the noisy image by a scalar so that it has the expected magnitude:
c(1 ctorotor) = VoDDDPN
To apply DifíC to Flux dev, we simply translate the noisy latent and timestep into DDPM format to apply the DiffC algorithm, and then back into OT flow format when performing inference with Flux.
A.4
VISUAL RESULTS
Here are Kodak reconstructions at five different bitrates using Stable Diffusion 1.5. These images all generated using our actual implementation of DiffC, except for the final image in Figure ?? which shows reconstructions with the theoretically ideal version of the algorithm.
A.5 CUDA KERNEL FOR RCC
In RCC, we generate a sequence of pseduorandom samples z and randomly select one with probability proportional to p(2)/q(z). The number of samples exponentially with the encoding length, so it is important that each sample be evaluated as efficiently as possible. our implementation, we take advantage the fact that our distributions isotropic Gaussians with known variance to simplify w to:
must evaluate increases
exp
In PyTorch, compute the weight of each sample in the following way:
def compute log w (mu K, generator, batch size=256)
log ws []
for in range (K // bateh_size);
log_ws. append (torch. randn size, len (mu_g), generator= generator) B mu_q)
return torch. cat [lg__ss
With a custom CUDA kernel, we can avoid materializing the samples in memory, making the caleu- lation more efficient:
14

---

Published as a conference paper at ICLR 2025
quarter mear
hair moan
min
mean
Dynamic
max
twico moan
quadruplc moan
PSNR
LPIPS J
CLIP Score
Q-Alinn
0.8
4.5
Kodak
0.4
4.0
20
0.3
18
16
0.3
per pixel
per pixc!
Bits per pixe!
per pixe!
Figure 5: R-D curves for Stable Diffusion 1.5 on the Kodak dataset with various ways determine the Drr. per step.
global_ void compute_log_w const float* mu_g, int dim, unsigned long long K,
unsigned long long shared_seed, float* _og_w W
int idx blockIdx.x blockDim.x threadIdx.xi if (idx K) return;
10
11
curandstate state; curand ssoared_scod, 0, idx dim, astate);
13
14
float log_v_value 0.0f;
15
for (int i 0, i < dim; i++)
16
float sample_vail vaauee surand_normal (astate);
Log_w_value += sample sample_valuetnte valuetmu_qliil
18
19
20
log_w[idx] log_w_value;
A.6 VARYING DKL PER STEP
At each step of the reverse channel coding process, must communicate ((0,-112,,,0) zo) using p(z)-1/z;)). With ideal reverse channel coding, this should require (Kz(qlp) bits. However, both the sender and receiver must agree in advance this DEL value before we communicate the noisy sample using the PFR algorithm (Algorithm 4 This could be communicated as side information, compressed with its entropy model. Instead, we find that these DKL per step values can just be hard-coded with very little impact on the performance of the DiffC algorithm. To choose the hard-coded sequence of DxL. values, first run DifIC for each image in our dataset while allowing Dµ per step to be determined by the actual value of DxI at each step. Then, instead of using those dynamically generated values, fix the DEL values to their mean values across all images. We also try using the minimum Dkl step across all images, and the maximum. To explore a wider range of values, we also try several multiples of the mean DxL schedule: one quarter the mean value, 1/2 the mean value, twice the mean value, and 4x the mean value. Figure 5 shows the R-D curves from hard-coding the DeL per step schedule to these different values. We find that hard-coding DRL values lower than the actual KL divergence results in surprisingly little distortion. Better understanding of the impact of the DKL schedule on the DiffC algorithm remains an important objective for future research.
A.7 PERFORMANCE ON
IMAGE SIZES
Figure 6 demonstrates the challenge of trying to compress images which are outside of the model's
training distribution. Stable Diffusion 2.1 is trained on images from 512x768 to 768x768 px in size (such the Kodak dataset), while SDXL is trained images around 1024x 1024 pixels (such
15

---

Published as a conference paper at ICLR 2025
SDXL
50 2.1
PSNR T
LPIPS
CLIP Score
Q-Align
0.6
4.4
Kodak
22
0.5
85
20
75
3.8
70
0.002
0.01
Bits per pixel
Bits per pixe!
Bits per pixe!
Bits per pixel
SDXL
5D 2.1
PSNR
LPIPS
CLIP Score
Q-Align
4.5
95
DIV2K-1024 100-1024
0.5
90
0.4
85
80
75
0.2
70
Bits per pixel
Bits per pixe!
Bits per pixel
Bits per pixel
Figure 6: Rate-distortion curves for Stable Diffusion 2.1 and SDXL on Kodak vs. Div2k-1024 datasets.
S.XL
Flux
SIDX
Flux m/ prompt
PSNR
LPIPS
CLIP Score
Q-Align
30.0
0.5
100
DIV2X-1044
0.4
0.3
94
22.5
4.4
20.0
0.2
17.
0.1
88
0.2
0.0
0.002
0.01
0.1
1.0
0.002
0.01
0.1
3.0
0.002
0.01
0.1
1.0
0.002
0.01
0.1
1.0
Bits
Bits
Bits pixe!
Bits pixe!
Figure 7: Rate-distortion curves for unconditional vs. prompt- guipt- reconstructions with SDXL and Flux. The prompt offers no improvement in distortion (PSNR, LPIPS, CLIP scores), but does considerably improve visual quality (Q-Align).
as the Div2k-1024 dataset). Rate/distorriin//erceptti curves demonstrate a strong "home-field" advantage of these models for the resolutions they trained for. While all other metrics follow this trend, the R-D curve for PSNR on the Div2k dataset is strangely better for SD 2.1. This is one example of few counterintuitive PSNR results which we observe but cannot explain.
A.8 FLUX SDXL R-D CURVES WITH PROMPT CONDITIONING
16


Reviews:
[{"review_id": "Tx4rxFNIPP", "reviewer": "Reviewer_kfZ4", "summary": "This submission presents a set of lossy compression method based on DiffC, with pretrained StableDiffusion1.5,2.1, and XL. To enable this utilization, a solution to the reverse-channel coding problem is proposed. This submission is also the first one to publicly release an implementation of the DiffC algorithm.", "questions": "Could you present the comparisons of this idea with other SOTA image compression methods?", "limitations": "", "rating": 5, "confidence": 2, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "The paper implemented the SOTA stable diffusion for single image compression usage, and also released an implementation of DiffC to the public.", "weaknesses": "First, I don't think the main contribution is significant. The major idea had already been proposed several years ago. \nSecond, I failed to find the comparisons of this idea with other SOTA image compression methods.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "cZx82qxHTO", "reviewer": "Reviewer_zBvU", "summary": "This paper aims to solve the reverse channel coding remaining in DiffC and to accelerate the diffusion-based compression method under 10 seconds without additional training.", "questions": "In Section 4.1, the prompts do not support image compression or reconstruction. This result contradicts many previous prompt-guided image compression methods. The author should add more analysis and explain why they do not work.", "limitations": "", "rating": 6, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "Originality: The authors try their best to implement the DiffC and apply it to existing pre-trained diffusion models, such as Stable Diffusion 1.5, 2, and XL. In addition, they propose a greedy optimization technique to speed up the diffusion process and to select the best denoising timestep schedule.\n\nQuality: The manuscript is well organized and written.\n\nClarity: The authors have explained their method in detail.\n\nSignificance: The significance of this work is profound because it addresses the remaining problems in the DiffC algorithm, such as inference time and reverse-channel coding.", "weaknesses": "(1)The manuscript looks more like a technology report than an academic paper.\n\n(2)The authors do not provide quantitative comparisons with state-of-the-art extreme image compression methods (VQ-based methods, diffusion-based methods) and show the advantages of the proposed method.  \n\n(3) Although the main contribution of the paper is the implementation of the DiffC algorithm, the author should provide the model complexity of the proposed method (e.g., network parameters, FLOPs, encoding/decoding time, etc.).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "sH1MtuiGZg", "reviewer": "Reviewer_2xTu", "summary": "The paper applies the DiffC algorithm to pre-trained Stable Diffusion models (1.5, 2.1, and XL) for high-fidelity image compression at low bitrates. This approach leverages reverse-channel coding (RCC) to efficiently steer the denoising process toward realistic image reconstructions without additional training. By optimizing the denoising schedule and utilizing CUDA acceleration, the method achieves competitive reconstruction performance and compression times (under 10 seconds) compared to other state-of-the-art compression methods.", "questions": "1. On page 2, line 100, the description of CDC appears to be inaccurate. CDC is trained from scratch rather than by fine-tuning conditional diffusion models.\n\n2. In Section 4.2, why is there no R-D curve provided to illustrate the comparison between Stable Diffusion XL Base and Refiner?\n\n3. In Section 3.5, it is mentioned, ``But in practice, we have found that just hard-coding a sequence of expected DKL values into the protocol based on their averages does not affect the performance of our method too much.`` Does this mean that the $D_{KL}$ values used in practice are manually set? Could the authors provide empirical results (e.g., R-D curves) to support the conclusion that this strategy does not affect the performance too much? Additionally, what impact would increasing or decreasing the preset $D_{KL}$ values have on performance? Could RD trade-offs be achieved by adjusting the $D_{KL}$ values?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The paper extends DiffC to stable diffusion, provides an open-source implementation, and accelerates RCC with CUDA. These contributions promote the practicality of the method and pave the way for further exploration.\n\n2. The paper is well written and easy to follow. Enough background information is provided to general readers.\n\n3. The paper provides guidance on potential research directions in the Future Work section, offering insights for subsequent studies.", "weaknesses": "1. The paper lacks sufficient innovation and resembles more of an engineering improvement on existing methods.\n\n2. The diffusion model requires multiple inference steps for both encoding and decoding, resulting in significant computational overhead.\n\n3. Although the authors claim that their method is competitive with other state-of-the-art compression methods, no comparisons are provided in the paper. To substantiate this claim, the authors should compare their method with existing approaches such as PerCO [1], DiffEIC [2], and MS-ILLM [3] in terms of compression performance (e.g., PSNR, MS-SSIM, LPIPS, FID, etc.) and computational complexity (e.g., encoding/decoding time), presenting corresponding results in the paper.\n\n[1] Marlene Careil, Matthew J. Muckley, Jakob Verbeek, and St{\\'e}phane Lathuili{\\`e}re. Towards image compression with perfect realism at ultra-low bitrates. In The Twelfth International Conference on Learning Representations, 2024\n\n[2] Zhiyuan Li, Yanhui Zhou, Hao Wei, Chenyang Ge, and Jingwen Jiang. Towards extreme image compression with latent feature guidance and diffusion prior. IEEE Transactions on Circuits and Systems for Video Technology, 2024.\n\n[3] Matthew J Muckley, Alaaeldin El-Nouby, Karen Ullrich, Herv{\\'e} J{\\'e}gou, and Jakob Verbeek. Improving statistical fidelity for neural image compression with implicit local likelihood models. In International Conference on Machine Learning, 2023.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "36SfjV6PVq", "reviewer": "Reviewer_5GS8", "summary": "The paper presents a novel application of pretrained diffusion models, specifically Stable Diffusion versions 1.5, 2.1, and XL, for lossy image compression using the DiffC algorithm. The authors introduce practical workarounds to overcome challenges in reverse-channel coding, enabling efficient image compression and decompression within 10 seconds without additional training. The method is shown to be competitive with state-of-the-art compression techniques at low bitrates (0.005-0.05 bpp), demonstrating its efficacy and potential in real-world applications.", "questions": "1)  How does the performance of your method vary with image sizes that deviate from the training distribution of Stable Diffusion? Are there any strategies you are considering to mitigate performance degradation for such cases?\n2) Can you provide a more detailed comparison between your method and other state-of-the-art compression techniques, particularly in terms of computational efficiency, quality metrics, and bitrate?\n3) In the leftmost plot of Figure 3, could the authors explain why the performance of the 163-step is worse than that of the 54-step?", "limitations": "", "rating": 5, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "1) The paper introduces an innovative use of pretrained diffusion models for image compression, expanding the utility of these models beyond their typical generative applications.\n2) The proposed method achieves competitive performance with state-of-the-art compression techniques at low bitrates, showcasing its practical value.\n3) Implementation details and optimizations, such as the use of CUDA for reverse-channel coding, significantly enhance the efficiency and feasibility of the approach.\n4) The publicly available implementation of the DiffC algorithm promotes transparency and facilitates further research and development in this area.", "weaknesses": "1) The method's performance is constrained by the fidelity limits of the Stable Diffusion models' variational autoencoders, affecting compression quality at higher bitrates.\n2) The reverse-channel coding process, while optimized, still presents computational challenges, particularly for very large or very small DKL values.\n3) The paper does not extensively explore the combination of DiffC with other conditional diffusion approaches, which could potentially enhance performance further.\n4) The reliance on Stable Diffusion's image size limitations may affect the generalizability of the method to images outside the training distribution.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)