ICLR 2025

Published as a conference paper at ICLR 2025
LOSSY COMPRESSION WITH
PRETRAINED DIFFUSION MODELS
Jeremy Vonderfecht & Feng Liu
Department of Computer Science
Portland State University
{vonder2,fliu}@pdx.edu
ABSTRACT
We apply the DiffC algorithm (Theis et al., 2022) to Stable Diffusion 1.5, 2.1,
XL, and Flux-dev, and demonstrate that these pretrained models are remarkably
capable lossy image compressors. A principled algorithm for lossy compression
using pretrained diffusion models has been understood since at least (Ho et al.,
2020), but challenges in reverse-channel coding have prevented such algorithms
from ever being fully implemented. We introduce simple workarounds that lead
to the first complete implementation of DiffC, which is capable of compressing
and decompressing images using Stable Diffusion in under 10 seconds. Despite
requiring no additional training, our method is competitive with other state-of-the-
art generative compression methods at low ultra-low bitrates.
1
INTRODUCTION
Today’s AI labs are spending millions of dollars training state-of-the-art diffusion models. The
original Stable Diffusion cost around $600,000 to train, while Stable Diffusion 3 is rumored to have
cost around $10 million (Wikipedia, 2024). The sophistication and expense of these models are only
likely to increase as generative modelling expands into the video domain.
While diffusion models are used primarily for prompt-based image generation, they are also highly
powerful and flexible image priors. They provide many other affordances, such as easy multiplica-
tion with other distributions (i.e., posterior sampling), and log-likelihood estimation (Sohl-Dickstein
et al., 2015). Researchers have discovered many remarkable applications of pretrained diffusion
models, including image restoration (Chung et al., 2022), optical illusions (Geng et al., 2024b;a),
and generative 3D modeling (Poole et al., 2022).
Ho et al. (2020) proposed an algorithm by which pretrained diffusion models ca...