ICML 2025

Modulated Diffusion:
Accelerating Generative Modeling with Modulated Quantization
Weizhi Gao 1 Zhichao Hou 1 Junqi Yin 2 Feiyi Wang 2 Linyu Peng 3 Xiaorui Liu 1
Abstract
Diffusion models have emerged as powerful gen-
erative models, but their high computational cost
in iterative sampling remains a significant bot-
tleneck. In this work, we present an in-depth
and insightful study of state-of-the-art accelera-
tion techniques for diffusion models, including
caching and quantization, and reveal their limita-
tions in computation error and generation quality.
To break these limits, this work introduces Modu-
lated Diffusion (MoDiff), an innovative, rigorous,
and principled framework that accelerates gener-
ative modeling through modulated quantization
and error compensation. MoDiff not only inherits
the advantages of existing caching and quantiza-
tion methods but also serves as a general frame-
work to accelerate all diffusion models. The ad-
vantages of MoDiff are supported by solid theoret-
ical insight and analysis. In addition, extensive ex-
periments on CIFAR-10 and LSUN demonstrate
that MoDiff significantly reduces activation quan-
tization from 8 bits to 3 bits without performance
degradation in post-training quantization (PTQ).
Our code implementation is available at https:
//github.com/WeizhiGao/MoDiff.
1. Introduction
Diffusion models have emerged as powerful generative mod-
els for producing high-quality data samples, ranging from
images to audio and beyond (Ho et al., 2020; Song et al.,
2021a;b). These models work by iteratively transforming
a simple noise distribution into complex, structured out-
puts, guided by a learned reverse diffusion process. Despite
their effectiveness, diffusion models come with significant
1Department of Computer Science, North Carolina State Uni-
versity 2National Center for Computational Science, Oak Ridge
National Lab 3Department of Mechanical Engineering, Keio Uni-
versity. Correspondence to: Xiaorui Liu <xliu96@ncsu.edu>.
P...