Title: The Accumulation of Score Estimation Error in Diffusion Models
Primary area: generative models
Keywords: Diffusion models, Score estimation error

Abstract: Diffusion models are widely used for high-quality generation, but their performance is sensitive to the accuracy of the estimated score.  Our main results are established first in the setting where the forward process is initialized from a Gaussian mixture, where we derive Wasserstein bounds by leveraging the structure of the score and its Hessian. We then extend the analysis to general data distributions, where we provide a more general but looser upper bound. Our analysis reveals how discretization steps directly shape the accumulation of score estimation error, thereby explaining previously observed empirical phenomena regarding the advantage of certain discretization schedules. In addition, we show that SDE samplers accumulate less error than ODE samplers in the small step-size regime, which explains their superior empirical performance in this setting. The results apply to both variance-preserving (VP) and variance-exploding (VE) diffusions.

TL;DR: We analyze how score estimation error accumulates in diffusion models and provide Wasserstein bounds between the final and target distributions, explaining empirical differences across discretization schedules and samplers.