Title: Regularization can make diffusion models more efficient
Primary area: learning theory
Keywords: diffusion models, Regularization

Abstract: Diffusion models are one of the key architectures of generative AI. Their main
drawback, however, is the computational costs. This study indicates that the
concept of sparsity, well known especially in statistics, can provide a pathway to
more efficient diffusion pipelines. Our mathematical guarantees prove that sparsity
can reduce the input dimension’s influence on the computational complexity to that
of a much smaller intrinsic dimension of the data. Our empirical findings confirm
that inducing sparsity can indeed lead to better samples at a lower cost.

TL;DR: Our mathematical guarantees prove that sparsity can reduce the input dimension’s influence on the computational complexity to that of a much smaller intrinsic dimension of the data.