Mitigating the Noise Shift for Denoising Generative Models via Noise
  Awareness Guidance
Authors: name: Jincheng Zhong, name: Boyuan Jiang, name: Xin Tao, name: Pengfei Wan, name: Kun Gai, name: Mingsheng Long
Organization: THUML @ Tsinghua University

Existing denoising generative models rely on solving discretized reverse-time
SDEs or ODEs. In this paper, we identify a long-overlooked yet pervasive issue
in this family of models: a misalignment between the pre-defined noise level
and the actual noise level encoded in intermediate states during sampling. We
refer to this misalignment as noise shift. Through empirical analysis, we
demonstrate that noise shift is widespread in modern diffusion models and
exhibits a systematic bias, leading to sub-optimal generation due to both
out-of-distribution generalization and inaccurate denoising updates. To address
this problem, we propose Noise Awareness Guidance (NAG), a simple yet effective
correction method that explicitly steers sampling trajectories to remain
consistent with the pre-defined noise schedule. We further introduce a
classifier-free variant of NAG, which jointly trains a noise-conditional and a
noise-unconditional model via noise-condition dropout, thereby eliminating the
need for external classifiers. Extensive experiments, including ImageNet
generation and various supervised fine-tuning tasks, show that NAG consistently
mitigates noise shift and substantially improves the generation quality of
mainstream diffusion models.