Noise Calibration: Plug-and-play Content-Preserving Video Enhancement
  using Pre-trained Video Diffusion Models
Authors: name: Qinyu Yang; name: Haoxin Chen; name: Yong Zhang; name: Menghan Xia; name: Xiaodong Cun; name: Zhixun Su; name: Ying Shan
Published: 2024-07-14 17:59:56

Abstract: In order to improve the quality of synthesized videos, currently, one
predominant method involves retraining an expert diffusion model and then
implementing a noising-denoising process for refinement. Despite the
significant training costs, maintaining consistency of content between the
original and enhanced videos remains a major challenge. To tackle this
challenge, we propose a novel formulation that considers both visual quality
and consistency of content. Consistency of content is ensured by a proposed
loss function that maintains the structure of the input, while visual quality
is improved by utilizing the denoising process of pretrained diffusion models.
To address the formulated optimization problem, we have developed a
plug-and-play noise optimization strategy, referred to as Noise Calibration. By
refining the initial random noise through a few iterations, the content of
original video can be largely preserved, and the enhancement effect
demonstrates a notable improvement. Extensive experiments have demonstrated the
effectiveness of the proposed method.