OSV: One Step is Enough for High-Quality Image to Video Generation
Authors: name: Xiaofeng Mao; name: Zhengkai Jiang; name: Fu-Yun Wang; name: Wenbing Zhu; name: Jiangning Zhang; name: Hao Chen; name: Mingmin Chi; name: Yabiao Wang
Published: 2024-09-17 17:16:37

Abstract: Video diffusion models have shown great potential in generating high-quality
videos, making them an increasingly popular focus. However, their inherent
iterative nature leads to substantial computational and time costs. While
efforts have been made to accelerate video diffusion by reducing inference
steps (through techniques like consistency distillation) and GAN training
(these approaches often fall short in either performance or training
stability). In this work, we introduce a two-stage training framework that
effectively combines consistency distillation with GAN training to address
these challenges. Additionally, we propose a novel video discriminator design,
which eliminates the need for decoding the video latents and improves the final
performance. Our model is capable of producing high-quality videos in merely
one-step, with the flexibility to perform multi-step refinement for further
performance enhancement. Our quantitative evaluation on the OpenWebVid-1M
benchmark shows that our model significantly outperforms existing methods.
Notably, our 1-step performance(FVD 171.15) exceeds the 8-step performance of
the consistency distillation based method, AnimateLCM (FVD 184.79), and
approaches the 25-step performance of advanced Stable Video Diffusion (FVD
156.94).