VideoLCM: Video Latent Consistency Model
Authors: name: Xiang Wang; name: Shiwei Zhang; name: Han Zhang; name: Yu Liu; name: Yingya Zhang; name: Changxin Gao; name: Nong Sang
Published: 2023-12-14 16:45:36

Abstract: Consistency models have demonstrated powerful capability in efficient image
generation and allowed synthesis within a few sampling steps, alleviating the
high computational cost in diffusion models. However, the consistency model in
the more challenging and resource-consuming video generation is still less
explored. In this report, we present the VideoLCM framework to fill this gap,
which leverages the concept of consistency models from image generation to
efficiently synthesize videos with minimal steps while maintaining high
quality. VideoLCM builds upon existing latent video diffusion models and
incorporates consistency distillation techniques for training the latent
consistency model. Experimental results reveal the effectiveness of our
VideoLCM in terms of computational efficiency, fidelity and temporal
consistency. Notably, VideoLCM achieves high-fidelity and smooth video
synthesis with only four sampling steps, showcasing the potential for real-time
synthesis. We hope that VideoLCM can serve as a simple yet effective baseline
for subsequent research. The source code and models will be publicly available.