Open-Sora Plan: Open-Source Large Video Generation Model
Authors: name: Bin Lin; name: Yunyang Ge; name: Xinhua Cheng; name: Zongjian Li; name: Bin Zhu; name: Shaodong Wang; name: Xianyi He; name: Yang Ye; name: Shenghai Yuan; name: Liuhan Chen; name: Tanghui Jia; name: Junwu Zhang; name: Zhenyu Tang; name: Yatian Pang; name: Bin She; name: Cen Yan; name: Zhiheng Hu; name: Xiaoyi Dong; name: Lin Chen; name: Zhang Pan; name: Xing Zhou; name: Shaoling Dong; name: Yonghong Tian; name: Li Yuan
Published: 2024-11-28 14:07:45

Abstract: We introduce Open-Sora Plan, an open-source project that aims to contribute a
large generation model for generating desired high-resolution videos with long
durations based on various user inputs. Our project comprises multiple
components for the entire video generation process, including a Wavelet-Flow
Variational Autoencoder, a Joint Image-Video Skiparse Denoiser, and various
condition controllers. Moreover, many assistant strategies for efficient
training and inference are designed, and a multi-dimensional data curation
pipeline is proposed for obtaining desired high-quality data. Benefiting from
efficient thoughts, our Open-Sora Plan achieves impressive video generation
results in both qualitative and quantitative evaluations. We hope our careful
design and practical experience can inspire the video generation research
community. All our codes and model weights are publicly available at
https://github.com/PKU-YuanGroup/Open-Sora-Plan.