VidGen-1M: A Large-Scale Dataset for Text-to-video Generation
Authors: name: Zhiyu Tan; name: Xiaomeng Yang; name: Luozheng Qin; name: Hao Li
Published: 2024-08-05 16:53:23

Abstract: The quality of video-text pairs fundamentally determines the upper bound of
text-to-video models. Currently, the datasets used for training these models
suffer from significant shortcomings, including low temporal consistency,
poor-quality captions, substandard video quality, and imbalanced data
distribution. The prevailing video curation process, which depends on image
models for tagging and manual rule-based curation, leads to a high
computational load and leaves behind unclean data. As a result, there is a lack
of appropriate training datasets for text-to-video models. To address this
problem, we present VidGen-1M, a superior training dataset for text-to-video
models. Produced through a coarse-to-fine curation strategy, this dataset
guarantees high-quality videos and detailed captions with excellent temporal
consistency. When used to train the video generation model, this dataset has
led to experimental results that surpass those obtained with other models.