OmniWorld: A Multi-Domain and Multi-Modal Dataset for 4D World Modeling
Authors: name: Yang Zhou, name: Yifan Wang, name: Jianjun Zhou, name: Wenzheng Chang, name: Haoyu Guo, name: Zizun Li, name: Kaijing Ma, name: Xinyue Li, name: Yating Wang, name: Haoyi Zhu, name: Mingyu Liu, name: Dingning Liu, name: Jiange Yang, name: Zhoujie Fu, name: Junyi Chen, name: Chunhua Shen, name: Jiangmiao Pang, name: Kaipeng Zhang, name: Tong He

The field of 4D world modeling - aiming to jointly capture spatial geometry
and temporal dynamics - has witnessed remarkable progress in recent years,
driven by advances in large-scale generative models and multimodal learning.
However, the development of truly general 4D world models remains fundamentally
constrained by the availability of high-quality data. Existing datasets and
benchmarks often lack the dynamic complexity, multi-domain diversity, and
spatial-temporal annotations required to support key tasks such as 4D geometric
reconstruction, future prediction, and camera-control video generation. To
address this gap, we introduce OmniWorld, a large-scale, multi-domain,
multi-modal dataset specifically designed for 4D world modeling. OmniWorld
consists of a newly collected OmniWorld-Game dataset and several curated public
datasets spanning diverse domains. Compared with existing synthetic datasets,
OmniWorld-Game provides richer modality coverage, larger scale, and more
realistic dynamic interactions. Based on this dataset, we establish a
challenging benchmark that exposes the limitations of current state-of-the-art
(SOTA) approaches in modeling complex 4D environments. Moreover, fine-tuning
existing SOTA methods on OmniWorld leads to significant performance gains
across 4D reconstruction and video generation tasks, strongly validating
OmniWorld as a powerful resource for training and evaluation. We envision
OmniWorld as a catalyst for accelerating the development of general-purpose 4D
world models, ultimately advancing machines' holistic understanding of the
physical world.