GameFactory: Creating New Games with Generative Interactive Videos
Authors: name: Jiwen Yu; name: Yiran Qin; name: Xintao Wang; name: Pengfei Wan; name: Di Zhang; name: Xihui Liu
Published: 2025-01-14 18:57:21

Abstract: Generative game engines have the potential to revolutionize game development
by autonomously creating new content and reducing manual workload. However,
existing video-based game generation methods fail to address the critical
challenge of scene generalization, limiting their applicability to existing
games with fixed styles and scenes. In this paper, we present GameFactory, a
framework focused on exploring scene generalization in game video generation.
To enable the creation of entirely new and diverse games, we leverage
pre-trained video diffusion models trained on open-domain video data. To bridge
the domain gap between open-domain priors and small-scale game dataset, we
propose a multi-phase training strategy that decouples game style learning from
action control, preserving open-domain generalization while achieving action
controllability. Using Minecraft as our data source, we release GF-Minecraft, a
high-quality and diversity action-annotated video dataset for research.
Furthermore, we extend our framework to enable autoregressive
action-controllable game video generation, allowing the production of
unlimited-length interactive game videos. Experimental results demonstrate that
GameFactory effectively generates open-domain, diverse, and action-controllable
game videos, representing a significant step forward in AI-driven game
generation. Our dataset and project page are publicly available at
https://vvictoryuki.github.io/gamefactory/.