NitroGen: An Open Foundation Model for Generalist Gaming Agents
Authors: name: Loïc Magne, name: Anas Awadalla, name: Guanzhi Wang, name: Yinzhen Xu, name: Joshua Belofsky, name: Fengyuan Hu, name: Joohwan Kim, name: Ludwig Schmidt, name: Georgia Gkioxari, name: Jan Kautz, name: Yisong Yue, name: Yejin Choi, name: Yuke Zhu, name: Linxi "Jim" Fan
Organization: NVIDIA

We introduce NitroGen, a vision-action foundation model for generalist gaming agents that is trained on 40,000 hours of gameplay videos across more than 1,000 games. We incorporate three key ingredients: 1) an internet-scale video-action dataset constructed by automatically extracting player actions from publicly available gameplay videos, 2) a multi-game benchmark environment that can measure cross-game generalization, and 3) a unified vision-action model trained with large-scale behavior cloning. NitroGen exhibits strong competence across diverse domains, including combat encounters in 3D action games, high-precision control in 2D platformers, and exploration in procedurally generated worlds. It transfers effectively to unseen games, achieving up to 52% relative improvement in task success rates over models trained from scratch. We release the dataset, evaluation suite, and model weights to advance research on generalist embodied agents.