World-in-World: World Models in a Closed-Loop World
Authors: name: Jiahan Zhang, name: Muqing Jiang, name: Nanru Dai, name: Taiming Lu, name: Arda Uzunoglu, name: Shunchi Zhang, name: Yana Wei, name: Jiahao Wang, name: Vishal M. Patel, name: Paul Pu Liang, name: Daniel Khashabi, name: Cheng Peng, name: Rama Chellappa, name: Tianmin Shu, name: Alan Yuille, name: Yilun Du, name: Jieneng Chen

Generative world models (WMs) can now simulate worlds with striking visual
realism, which naturally raises the question of whether they can endow embodied
agents with predictive perception for decision making. Progress on this
question has been limited by fragmented evaluation: most existing benchmarks
adopt open-loop protocols that emphasize visual quality in isolation, leaving
the core issue of embodied utility unresolved, i.e., do WMs actually help
agents succeed at embodied tasks? To address this gap, we introduce
World-in-World, the first open platform that benchmarks WMs in a closed-loop
world that mirrors real agent-environment interactions. World-in-World provides
a unified online planning strategy and a standardized action API, enabling
heterogeneous WMs for decision making. We curate four closed-loop environments
that rigorously evaluate diverse WMs, prioritize task success as the primary
metric, and move beyond the common focus on visual quality; we also present the
first data scaling law for world models in embodied settings. Our study
uncovers three surprises: (1) visual quality alone does not guarantee task
success, controllability matters more; (2) scaling post-training with
action-observation data is more effective than upgrading the pretrained video
generators; and (3) allocating more inference-time compute allows WMs to
substantially improve closed-loop performance.