Title: Virtual Community: An Open World for Humans, Robots, and Society
Primary area: datasets and benchmarks
Keywords: embodied AI, multi-agent, simulation

Abstract: The rapid progress of AI and robotics may profoundly transform society, as humans and robots begin to coexist in shared communities, bringing both opportunities and challenges. To explore this future, we present Virtual Community—an open-world platform for humans, robots, and society—built on a universal physics engine and grounded in real-world 3D scenes.
With Virtual Community, we aim to study embodied social intelligence at scale: 1) How robots can intelligently cooperate or compete, 2) how humans form social relations and build community, and more importantly, 3) how humans and robots can co-exist in open worlds. To support these, Virtual Community features: 1) An open-source multi-agent physics simulator that supports robots, humans, and their interactions within a society; 2) A large‑scale, real‑world aligned community generation pipeline, including vast outdoor space, diverse indoor scenes, and a community of grounded agents with rich characters and appearances. Leveraging Virtual Community, we propose two novel challenges. The Community Planning Challenge evaluates multi‑agent reasoning and planning in open‑world settings, such as cooperating to help agents with daily activities and efficiently connecting other agents. The Community Robot Challenge requires multiple heterogeneous robots to collaborate in solving complex open‑world tasks. We evaluate various baselines and demonstrate the challenges in both high‑level open‑world task planning and low‑level cooperation controls.