V-IRL: Grounding Virtual Intelligence in Real Life
Authors: name: Jihan Yang; name: Runyu Ding; name: Ellis Brown; name: Xiaojuan Qi; name: Saining Xie
Published: 2024-02-05 18:59:36

Abstract: There is a sensory gulf between the Earth that humans inhabit and the digital
realms in which modern AI agents are created. To develop AI agents that can
sense, think, and act as flexibly as humans in real-world settings, it is
imperative to bridge the realism gap between the digital and physical worlds.
How can we embody agents in an environment as rich and diverse as the one we
inhabit, without the constraints imposed by real hardware and control? Towards
this end, we introduce V-IRL: a platform that enables agents to scalably
interact with the real world in a virtual yet realistic environment. Our
platform serves as a playground for developing agents that can accomplish
various practical tasks and as a vast testbed for measuring progress in
capabilities spanning perception, decision-making, and interaction with
real-world data across the entire globe.