GPUDrive: Data-driven, multi-agent driving simulation at 1 million FPS
Authors: name: Saman Kazemkhani, name: Aarav Pandya, name: Daphne Cornelisse, name: Brennan Shacklett, name: Eugene Vinitsky

Multi-agent learning algorithms have been successful at generating superhuman
planning in a wide variety of games but have had little impact on the design of
deployed multi-agent planners. A key bottleneck in applying these techniques to
multi-agent planning is that they require billions of steps of experience. To
enable the study of multi-agent planning at this scale, we present GPUDrive, a
GPU-accelerated, multi-agent simulator built on top of the Madrona Game Engine
that can generate over a million steps of experience per second. Observation,
reward, and dynamics functions are written directly in C++, allowing users to
define complex, heterogeneous agent behaviors that are lowered to
high-performance CUDA. We show that using GPUDrive we are able to effectively
train reinforcement learning agents over many scenes in the Waymo Motion
dataset, yielding highly effective goal-reaching agents in minutes for
individual scenes and generally capable agents in a few hours. We ship these
trained agents as part of the code base at
https://github.com/Emerge-Lab/gpudrive.