MotionLM: Multi-Agent Motion Forecasting as Language Modeling
Authors: name: Ari Seff; name: Brian Cera; name: Dian Chen; name: Mason Ng; name: Aurick Zhou; name: Nigamaa Nayakanti; name: Khaled S. Refaat; name: Rami Al-Rfou; name: Benjamin Sapp
Published: 2023-09-28 15:46:25

Abstract: Reliable forecasting of the future behavior of road agents is a critical
component to safe planning in autonomous vehicles. Here, we represent
continuous trajectories as sequences of discrete motion tokens and cast
multi-agent motion prediction as a language modeling task over this domain. Our
model, MotionLM, provides several advantages: First, it does not require
anchors or explicit latent variable optimization to learn multimodal
distributions. Instead, we leverage a single standard language modeling
objective, maximizing the average log probability over sequence tokens. Second,
our approach bypasses post-hoc interaction heuristics where individual agent
trajectory generation is conducted prior to interactive scoring. Instead,
MotionLM produces joint distributions over interactive agent futures in a
single autoregressive decoding process. In addition, the model's sequential
factorization enables temporally causal conditional rollouts. The proposed
approach establishes new state-of-the-art performance for multi-agent motion
prediction on the Waymo Open Motion Dataset, ranking 1st on the interactive
challenge leaderboard.