  Self-driving vehicles need to anticipate a diverse set of future traffic
scenarios in order to safely share the road with other traffic participants
that may exhibit rare but dangerous driving. In this paper, we present LookOut,
an approach to jointly perceive the environment and predict a diverse set of
futures from sensor data, estimate their probability, and optimize a
contingency plan over these diverse future realizations. In particular, we
learn a diverse joint distribution over multi-agent future trajectories in a
traffic scene that allows us to cover a wide range of future modes with high
sample efficiency while leveraging the expressive power of generative models.
Unlike previous work in diverse motion forecasting, our diversity objective
explicitly rewards sampling future scenarios that require distinct reactions
from the self-driving vehicle for improved safety. Our contingency planner then
finds comfortable trajectories that ensure safe reactions to a wide range of
future scenarios. Through extensive evaluations, we show that our model
demonstrates significantly more diverse and sample-efficient motion forecasting
in a large-scale self-driving dataset as well as safer and more comfortable
motion plans in long-term closed-loop simulations than current state-of-the-art
models.
