  In autonomous driving (AD), accurately predicting changes in the environment
can effectively improve safety and comfort. Due to complex interactions among
traffic participants, however, it is very hard to achieve accurate prediction
for a long horizon. To address this challenge, we propose prediction by
anticipation, which views interaction in terms of a latent probabilistic
generative process wherein some vehicles move partly in response to the
anticipated motion of other vehicles. Under this view, consecutive data frames
can be factorized into sequential samples from an action-conditional
distribution that effectively generalizes to a wider range of actions and
driving situations. Our proposed prediction model, variational Bayesian in
nature, is trained to maximize the evidence lower bound (ELBO) of the
log-likelihood of this conditional distribution. Evaluations of our approach
with prominent AD datasets NGSIM I-80 and Argoverse show significant
improvement over current state-of-the-art in both accuracy and generalization.
