  Explainability is essential for autonomous vehicles and other robotics
systems interacting with humans and other objects during operation. Humans need
to understand and anticipate the actions taken by the machines for trustful and
safe cooperation. In this work, we aim to enable the explainability of an
autonomous driving system at the design stage by incorporating expert domain
knowledge into the model. We propose Grounded Relational Inference (GRI). It
models an interactive system's underlying dynamics by inferring an interaction
graph representing the agents' relations. We ensure an interpretable
interaction graph by grounding the relational latent space into semantic
behaviors defined with expert domain knowledge. We demonstrate that it can
model interactive traffic scenarios under both simulation and real-world
settings, and generate interpretable graphs explaining the vehicle's behavior
by their interactions.
