  In this paper, we address the important problem in self-driving of
forecasting multi-pedestrian motion and their shared scene occupancy map,
critical for safe navigation. Our contributions are two-fold. First, we
advocate for predicting both the individual motions as well as the scene
occupancy map in order to effectively deal with missing detections caused by
postprocessing, e.g., confidence thresholding and non-maximum suppression.
Second, we propose a Scene-Actor Graph Neural Network (SA-GNN) which preserves
the relative spatial information of pedestrians via 2D convolution, and
captures the interactions among pedestrians within the same scene, including
those that have not been detected, via message passing. On two large-scale
real-world datasets, nuScenes and ATG4D, we showcase that our scene-occupancy
predictions are more accurate and better calibrated than those from
state-of-the-art motion forecasting methods, while also matching their
performance in pedestrian motion forecasting metrics.
