  Autonomous driving consists of a multitude of interacting modules, where each
module must contend with errors from the others. Typically, the motion
prediction module depends on a robust tracking system to capture each agent's
past movement. In this work, we systematically explore the importance of the
tracking module for the motion prediction task and ultimately conclude that the
tracking module is detrimental to overall motion prediction performance when
the module is imperfect (with as low as 1% error). We explicitly compare models
that use tracking information to models that do not across multiple scenarios
and conditions. We find that the tracking information only improves performance
in noise-free conditions. A noise-free tracker is unlikely to remain noise-free
in real-world scenarios, and the inevitable noise will subsequently negatively
affect performance. We thus argue future work should be mindful of noise when
developing and testing motion/tracking modules, or that they should do away
with the tracking component entirely.
