Tracking From the Ground Up