  Estimating the states of surrounding traffic participants stays at the core
of autonomous driving. In this paper, we study a novel setting of this problem:
model-free single object tracking (SOT), which takes the object state in the
first frame as input, and jointly solves state estimation and tracking in
subsequent frames. The main purpose for this new setting is to break the strong
limitation of the popular "detection and tracking" scheme in multi-object
tracking. Moreover, we notice that shape completion by overlaying the point
clouds, which is a by-product of our proposed task, not only improves the
performance of state estimation but also has numerous applications. As no
benchmark for this task is available so far, we construct a new dataset
LiDAR-SOT and corresponding evaluation protocols based on the Waymo Open
dataset. We then propose an optimization-based algorithm called SOTracker based
on point cloud registration, vehicle shapes, and motion priors. Our
quantitative and qualitative results prove the effectiveness of our SOTracker
and reveal the challenging cases for SOT in point clouds, including the
sparsity of LiDAR data, abrupt motion variation, etc. Finally, we also explore
how the proposed task and algorithm may benefit other autonomous driving
applications, including simulating LiDAR scans, generating motion data, and
annotating optical flow. The code and protocols for our benchmark and algorithm
are available at https://github.com/TuSimple/LiDAR_SOT/ . A video demonstration
is at https://www.youtube.com/watch?v=BpHixKs91i8 .
