  The availability of many real-world driving datasets is a key reason behind
the recent progress of object detection algorithms in autonomous driving.
However, there exist ambiguity or even failures in object labels due to
error-prone annotation process or sensor observation noise. Current public
object detection datasets only provide deterministic object labels without
considering their inherent uncertainty, as does the common training process or
evaluation metrics for object detectors. As a result, an in-depth evaluation
among different object detection methods remains challenging, and the training
process of object detectors is sub-optimal, especially in probabilistic object
detection. In this work, we infer the uncertainty in bounding box labels from
LiDAR point clouds based on a generative model, and define a new representation
of the probabilistic bounding box through a spatial uncertainty distribution.
Comprehensive experiments show that the proposed model reflects complex
environmental noises in LiDAR perception and the label quality. Furthermore, we
propose Jaccard IoU (JIoU) as a new evaluation metric that extends IoU by
incorporating label uncertainty. We conduct an in-depth comparison among
several LiDAR-based object detectors using the JIoU metric. Finally, we
incorporate the proposed label uncertainty in a loss function to train a
probabilistic object detector and to improve its detection accuracy. We verify
our proposed methods on two public datasets (KITTI, Waymo), as well as on
simulation data. Code is released at https://bit.ly/2W534yo.
