  Detecting dynamic objects and predicting static road information such as
drivable areas and ground heights are crucial for safe autonomous driving.
Previous works studied each perception task separately, and lacked a collective
quantitative analysis. In this work, we show that it is possible to perform all
perception tasks via a simple and efficient multi-task network. Our proposed
network, LidarMTL, takes raw LiDAR point cloud as inputs, and predicts six
perception outputs for 3D object detection and road understanding. The network
is based on an encoder-decoder architecture with 3D sparse convolution and
deconvolution operations. Extensive experiments verify the proposed method with
competitive accuracies compared to state-of-the-art object detectors and other
task-specific networks. LidarMTL is also leveraged for online localization.
Code and pre-trained model have been made available at
https://github.com/frankfengdi/LidarMTL.
