# Towards Deep Radar Perception for Autonomous Driving: Datasets, Methods, and Challenges
## 9 Conclusions  
The purpose of this review article is to provide a big picture of deep radar perception. We first summarised the principles of radar signal processing. Then, we presented a detailed summary of radar datasets for autonomous driving. To encourage researchers to build their own datasets, we also presented methods for calibration and labelling. We further investigated data augmentation and synthetic radar data to improve data diversity. Radar can be used for depth completion and velocity estimation. For the ease of depth completion, several expansion methods were introduced to better associate the radar detections with the image pixels. The full velocity can be recovered as a geometric optimisation problem or in a self-supervised learning way.  
Radar detection was the main focus of this paper. We classified deep radar detectors into point-cloud-based and pre-CFAR-based. PointNet variant networks and multi-view encoder-decoders are popular choices for radar point clouds and radar tensors, respectively. By increasing spatial density and exploiting temporal information, significant performance improvements can be achieved. Some new operators, such as depthwise convolution, attention, and graph convolution, are leveraged for larger receptive field. In practical applications, radar is often fused with cameras and LiDAR. We classified fusion frameworks into four categories. Input fusion requires a lightweight preprocessing to explicitly handle radar position imprecision. Cascaded ROI fusion is not robust to sensor failures, while parallel ROI fusion improves it. Feature map fusion provides the network with greater flexibility to combine radar and visual semantics, but requires specific training techniques for effective learning. Decision fusion takes advantage of modal redundancy and is therefore popular in real-world applications. Location information can be robustly fused in a track-to-track architecture or with the help of network semantics. Category information can be fused with Bayesian inference or evidence theory.  
We summarised three challenges for deep radar perception. Firstly, multi-path effects need to be explicitly considered in object detection. Secondly, we need to alleviate the problem of overconfidence in radar classification and estimate the uncertainty in bounding box regression. Thirdly, the fusion architecture should have adaptive mechanisms to take full advantage of radar's all-weather capabilities. Finally, some future research directions were proposed. There is an urgent need for high-quality radar datasets. Radar domain knowledge and uncertainty quantification can help us to develop a generalizable AI model. Considering the perceptual system as a whole, we can extend the end-to-end learning framework forward, i.e., joint learning with interference mitigation, or backward, i.e., predicting motion.  
Conceptualisation, Y.Z., L.L. and H.Z.; investigation, Y.Z.; writing--original draft preparation, Y.Z.; writing--review and editing, Y.Z.; visualisation, Y.Z.; supervision, M.L.-B., L.Y. and Y.Y.; project administration, Y.Y. All authors have read and agreed to the published version of the manuscript.  
This research was partially funded by the XJTLU-JITRI Academy of Industrial Technology, the Institute of Deep Perception Technology (IDPT), and the Research Enhancement Fund of XJTLU (REF-19-01-04).  
Not applicable.  
The authors declare no conflict of interest.