## Vehicle Detection for Unmanned Systems Based on Multimodal Feature Fusion  
**[PERSON], [PERSON] and [PERSON]** \({}^{\star}\)  
College of Mechanical Engineering, Southeast University, Nanjing 211189, China; [EMAIL_ADDRESS] (Y.W. ); [EMAIL_ADDRESS] (H.L.) \({}^{\star}\) Correspondence: [EMAIL_ADDRESS]  
**Abstract:** This paper proposes a 3D vehicle-detection algorithm based on multimodal feature fusion to address the problem of low vehicle-detection accuracy in unmanned system environment awareness. The algorithm matches the coordinate relationships between the two sensors and reduces sampling errors by combining the millimeter-wave radar and camera calibration. Statistical filtering is used to remove redundant points from the millimeter-wave radar data to reduce outlier interference; a multimodal feature fusion module is constructed to fuse the point cloud and image information using pixel-by-pixel averaging. Moreover, feature pyramids are added to extract fused high-level feature information, which is used to improve detection accuracy in complex road scenarios. A feature fusion region proposal structure was established to generate region proposals based on the high-level feature information. The vehicle detection results were obtained by matching the detection frames in their vertices after removal of the redundant detection frames using non-maximum suppression. Experimental results from the KITTI dataset show that the proposed method improved the efficiency and accuracy of vehicle detection with the corresponding average of 0.14 s and 84.71%. **Keywords:** millimeter-wave radar; environmental awareness; multimodal fusion; vehicle detection; unmanned vehicle systems  
**1. Introduction**  
With the continued development and depth of artificial intelligence and robotics, unmanned systems have become a research hotspot [1; 2; 3]. Unmanned driving systems contribute to path planning and decision control of the subject through three-dimensional vehicle detection, in which multimodal feature fusion of radar, camera, GPS and other sensors is an important element in three-dimensional vehicle detection [4; 5; 6]. Besides, multimodal feature fusion of a millimeter-wave radar and camera is beneficial for realizing unmanned driving in complex traffic environments because of its high detection resolution and accuracy, high interference immunity, wide sensing range and freedom from light and shadow occlusion [7; 8; 9]. In recent years, vehicle detection algorithms based on multimodal feature fusion have been rapidly developed and a large number of excellent algorithms have been widely employed [10; 11; 12]. [PERSON] et al. [13] used multimodal fusion deep neural networks to layer features from different modalities in multiple channels and extracted multichannels in the hidden layer. The feature tensor was extracted in the hidden layer to achieve feature fusion, and then to predict vehicle position and steer angle and speed. [PERSON] et al. [14] interpolated the point cloud according to the normalized pixel-distance weighted average and fused it with the pixel points to assign feature channel weights through an attention mechanism to suppress interference channels and enhance vehicle feature channel information. [PERSON] et al. [15] used a random sample consistency algorithm to locate and calibrate the sparse point cloud, extract the point cloud structure, align it directly with the pixels to avoid the accumulation of errors from multiple coordinate conversions and calibration and finally determine the vehicle position based on matching the target corner points with the point cloud position in the image. [PERSON] et al. [16] first rasterized the point cloud in real time to filter the pavement information, then expanded the features to fan detection cells according to the coarse-grained features of the road to reduce the interference of pavement texture, and lastly fused the obstacle-detection results by a 3D occupancy raster with an octree. [PERSON] et al.