# Radar Voxel Fusion for 3D Object Detection
## 4 Results
### Sensor Fusion  
Table 2 shows the results of the proposed model with different input sensor data. The networks have been trained several times to rule out that the different AP scores are causedby random effects. The lidar baseline outperforms the radar baseline by a great margin. This is expected as the data density and accuracy of the lidar input data are higher than that of the radar data.  
The fusion of camera RGB and lidar data does not result in an increased detection accuracy for the proposed network. We assume that this is due to the increased complexity that the additional image data brings into the optimization process. At the same time, the additional color feature does not distinguish vehicles from the background, as the same colors are also widely found in the environment.  
The early fusion of radar and lidar data increases the network performance against the baseline. The fusion of all three modalities increases the detection performance by a greater margin for most of the evaluated data sets. Only for night scenes, where the camera data deteriorates most, does the fusion of lidar and radar outperform the RVF-Net. Example detection results in the BEV perspective from the lidar, RGB input, and the RVF-Net input are compared in Figure 4.  
The simulated depth camera approach does not increase the detection performance. The approach adds additional input data by depth-completing the lidar points. However, the informativeness in this data cannot compensate for the increased complexity introduced by its addition.  
The absolute AP scores between the different columns of Table 2 cannot be compared since the underlying data varies between the columns. The data source has the greatest  
\begin{table}
\begin{tabular}{l c c c c} \hline \hline
**Network Input** & **nuScenes** & **Rain and Night** & **Rain** & **Night** \\ \hline Lidar & 52.18\% & 50.09\% & 43.94\% & 63.56\% \\ Radar & 17.43\% & 16.00\% & 16.42\% & 22.46\% \\ Lidar, RGB & 49.96\% & 46.59\% & 42.72\% & 61.66\% \\ Lidar, Radar & 54.18\% & 53.10\% & 47.51\% & **68.01\%** \\ Lidar, RGB, Radar **(RVF-Net)** & **54.86\%** & **53.12\%** & **48.32\%** & 67.39\% \\ Simulated Depth Cam & 48.02\% & 46.07\% & 39.07\% & 57.33\% \\ Simulated Depth Cam, Radar & 52.06\% & 48.31\% & 41.65\% & 61.04\% \\ \hline \hline \end{tabular}
\end{table}
Table 2: AP scores for different environment (data) and network configurations on the respective validation data set.  
Figure 4: BEV of the detection results: Lidar and RGB fusion in the top row. RVF-Net fusion in the bottom row. Detected bounding boxes in orange. Ground truth bounding boxes in black. Lidar point cloud in blue. Radar point cloud and measured velocity in green. The street is shown in gray. (**a**) Only RVF-Net is able to detect the vehicle from the point cloud. (**b**) RVF-Net detects both bounding boxes. (**c**) RVF-Net detects both bounding boxes. However, the detected box on the right has a high translation and rotation error towards the ground truth bounding box.  
influence for the performance of machine learning models. All models have a significantly higher scores for the night scenes split than for the other splits. This is most likely due to the lower complexity of the night scenes present in the data set.  
The relative performance gain of different input data within each column shows a valid comparison of the fusion methods since they are trained and evaluated on the same data. The radar data fusion of the RVF-Net outperforms the lidar baseline by 5.1% on the nuScenes split, while it outperforms the baseline on the rain split by 10.0% and on the night split by 6.0%. The increased performance of the radar fusion is especially notable for the rain split where lidar and camera data quality is limited. The fusion of lidar and radar is also especially beneficial for night scenes, even though the lidar data quality should not be affected by these conditions.