# A 3D Object Detection Based on Multi-Modality Sensors of USV
## 4 Experiments and Results
### Evaluation on the KITTI Validation Set  
We evaluated our network against the KITTI 3D object detection benchmarks. The training set provided 7481 images, and the test set provided 7518 images. We followed [PERSON] et al. [31] to divide the training set into a training set and validation set in a 1:1 ratio. We concentrated our experiments on the category of cars. To facilitate the evaluation, we followed KITTI's simple, moderate, and difficult classification methods to evaluate and compare our network.  
1. Evaluation of Object Detection For 3D overlap standard, in the radar-based method, we used the IoU threshold of 0.7 to conduct the 3D object detection. Because our model focuses on BEV and images, we also compared it with f-point net, while comparing MV3D and AVOD. The results are shown in Table 1. The results are significantly more than 10% higher than MV3D in terms of average precision. The results are also superior to AVOD in the easy and moderate mode. On the hard mode, however, the results are weaker. However, it can be tolerated in practice. We also carried out 3D detection in the BEV, and the results are shown in Table 2. AVOD did not release its results, thus we could not compare our results with AVOD's results. Our model was superior to MV3D in all aspects. Our model also performed better than f-point net in easy and hard mode. Inspired by AVOD, we also included orientation estimation in the network. We compared the Average Orientation Similarity (AOS). In AVOD, they are called Average Heading Similarity (AHS). As shown in Table 3, our model had better results than AVOD.  
\begin{table}
\begin{tabular}{c c c c} \hline \hline
**Method** & **Easy** & **Moderate** & **Hard** \\ \hline MV3D & 71.29 & 62.68 & 56.56 \\ F-Point & 83.76 & 70.92 & 63.65 \\ AVOD & 84.41 & 74.44 & 68.65 \\ Ours & 88.19 & 74.54 & 65.86 \\ \hline \hline \end{tabular}
\end{table}
Table 1: 3D detection performance: Average precision (AP) (in %) for 3D boxes.  
\begin{table}
\begin{tabular}{c c c} \hline \hline
**Method** & **Easy** & **Moderate** & **Hard** \\ \hline MV3D & 86.55 & 78.10 & 76.67 \\ F-Point & 88.16 & 84.02 & 76.44 \\ Ours & 88.73 & 79.07 & 77.89 \\ \hline \hline \end{tabular}
\end{table}
Table 2: Bird’s eye view detection performance: Average precision (AP) (in %) for BEV boxes.  
Runtime and Memory Requirements  
We used several parameters to assess the computational efficiency and the memory requirements of the proposed network. Our object detection architecture employed roughly 52.04 million parameters. We significantly reduced the number of parameters compared with the method in [6], the second stage detection network of which has three fully connected layers. Because we chose the Resnet-50 with more layers, our parameter number was higher than the method in [12]. Each frame was processed in 0.14 s on Titan X and 0.12 s in 2080 TI. The inference time of the network for each picture was 90 ms at 2080 TI.
4. Ablation Studies As shown in Tables 4 and 5, we first compared our proposed Deep Fusion Detection Network with early fusion method. In the same case that our modified Resnet-50 feature extractor was used, the average precision of our Deep Fusion Detection Network in 3D boxes and BEV boxes was higher than that of the earlier fusion method. Especially in BEV boxes, our networks performed best, about 10% better. To study the contribution of our proposed feature extractor, we replaced the Resnet-50 with the VGG16 used in [6, 12] for comparison. Our test results were also slightly higher than VGG16.