# A 3D Object Detection Based on Multi-Modality Sensors of USV
## 1 Introduction  
In the new era of ocean observations, Unmanned Surface Vehicles (USVs) are of vital significance in scientific investigation, ocean monitoring and disaster relief [1]. Currently, most applications of USVs, such as collision avoidance and navigation, heavily rely on manual operation. Achievement of reliable, autonomous, all-weather marine object detection and characterization can be highly beneficial if the capability for autonomous collision avoidance of USVs could be realized. Meanwhile, other advanced tasks such as port surveillance require semantic reconstruction of the environment. Accurate object detection could significantly improve the performance of autonomous navigation and related advanced tasks. However, existing methods of object detection face some challenges in USVs' operating environment.  
Previous methods cannot achieve satisfied results on detection and characterization of the sea surface objects of USVs, such as single visual-based object detection and multi-model based object detection methods. Single visual-based object detection has poor performance in identifying objects on sea surface. It often fails to achieve good results [2; 3; 4]. In ground environment, several cases using multi-model based object detection methods on UGVs have proven their effectiveness [5; 6], but these methods cannot be applied in marine environment because the operating environment of USVs lacks abundant signs. Moreover, modern USVs are usually equipped with a variety of sensors, such as camera and lidar, which produce additional information, such as depth and density. In this condition,3D object detection based on point cloud generated by lidar is time-consuming. It cannot meet the real-time requirement of USVs.  
This paper proposes a 3D object detection Depth Neural Network (DNN) based on multi-modality data of USVs. This new model containstwo significant features. Firstly, this model converts the raw point cloud to Bird's Eye View (BEV). This reduces the computation requirements of the algorithm and csn work smoothly in lower memory spaces of computer. This model contains a deep network that can generate area proposals to overcome the problems of low-resolution object in images and BEV. In this deep network, a modified Resnet [7] is utilized to extract the feature map of images and BEV. Secondly, this model includes a deep fusion detection network. This fusion detection network improves the accuracy of object detection on the sea surface.  
We tested our model on the KITTI dataset and a self-built datasets. Experimental results show that our model can achieve better results than other methods did in the USV's operating environment. The final performance of object detection on the sea surface has been greatly improved. The new model has apparent advantages in performance and accuracy, even in small memory conditions.  
The rest of the paper is organized as follows: after reviewing the related work in Section 2, we introduce the proposed object detection architecture in Section 3. Then, we provide a thorough experimental result in Section 4. Finally, the paper is concluded in Section 5.