# Multi-Layer Fusion 3D Object Detection via Lidar Point Cloud and Camera Image
## 1 Introduction  
Object detection is a fundamental and crucial task in computer vision, which is aimed at identifying and locating specific classes of objects within the environment (e.g., cars, people). It has wide-ranging applications, particularly in the field of autonomous driving. Autonomous driving has requirements in many aspects: 3D detection, fast detection speeds, low costs, and so on. In response to the demand for 3D detection, cameras and Lidar have been installed in vehicles. With the development of deep learning, using neural network methods to detect objects has become a current trend. This paper focuses on object detection technology. High-performance object detection technology is of great significance in improving the safety of automatic driving. To achieve greater autonomous driving safety, more tasks and sensors will need to work together, and the technology in this paper is one of the many key technologies in autonomous driving systems. At present, 3D object detection methods are divided into three kinds: image-based, point cloud-based and fusion-based. Images can provide vivid color and scale information and are the most commonly used data form in traditional object detection networks. Images have many advantages, such as containing continuous information, occupying small memory, having a low cost, and being easy to process. However, images are two-dimensional and cannot provide complete three-dimensional information. Lidar can provide accurate three-dimensional point cloud data, can directly obtain the spatial information of objects, can capture the details and shape of objects, and can replace the estimation of depth. It has relatively little impact on occlusion, viewing angle changes and lighting conditions and is suitable for accurate object detection in complex environments. However, the amount of Lidar data is large, the requirements for computing resources are high, and a large amountof point cloud data needs to be processed and stored. In order to make comprehensive use of the advantages of these two kinds of data, the object detection method based on fusion has become a research hotspot in recent years. In this paper, an object detection network based on multi-sensor fusion is given priority, which is the inevitable direction of object detection in the field of autonomous driving. Most fusion networks choose to fuse after extracting features from the image and point cloud data, respectively, which places higher requirements on the network; that is, it needs to retain more object information. With the deepening of the backbone network, the pixels of small objects in the feature map are decreased in number or even lost. Feature pyramid networks (FPN) [1] preserve small objects by fusing multi-scale features, but even simple additions can cause small objects' features to be overridden by large objects. Therefore, we believe that reasonable ways to guide the method of fusion are needed to preserve small object information. The reference to'small objects' in this article refers specifically to situations in autonomous driving images and Lidar data that occupy fewer pixels or points in image or point cloud data due to the relative distance between the object and the sensor. Although these objects are small in size in terms of data, they are of great importance in ensuring road safety. Aiming at the problem that the detection performance of small objects in automatic driving scenes is poor, affecting the detection accuracy of 3D objects, a multi-layer fusion 3D object detection network combining Lidar point cloud data and RGB images is proposed in this paper. This system is developed for use in autonomous vehicles to detect cars, pedestrians and cyclists. Compared with the method of using only RGB images, this method can use point cloud information to estimate the position and direction of objects more accurately, especially under poor lighting conditions.