# MPVF: Multi-Modal 3D Object Detection Algorithm with Pointwise and Voxelwise Fusion
## 1 Introduction  
3D object detection is fundamental to cutting-edge applications such as autonomous driving [1], drones [2], and robotics [3]. These technologies depend on precise environmental perception to ensure that systems can accurately identify and locate surrounding objects, thereby facilitating safe navigation and operation. Autonomous driving, in particular, has been a major driving force behind the advancement of 3D object detection technologies. In autonomous driving systems, vehicles must continuously perceive their surroundings in real time, accurately detecting and tracking pedestrians, cyclists, vehicles, and other obstacles on the road. This real-time perception is essential for making precise drivingdecisions, ultimately ensuring safety and efficiency in dynamic traffic environments. Based on the type of input data, 3D object detection methods can be broadly classified into three categories: camera-based 3D object detection, LIDAR-based 3D object detection, and multi-modal fusion-based 3D object detection. Camera-based 3D object detection methods [4; 5] typically utilize monocular [4] or multi-view images [5] as input. While image data offers rich color and texture information, it lacks accurate depth information and 3D structural details, resulting in inherent depth deficiency [6]. Conversely, LIDAR-based 3D object detection methods [7; 8] generally use laser radar point clouds as input. Although point cloud data provide precise depth and 3D structural information, they are characterized by low resolution and insufficient texture and color information. This deficiency can lead to misidentifications when the background and target objects exhibit similar geometric structures [6]. By integrating image and point cloud data, the accuracy and robustness of detection can be significantly enhanced in complex environments; thus, the exploration of multi-modal fusion techniques for 3D object detection is of paramount importance. Within the realm of existing multi-modal fusion algorithms, the integration of camera and LIDAR point clouds is typically categorized according to the stage of fusion: early fusion, intermediate fusion, and late fusion. Early fusion methods [9; 10] merge image information prior to the entry of point cloud data into the LIDAR detection pipeline, thereby enabling subsequent detection processes to fully leverage multi-modal information. However, these methods often necessitate preliminary 2D detection or semantic segmentation of images, which must then be mapped to the point clouds, resulting in increased computational costs and inference times. Furthermore, if alignment errors occur between images and point clouds, these errors may be exacerbated in subsequent 3D object detection stages, negatively impacting the accuracy of the detection results [11]. Late fusion methods [12; 13], on the other hand, postpone the fusion of multi-modal data until the final detection stage, thereby enhancing computational efficiency; however, the lack of early interaction between modalities may lead to inadequate feature representation, thereby limiting detection performance [11]. In contrast, intermediate fusion methods [14; 15] integrate image and point cloud features within the backbone network or at intermediate stages of the detector, facilitating the generation of finer feature representations and offering greater flexibility through adjustable fusion operators and network structures based on specific requirements. These methods also exhibit superior performance in 3D detection, which justifies the adoption of an intermediate fusion strategy in our proposed model. While MVX-Net [14] enhances detection performance through image feature fusion, the interaction between local and global features remains insufficient, which may lead to decreased detection accuracy when backgrounds and targets are similar. Moreover, MVX-Net demonstrates weak fine-grained feature extraction capabilities when detecting small and complex-shaped objects.