# MPVF: Multi-Modal 3D Object Detection Algorithm with Pointwise and Voxelwise Fusion
## 3 Methodology
### IRFP Module  
To address the resolution of 375 \(\times\) 1242 for three-channel camera images, we designed a more expressive feature extraction module, IRFP, as shown in Figure 2. IRFP improves the feature extraction strategy of RetinaNet [36] by optimizing the feature pyramid structure and incorporating group convolutions, thereby enhancing multi-scale feature representation. Unlike YOLO's single-stage detection framework [26], IRFP employs multi-stage feature optimization, improving small object detection performance, particularly in complex scenarios. This module consists of the IR module and the FP module.  
Figure 1: Overall Network Architecture of MPVF.  
In the standard data augmentation process, random flipping and random noise addition are applied to the original RGB images with a resolution of 375 \(\times\) 1242. Meanwhile, the shortest side of the image is scaled to 600 pixels while maintaining the aspect ratio, resulting in the RGB image size being adjusted to 600 \(\times\) 1988 \(\times\) 3. This preprocessing step is designed to ensure that the model is trained with a uniform input size, thereby improving computational efficiency and enhancing the stability of the model's performance. The IR module is an enhancement of ResNet-101 [39], containing five convolutional layers (C1 to C5). The height and width of the feature map output by layers C1 to C5 are sequentially halved, with the number of channels being 64, 256, 512, 1024, and 2048, respectively. Layers C2 to C5 consist of 3, 4, 23, and 3 residual blocks, respectively. Taking the first residual block of C3 as an example, the input feature map has 256 channels. After a series of operations involving 1 \(\times\) 1 convolution, 3 \(\times\) 3 convolution, and another 1 \(\times\) 1 convolution, the output channel counts become 128, 128, and 512, respectively. The convolution operations within the residual block are divided into 32 groups, with each group performing convolution independently, followed by concatenation of the results. This grouped convolution strategy not only reduces computational complexity but also enhances the model's expressive capability.  
The FP module is constructed on the output feature maps of the C3 to C5 layers in the IR module. It modifies the feature map's height and width through 2\(\times\) upsampling in a bottom-up manner, while a 1 \(\times\) 1 convolution adjusts the number of channels. The results from these two paths are then summed and processed through a 3 \(\times\) 3 convolution to  
Figure 2: Specific Structure of the IRFP Module.  
yield the P3 to P5 feature maps, effectively incorporating high-level semantic information from the upper layers and high-resolution details from the lower layers. To optimize computational resource utilization, the FP module does not integrate features from the C2 layer, as this would significantly increase computational complexity due to the P2 feature maps produced by the C2 layer. A 3 \(\times\) 3 convolution with a stride of 2 is applied to the P5 layer to generate the P6 feature map, while the P7 feature map is obtained by applying another 3 \(\times\) 3 convolution with a stride of 2 on the P6 layer. Both P6 and P7 are downsampled from P5, aiming to enlarge the receptive field and capture global information from the image. The FP module effectively captures fine-grained features at varying resolutions, enhancing the model's detection performance in complex scenes.