# Multi-View Joint Learning and BEV Feature-Fusion Network for 3D Object Detection
## 3 Method  
In this section, we introduce our 3D object-detection network, which is a two-stage 3D object-detection framework based on voxel and pseudo images.  
The voxel-based two-stage 3D object-detection network generally uses the voxel CNN with sparse convolution [10; 12] as the backbone of feature extraction, directly extracts 3D features, projects the features onto the BEV, predicts 3D ROI using the RPN, and refines the ROI in the detection head to obtain the detection results. The pseudo-image-based 3D object-detection [7; 9] network uses sophisticated feature extractors to extract richer feature information, including 2D features and 3D features, for final result prediction.  
Our proposed 3D object-detection network combines the feature extraction methods of the above two networks. We use multiple methods to extract the spatial geometric features of the point clouds and then perform feature fusion. The network mainly consists of four parts: (1) the pseudo-image-based feature extraction branch; (2) the voxel-based feature extraction branch, where we propose a multiple-view feature-fusion (MVFF) module that effectively fuses the features projected on the other two views (RV and SV) into BEV features; (3) the multi-BEV feature-fusion module (M- BEV FF) which can adaptively fuse the BEV feature map generated by two branches; and (4) a detection head used to refine the detection results. The structure of our network is shown in Figure 2 and the pseudo-code of our method in PyTorch style is shown in Algorithm 1.  
```
1:\(F_{pc}\) : Point Cloud Data
2: Initialize Hyper-parameters: 4 NVIDIA Tesla T4 GPUs, Adam optimizer, Learning Rate = 0.01, Batch Size = 2, Epochs = 80;
3: Detected Objects(x,y,z,dx,dy,dz,heading-angle);
4:while train is True: # whether the work is training
5: # Point Cloud Data Preprocessing
6: \(F_{p}\) = mean(voxelization(\(F_{pc}\))), \(F_{pseudo-img}\) = mapToBev(\(F_{pc}\))
7: # Feature Extraction and Fusion
8: # Image Feature Extractor Module(2 DExtractor)
9: # Voxel Feature Extractor Module(3 DExtractor)
10: # Multi-view Feature Fusion Module(MVFF)
11: \(F_{p}^{\prime}\) = 3 DExtractor(\(F_{p}\))
12: \(F_{rev}\), \(F_{sv}\), \(F_{bev}\) = mapToXYZ(\(F_{p}^{\prime}\))
13: \(F_{bev-oxel}\) = 2 DExtractor(MVFF(\(F_{rev}\), \(F_{sv}\), \(F_{bev}\)))
14: \(F_{bev-pseudo-img}\) = 2 DExtractor(\(F_{pseudo-img}\))
15: \(F_{bev-fusion}\) = M-BEV FF(\(F_{bev-oxel}\), \(F_{bev-pseudo-img}\))
16: # Detection Head
17: prediction = refine(RPN(\(F_{bev-fusion}\)))
18: loss = \(L\)(prediction)
19: loss.backward()
20: Update()
```  
**Algorithm 1** Pseudo-code of our proposed method in PyTorch style  
Figure 2: Overview of 3D object detection based on multi-BEV feature fusion. The point cloud obtained from Lidar is input into two branches. The feature extraction branch (yellow part) based on the pseudo image supports the point cloud and encodes it, directly projects it in the Z-axis direction to obtain the BEV representation of the point cloud, and then uses the 2D trunk for feature extraction. The voxel-based feature-extraction branch (green part) discretizes the point cloud into voxels, uses the 3D trunk for feature extraction, projects the 3D features in BEV, and uses the 2D trunk to further extract features. The BEV feature maps output from the two branches are input into the multi-BEV feature map fusion module for adaptive feature fusion. The fused BEV feature map is input into the RPN to generate the ROI. The detection head refines the ROI to obtain better detection results.