# Multi-View Joint Learning and BEV Feature-Fusion Network for 3D Object Detection
## 3 Method
### Voxel-Based Feature Extraction Branch  
This branch discretizes the point cloud into voxels, extracts spatial features at the voxel level, and then transforms the features into the BEV to complete the task of the feature-extraction  
Figure 3: 2D feature-extraction backbone. The convolution and maximum pooling downsampling part (dotted line box) is only used in the pseudo-image-based feature-extraction branch. phase. Figure 4 shows the overall structure of the voxel-based feature-extraction branch, which is divided into two parts. The first part (red part) projects voxel features in the Z direction (BEV), and the second part (blue part) projects voxel features in the X and Y directions. The multi-view feature-fusion module fuses the feature maps from three views as output. Specifically, the point cloud is discretized into voxels according to a fixed size, with K points in each voxel. For the convenience of uniformly coding point features into voxel features, we suppose that there are N points in each voxel. If \(K<N\), we fill the vacancy with 0; if \(K>N\), we randomly sample N points from K points [11]. Then, the original features \((x_{i},\,y_{i},\,z_{i},\,c_{raw})\) of all points in the nonempty voxel are encoded as voxel features \((f_{1},\,f_{2},\,\cdots,\,f_{N})\) by using the averaging operation. Here, \(c_{raw}\) represents other attributes of the point cloud, such as reflectivity, color, and so on, in addition to the three-dimensional coordinates of the point. To accelerate the calculation process of 3D convolution, we choose sparse convolution [12] as the main operation of feature extraction. For a feature map with input dimension \(C_{in}\times D_{in}\times H_{in}\times W_{in}\), where the \(D_{in}\) represents the size of the input feature map in the Z direction, we conducted four downsamplings with \(1\times,2\times,4\times,8\times\), and each downsampling uses 3D ResNet structure [24]. In the process of downsampling, we also introduce the residual structure and obtain a 3D feature map with shape \(C_{out}\times\frac{D_{in}}{8}\times\frac{H_{in}}{8}\times\frac{W_{in}}{8}\). The 3D feature map will be projected onto the Z axis. The final output shape of the BEV feature map is \(C_{lev}\times\frac{H_{in}}{8}\times\frac{W_{in}}{8}\), where \(C_{lev}=C_{out}\times\frac{D_{in}}{8}\). The above feature-extraction operation has been downsampled 4 times. With the exception of the first time, each downsampling will reduce the resolution of the feature map by half. The resolution on the Z-axis will gradually decrease; thus, some of the geometric information in this direction will be lost. We believe that the high resolution on the Z-axis can provide more geometric information in the Z-axis direction. Adding this information to the BEV feature map will help predict better 3D object proposals. Therefore, as shown in the blue part of Figure 4, we introduced another feature extraction module to enhance the Z-axis information in the BEV feature map. This part also uses sparse convolution to extract features of voxels. However, differently from the other part, the resolution is compressed only in the X and Y directions during the downsampling process, while the resolution is always fixed in the Z direction, that is, we enter a feature map with the shape of \(C_{in}\times D_{in}\times H_{in}\times W_{in}\) and obtain a feature map with the shape of \(C_{out}\times D_{in}\times\frac{H_{in}}{8}\times\frac{W_{in}}{8}\) after sparse convolution downsampling. Sparse convolution results are projected onto the X-axis and Y-axis to obtain the projection features in the RV and SV. Figure 4: 3D feature-extraction backbone. The red part uses the traditional sparse convolution layer for downsampling and feature extraction, and 3D features are projected in the Z direction. In the process of sparse convolution, the resolution of the blue part in the Z direction remains unchanged, and the 3D features are projected in the X and Y directions.