# BEVCorner: Enhancing Bird's-Eye View Object Detection with Monocular Features via Depth Fusion
## 3 BEVCorner
### Overall Architecture  
The overall architecture is shown in Figure 1. We have the two image backbones, which extract 2D features, \(F_{mono\_i}^{2d}\) and \(F_{BEV\_i}^{2d}\), for each monocular and multi-view BEV pipeline (see Equation (1)) from N view input images \(I=\{I_{i},\,i=1,2,\ldots,N\}\), where \(H,W\), and \(C_{F}\) stand for the feature's height, width, and channel number.  
\[F_{i}^{2d}=\{F_{i}^{2d}\in\mathbb{R}^{C_{F}\times H\times W},\,i=1,2,\ldots,N\} \tag{1}\]  
Each image feature \(F_{i}^{2d}\) goes through the different pipelines. \(F_{mono\_i}^{2d}\) is processed through a monocular pipeline that is extended from CenterNet [12]. Prediction heads  
Figure 1: Framework of BEVCorner. The framework comprises dual pipelines with separate backbones for monocular and multi-view feature extraction, ensuring optimal adaptation to the unique demands of each modality. The extracted features are fused through a Depth Fusion module, and Voxel Pooling is employed to unify all point features into a single coordinate system, projecting them onto the BEV feature map.  
process \(F_{mono\_i}^{2d}\) by regressing object properties, including 2D bounding box, 3D dimension, geometrical-based 3D bounding box, orientation, keypoints, object depth, and keypoint depths. Final depth estimation is an uncertainty guided combination of the regressed depth and the computed depths from estimated keypoints and dimensions, following the MonoFlex [1] framework. The point of the monocular pipeline is to obtain individual objects' depth distribution, \(D_{mono}(k)=\{D_{mono}(k)\in\mathbb{R}^{C_{D}\times H\times W}\), \(k=1,2,\ldots,K\}\), where \(D\) is the number of depth channels set in the BEV pipeline, and \(K\) is the number of detected objects from that \(i\)-th frame.  
\(F_{BEV\_j}^{2d}\) is processed through the BEV pipeline. Combined with the camera parameters, \(F_{BEV\_j}^{2d}\) is input into the depth network to generate depth-aware context features, \(C_{BEV}(x,y)\in\mathbb{R}^{C_{C}\times H\times W}\), and estimation of depth distribution, \(D_{BEV}(x,y)\in\mathbb{R}^{C_{D}\times H\times W}\). \(x,y\) refers to the image pixel at the \(i\)-th frame, \(C_{D}\) is the number of context channels, and \(C_{D}\) stands for the number of depth bins. Depth estimation, \(D_{BEV}(x,y)\), is explicitly supervised via LiDAR point clouds.  
A depth fusion module merges both depths \(D_{mono}(k)\) and \(D_{BEV}(x,y)\) at the \(i\)-th frame. This is broken down further in Section 3.3. The depth distribution, \(Dpred\), is gained from the combination of BEV's depth map and monocular's object-specific depth.  
\[D^{pred}=D_{fused}=D_{BEV}^{pred}\oplus D_{mono}^{pred} \tag{2}\]  
A view transformer projects \(F_{i}^{2d}\) into 3D representations \(F_{i}^{3d}\) using Equation (3), and then pools them into an union BEV representation, \(F^{bev}\). A 3D detection head predicts the class, 3D box offset, and other attributes from \(F^{bev}\).  
\[F_{i}^{3d}=F_{i}^{2d}\otimes D_{i}^{pred},\quad F_{i}^{3d}\in\mathbb{R}^{C_{F }\times C_{D}\times H\times W} \tag{3}\]