# BEVCorner: Enhancing Bird's-Eye View Object Detection with Monocular Features via Depth Fusion
## 3 BEVCorner
### Weighted Fusion  
To avoid abrupt transitions between object-level depth and depth maps, we apply a confidence map, \(C(x,y,z)\), that weighs the depth estimates based on the reliability of each source (monocular vs. BEV). The confidence map, \(C(x,y,z)\), indicates the trustworthiness of the estimated monocular depth at pixel \((x,y,z)\), taking into account factors such as object detection certainty and distance to the object. It is defined as:  
\[C(x,y,z)=\alpha\in\mathbb{R}^{C_{0}\times H\times W} \tag{5}\]  
where \((x,y)\in\text{object }i\) and \(z\in D_{\text{mono}}(i)\). The parameter \(\alpha\) is a fixed or learned weight that reflects the confidence in the object-specific depth from the monocular pipeline. Figure 2: Different depth fusion strategies. The top-left image displays the object across various depth bins. In the remaining images, different colors represent distinct depth estimation sources: blue indicates regions derived from the BEV depth map, red denotes regions from the monocular depth estimation of the object, and purple highlights regions where both depth estimates are fused. Additionally, in the visualization of the weighted fusion method, light blue signifies regions where the BEV depth map contributes less. A simple approach is to set \(\alpha\) as a fixed constant for all pixels inside the object region, typically close to 1, since monocular depth is generally more reliable in object regions than BEV depth. For example:  
\[\alpha=0.9, \tag{6}\]  
indicating 90% confidence in the monocular depth. Alternatively, \(\alpha\) can be learned using a neural network:  
\[\alpha=f(\mathbb{E}[D_{\text{mono}}(i)]+\mathbb{E}[D_{\text{BEV}}(x,y)]) \tag{7}\]  
where \(f\) is a function implemented by the neural network. \(\alpha\) can also vary based on object size or distance. For smaller or more distant objects, depth estimates are often less reliable, so \(\alpha\) can be decreased accordingly. For instance:  
\[\alpha=\frac{1}{1+\text{distance}(i)} \tag{8}\]  
or based on the area of the bounding box:  
\[\alpha=\frac{\text{area of bounding box}}{\text{max area}} \tag{9}\]  
where max area is the maximum area of any bounding box in the dataset or scene. Finally, the fused depth \(D_{\text{fused}}(x,y,z)\) is computed as a weighted sum:  
\[D_{\text{fused}}(x,y,z) =C(x,y,z)\cdot\mathbb{E}[D_{\text{mono}}(i)]+\] \[(1-C(x,y,z))\cdot\mathbb{E}[D_{\text{BEV}}(x,y)] \tag{10}\]  
This approach allows the object-specific depth to dominate when the object is detected (i.e., higher \(C(x,y,z)\)), while the depth map distribution from BEVDepth remains influential in areas without objects (i.e., lower \(C(x,y,z)\)). **Region-of-Interest Refinement**  
Another way to combine the two depth sources is to treat the monocular depth as a refinement over the BEV distribution in specific Regions Of Interest (ROIs). These ROIs correspond to the detected objects in the scene. This approach combines elements of both the direct replacement and weighted fusion methods. Like direct replacement, the refinement is confined strictly to the object regions, leaving the depth map from BEV untouched outside these regions. Like weighted fusion, the same fusion process takes place, but it occurs only within each enclosed detected object region. The blend of the two depths is achieved using a weighted sum, where BEV depth and monocular depth are mixed based on a confidence map or ROI-specific weighting:  
\[D_{\text{i\_fused}}(x,y,z) =C(x,y,z)\cdot\mathbb{E}[D_{\text{mono}}(i)] \tag{11}\] \[\quad+(1-C(x,y,z))\cdot\mathbb{E}[D_{\text{BEV}}(x,y)],\]  
where \(D_{\text{i\_fused}}(x,y,z)\) is the fused depth applied to the region enclosing object \(i\), and \(C(x,y,z)\) follows Equation (5). For the learned \(\alpha\) approach, the focus shifts to the object itself:  
Figure 2:0  
The final fused depth is computed as:  
\[D_{\text{fused}}(x,y,z)= \begin{cases}D_{\text{i\_fused}}(x,y,z),&\text{if }(x,y,z)\in\text{ROI}_{i},\\ \mathbb{E}[D_{\text{BEV}}(x,y)],&\text{otherwise}\end{cases}