## 4 Results
### Failure Analysis and Failure Detection
#### 4.3.2 Pose Quality Metric  
In failure cases, it may be useful to be able to recognize the fault. We previously defined a pose quality metric (apart from the obvious re-projection error). In order to use this, the object must be also semantically segmented in the image. We evaluate how accurate this metric is and what the ideal classification threshold is. This is accomplished using the following statistical method: let \(\mathbf{Q_{pose}}\) denote the pose quality metric. Higher values indicate poorer alignment. We classify a frame as a failure if its translation absolute pose error (APE) exceeds 1 m:  
\[y=\begin{cases}1,&\text{if APE}_{\text{trans}}>1\,\text{m},\\ 0,&\text{otherwise},\end{cases}\qquad\hat{y}(\tau)=\begin{cases}1,&\text{if }\mathbf{Q}_{\text{pose}}>\tau,\\ 0,&\text{otherwise}.\end{cases} \tag{33}\]  
The overall discrimination performance is summarized by the precision-recall curve (Figure 10a). The average precision (AP), computed as  
\[\text{AP}=\int_{0}^{1}P(R)\,dR=0.78, \tag{34}\]  
Figure 9: This figure depicts the detection process of a pose failure. The top half illustrates our pose quality metric. The reprojected _Semantic Mesh Model_ silhouette (yellow), keypoints (blue), and the detected instance segmentation mask by YOLO [34] (green) and semantic keypoints (red). The bottom half illustrates the associated pose estimates. On the left, the failed pose estimate (red wire-frame) as classified by our pose quality metric, and on the right, the (blue wire-frame), also correctly classified. The red mesh is the output of BAAM [13], which performs better under occlusion in this case.  
indicates good performance. Sorting frames by decreasing \(\mathbf{Q}_{\text{pose}}\) correctly identifies failures nearly 80% of the time, making it an effective ranking metric.  
To select a single practical threshold, we maximized the \(F_{1}\) score over thresholds in the range where \(\mathbf{Q}_{\text{pose}}\) varies significantly (0 to 0.1):  
\[F_{1}(\tau)=\frac{2\,P(\tau)\,R(\tau)}{P(\tau)+R(\tau)}. \tag{35}\]  
Figure 9(b) shows an optimal threshold around  
\[\tau^{\star}\approx 0.05,\quad P(\tau^{\star})=0.64,\quad R(\tau^{\star})=0.23,\quad F_{1}(\tau^{\star})=0.32. \tag{36}\]  
The practical implications of these results are that very low scores (\(\mathbf{Q}_{\text{pose}}<0.01\)) strongly indicate reliability. In a tracking application, \(\tau^{\star}=0.05\) is a reliable acceptance threshold. Overall, the metric provides a reliable high-precision safety filter. Low values strongly confirm accurate pose estimation, while values above the identified threshold reliably indicate the need for caution or a fallback strategy.  
Figure 9 shows our quality metric in action. In rare cases, when the target vehicle is partially occluded, the pose estimation fails even though the keypoint detections are accurate. This, however, is detected by our pose quality metric. Note that it is specifically in these cases that BAAM [13] outperforms our method, which can also be observed in the Figure 10.  
In conclusion of the experimental evaluation, it can be observed that the LiDAR-based pipeline significantly improves reconstruction accuracy and robustness, producing fewer errors and failures. This additionally means that the pose estimations are also more accurate. Our method, therefore, extends the state of the art in image-based vehicle pose estimation. We also show that our pose quality metric is capable of identifying faulty pose estimates, increasing the practical usability of the algorithm.