# Sparsity-Robust Feature Fusion for Vulnerable Road-User Detection with 4D Radar
## 4 Evaluation
### Results  
The evaluation employed the Mean Average Precision (mAP) with the three-dimensional Intersection over Union (IoU), which was calculated analogously to the evaluation of the KITTI [37] dataset. IoU and mAP are two metrics commonly used in object detection tasks. IoU quantifies the overlap between the predicted and ground-truth bounding boxes to determine whether a predicted bounding box correctly localizes an object. mAP considers both the precision and recall of a model across multiple classes and aggregates IoU scores to provide an overall assessment of detection performance. The threshold for correct predictions was defined as an overlap of the predicted bounding box with the ground truth of 50% for _cars_ and an overlap of 25% for _pedestrians_ and _cyclists_. Two areas in the View-of-Delft evaluation split were evaluated: the entire annotated area and the driving corridor, defined as a rectangle at ground level in front of a driving train. Thus, only objectsthat were at most four meters to the left or right and at most 25 m forward from the origin of the camera were analyzed. The results of our evaluation study are depicted in Table 1. We studied the impact of our SRFF neck, comparing its performance to the baseline. At the time of writing, the PointPillars version by [PERSON] et al. [5] is the only comparable method following the original method that uses the accumulated five scans for training and evaluation. Additionally, we studied a _combined_ version, where we fused the feature maps at three different resolutions (\(40\times 40,80\times 80\), and \(160\times 160\)), followed by three separate detection heads. Most notably, our results show that the resolution of the neck output had a large impact. For most comparisons, larger neck feature maps significantly improved not only the detection of VRUs but also the overall mAP. The largest increase was observed when using the CSAM, with the \(160\times 160\) version improving the mAP by 6 over the \(80\times 80\) variant. Larger neck features proved beneficial overall for the pillar network. Additionally, the choice of lightweight attention fusion had a large impact on the overall performance of smaller network neck resolutions. The CBAM outperformed the CSAM in most \(40\times 40\) and \(80\times 80\) variants. A probable explanation for this is that the initial channel reweighting in the CBAM already benefits the following spatial attention mechanism, which is not possible in the CSAM due to the combined attention calculation. Additionally, the 3D convolution kernel has a much smaller receptive field than the kernels in the CBAM, especially when reweighting the channel features. The _combined_ variant of our network performed better than the small neck output feature, but worse than the \(160\times 160\) variant. This indicates that the addition of small-resolution neck features can actually be detrimental to overall network performance. When observing VRUs, multi-scale feature fusion improved detection over the baseline. Attention fusion with the CBAM performed better than the CSAM. In the \(160\times 160\) variant, _pedestrian_ detection improved by 1 mAP over the baseline, with only an additional 0.2 M parameters (ca. 1.1% of the total 18 M), whereas _cyclist_ detection was the same. When considering only the driving corridor, _pedestrian_ detection improved by 1.7 mAP. The results indicate that multi-resolution feature fusion is beneficial in detecting small objects. The qualitative results in Figures 1 and 4 show that even with very few points or partial occlusion, our network reliably captured VRUs. The results were consistent in terms of visual quality for both the driving corridor and the entire scene. Notably, the height of the pedestrians was accurately determined, which can largely be attributed to the third dimension in 4D radar data. However, the images also reveal some incorrect predictions. Most notably, in the top-right image, a pedestrian was incorrectly predicted, where some points of the first car were misclassified.