## 4 Experiments and Results
### Evaluation Metrics  
This paper establishes two main criteria for an object detector to be considered successful in identifying target objects using the proposed multi-modal detection approach. Notably, a GIoU threshold of 0.5 is consistently applied across all models and datasets examined, employing a grid-based search approach to ensure consistency. This metric extends IoU by considering the minimum bounding box that can enclose both the predicted and ground-truth boxes, as explained in Section 3.4 within this paper. This technique penalizes predictions that are far away from the ground truth, even if they have some overlap. Evaluating the performance of object detectors requires a variety of metrics [38]. A popular choice for both assessment and comparison is the mean average precision (mAP). This metric represents the average performance across all object categories, calculated by taking the mean of the average precision (AP) for each class. AP itself is derived from the area under the precision-recall (PR) curve. Within this curve, precision (P) reflects a model's ability to correctly identify true objects, indicating the percentage of predictions that are positive. Conversely, recall (R) assesses a model's ability to find all actual positive instances present in the ground-truth data. These metrics can be expressed mathematically as follows:  
\[P =\frac{TP}{TP+FP} \tag{6}\] \[R =\frac{TP}{TP+FN}\] (7) \[AP =\frac{\Sigma P}{number\ of\ objects} \tag{8}\]  
Figure 10: An example of an occluded image obtained from the nuScenes dataset, captured by a front camera positioned at the vehicle’s top-front location. \[mAP=\frac{\Sigma AP}{number\ of\ classes} \tag{9}\]  
Here, the variables represent the following:  
* True positive (TP) represents correctly identified objects where the predicted bounding box overlaps with the ground-truth box. * False positive (FP) denotes erroneous detections where the predicted box has some overlap with the ground truth but falls, indicating an incorrect identification. * False negative (FN) occurs when the model fails to detect a valid object present in the scene, resulting in a missed target. Additionally, training parameters for experimental analyses are outline in Table 1 and average errors in different aspects of 3D object detection are evaluated on the nuScenes dataset in Table 2. The benchmarking metrics utilized for these analyses are as follows:  
* Mean average translation error (mATE): This metric represents the average Euclidean distance (in meters) between the predicted and ground-truth 3D translation (\(x,y,z\)) of an object across all objects and scenes in the dataset. * Mean average scale error (mASE): This metric represents the average absolute difference between the predicted and ground-truth scale (measured as the product of length, width, and height) of an object across all objects and scenes. * Mean average orientation error (mAOE): This metric represents the average minimum yaw angle difference (in radians) between the predicted and ground-truth orientation of an object across all objects and scenes. It is typically calculated on a per-class basis due to variations in object types. * Mean average velocity error (mAVE): This metric represents the average absolute difference between the predicted and ground-truth velocity (magnitude in m/s) of an object across all objects and scenes. It is usually calculated on a per-class basis. * Mean average attribute error (mAAE): This metric represents the average absolute difference between the predicted and ground-truth attributes (depending on the specific attribute considered) of an object across all objects and scenes. It is typically calculated on a per-class basis and depends on the specific set of attributes considered for the object category. These metrics are commonly used in the nuScenes dataset to evaluate the performance of object detection and tracking algorithms, particularly when dealing with 3D object detection and perception tasks.