## 4 VRFormer
### Evaluating Metrics for VRFormer Efficacy
#### 4.8.2 Motion Prediction and Object Detection Metrics  
The quality of motion prediction within the VRDeepSafety platform is evaluated using mIoU as well as FPQ. The mIoU quantifies VRFormer's ability to predict trajectories by comparing the overlap between predicted and actual paths over time. The FPQ benchmarks spatial prediction accuracy at future time steps, assessing precision at a fixed time horizon. Object detection is measured using mAP across distance thresholds, enabling an objective assessment of VRFormer's ability to identify, classify, as well as track objects in 3D space. Together, these metrics provide a comprehensive evaluation of VRFormer's capabilities.  
* Mean Intersection over Union (mIoU): We used mIoU metric to evaluate the accuracy of VRFormer's motion predictions by measuring the overlap between predicted and actual object trajectories over time. It is calculated as follows: \[\text{mIoU}=\frac{1}{N}\sum_{i=1}^{N}\frac{\left|\text{P}_{\text{pred}}\ \right.\left.\cap\text{P}_{\text{actual}}\right|}{\left|\text{P}_{\text{pred}} \ \right.\left.\sqcup\text{P}_{\text{actual}}\right|}\] (33) where mIoU represents the mIoU, \(N\) represents the total number of objects assessed, \(\text{P}_{\text{pred}}\) represents the predicted position of an object, \(\sqcap\) denotes the intersection of predicted and actual sets, and \(\sqcup\) denotes the union of predicted and actual sets.  
This metric provides a quantitative measure of how closely the model's predicted trajectories conform to reality, serving as a direct indication of its motion-tracking performance.  
* Final Prediction Quality (FPQ): To evaluate VRFormer's proficiency in predicting future object movements, we used the FPQ metric. This metric measures the accuracy of spatial predictions at future time-steps, evaluating the degree to which predicted paths align with the actual paths of objects over a defined time horizon. It is expressed as follows: \[\text{FPQ}=\frac{1}{M}\sum_{i=1}^{M}\text{IoU}(\text{P}_{\text{Pred}}\ (T),\text{P}_{\text{actual}}\ (T))\] (34) where FPQ represents the final prediction quality score, \(M\) is the total number of objects evaluated, \(T\) is the time horizon at which predictions are made, \(\text{P}_{\text{pred}}\ (T)\) is the predicted path of an object at time \(T\), \(\text{P}_{\text{acturl}}\ (T)\) is the actual path of the object at time \(T\), and \(\text{IoU}(\text{P}_{\text{Pred}}\ (T),\text{P}_{\text{actual}}\ (T))\) is the intersection over union (IoU) between the predicted and actual paths
* Mean Average Precision (mAP): The object detection capabilities of VRFormer are assessed using mAP. This metric quantifies VRFormer's proficiency in identifying, classifying, and tracking objects in the 3D environment. The mAP is calculated over various distance thresholds, thereby evaluating the model's ability to accurately detect and track objects regardless of their distance from the ego vehicle. The formulation for mAP is as follows: \[\text{mAP}=\frac{1}{N}\sum_{i=1}^{N}\frac{\text{TP}}{\text{TP}+\text{FP}}\] (35) where mAP represents the mAP score, \(N\) is the total number of distance thresholds evaluated, TP refers to the number of true positive detections, and FP represents the number of false positive detections.  
By using these evaluation approaches, we acquire a complete view of VRFormer's performance across the different tasks it performs in the VRDeepSafety virtual environment.  
The combination of these metrics provides a thorough and multi-faceted analysis of performance in motion prediction, accident forecasting, and object detection, which are critical for evaluating the model's overall performance within the VRDeepSafety framework.