## 4 VRFormer
### Accident Prediction with VRFormer
#### 4.9.2 Prediction Process  
VRFormer, operating within the VRDeepSafety framework, utilizes a carefully designed multi-step process to achieve accurate and robust accident prediction. This process methodically analyzes the simulated driving environment and identifies potential hazards, integrating both VR and V2X data to ensure a comprehensive approach to safety assessment. The initial, and vital, BEV event segmentation. This is where the system aggregates motion prediction results to generate detailed outlines of object trajectories in both present and anticipated future frames. At this stage, a strong and clear understanding of the spatial environment is generated, providing a view of the locations of all objects, both current and predicted. This detailed spatial information enables the system to track and predict how objects might interact, thereby forming the basis for subsequent analyses of potential risks. Using BEV ensures that this process creates a robust and clear scene representation by projecting 3D objects into a 2D perspective. Figure 8 demonstrates VRFormer's multi-source prediction system within VRDeepSafety, showcasing how V2X communication enhances accident prediction. This Unity-based implementation presents multiple synchronized views: (Top-left) Vehicle #1s limited sensor coverage without V2X, illustrating 'invisible' zones across various camera angles; (Bottom-left) additional environmental context provided by infrastructure camera views; (Top-right) a single-vehicle system failing to predict Vehicle #3s motion compared to enhanced V2X-5 agent prediction; (Bottom-right) Vehicle #3s perspective, showing how visibility changes with different viewpoints. These examples highlight VRFormer's ability to segment trajectories accurately, identify potential collision risks, and overcome single-vehicle limitations, leading to better accident prediction. Dashed lines indicate trajectories undetected without V2X. Vehicle positions are labeled, with all scenes derived directly from the Unity simulation. Vehicle colors in the V2X prediction match GT views: Vehicle 1 (blue), Vehicle 2 (yellow), and Vehicle 3 (purple). This Unity implementation enables real-time simulation and validation of our multi-agent prediction system. Following the segmentation, VRFormer carefully proceeds to accident candidate identification. The object segmentation data are approximated using polygonal representations at each timestamp. This approximation is vital for subsequent analysis. An algorithm is then used to examine spatial coincidences and identify where polygons overlap or are in proximity. It is these spatial proximities, where objects share the same space, that are flagged as potential accident candidates. The model identifies high-risk scenarios, such as possible collisions, or highlights data discrepancies requiring further analysis. These spatial coincidences provide important information about the subsequent stages of analysis. In the final stage, accident occurrence determination, the VRFormer system analyzes the object paths over time, assessing the likelihood of an impending collision; the system employs a threshold-based approach: if the distance between any two objects falls below a pre-determined value, typically 1.0 m, an alert is generated. VRFormer also meticulously records specific details of all flagged events, including comprehensive object documentation, their positions, and the precise timestamps at which the potential accidents occur; this detailed data capture enables rigorous analysis of the model's performance. Figure 9 shows the workflow for predicting movement and accidents in VRDeepSafety. VR observations and V2X communications are input, processed by the BEV extraction and V2X fusion blocks, and fed to the movement prediction module (event segmentation, accident candidate identification, accident occurrence determination). Performance is evaluated using APA, mIoU, and mAP. The output is predicted movement and accident reports.