## 4 VRFormer
### Data Fusion Techniques: Integrating VR and V2X Inputs
#### 4.3.1 V2X Communication Module  
To maintain precision and stability, the VRFormer uses Kalman filtering on the V2X data inputs to reduce noise and generate smooth estimates of positional and velocity data, which is especially valuable when there is variability or latency in V2X transmissions. By applying Kalman filters, the VRFormer maintains a stable and accurate representation of the surrounding environment, helping reliable real-time decision-making. The Kalman filter works by predicting the current state and updating the state estimate when new measurements become available. The state vector \(x_{t}\), which represents vehicle parameters such as position and velocity, is updated using the following prediction equations:  
\[\begin{split}\mathbf{x}_{t+1}&=\mathbf{A}\mathbf{x }_{t}+\mathbf{B}u_{t}+\mathbf{w}\\ \mathbf{P}_{t+1}&=\mathbf{A}\mathbf{P}_{t}\mathbf{A }^{\mathbf{T}}+\mathbf{Q}\end{split} \tag{22}\]  
where \(\mathbf{x}_{t+1}\) represents the predicted state at time \(t+1\), \(\mathbf{A}\) is the state transition matrix, \(u_{t}\) denotes the control input, \(\mathbf{w}\) accounts for process noise, \(\mathbf{P}_{t}\) is the uncertainty covariance matrix, and \(\mathbf{Q}\) represents the process noise covariance matrix.  
When receiving new sensor measurements, the Kalman filter updates the vehicle's state estimate using the following equations:  
\[\begin{split}\mathbf{K}_{t}&=\mathbf{P}_{t}\mathbf{ H}^{\mathbf{T}}\big{(}\mathbf{H}\mathbf{P}_{t}\mathbf{H}^{\mathbf{T}}+ \mathbf{R}\big{)}^{-1}\\ \mathbf{x}_{t+1}&=\mathbf{x}_{t+1}+\mathbf{K}_{t}( \mathbf{z}_{t}-\mathbf{H}\mathbf{x}_{t+1})\end{split} \tag{23}\]  
where \(\mathbf{K}_{t}\) is the Kalman gain, \(\mathbf{H}\) denotes the observation matrix, \(\mathbf{z}_{t}\) is the measurement vector, and \(\mathbf{R}\) represents the measurement noise covariance matrix.  
The Fusion Module is a crucial component that integrates the data streams from the VR detection module and the V2X communication module. Specifically, this module makes use of a combination of temporal synchronization, using timestamps, geometric transformations such as translations, scaling, and rotations to correctly align the data from different sources, and Kalman filtering, which is applied to the V2X data to smooth noisy measurements and give better estimates of velocity and location. In addition, the module uses a dynamic weighting mechanism that prioritizes sensor data depending on reliability or weather conditions. The final output of the Fusion Module is a single, unified representation that forms a basis for the prediction module.  
The accident prediction module employs a sophisticated risk-scoring system that assesses each potential collision candidate based on different criteria. The system computes a probability score from 0 to 1 for collision, using a sigmoid activation function with input features like: the predicted distance between two or more objects in BEV, the relative velocity between these objects, whether there are other objects nearby, and the potential for a lane change. The collision risk threshold, set initially at 0.7, can be dynamically adjusted based on the prevailing traffic conditions and the desired sensitivity of the safety mechanisms. When the risk score goes above this threshold, the system initiates safety protocols. These can vary based on the predicted action of the vehicle, from generating a warning or adjusting the vehicle trajectory by steering or braking to avoid the accident. The choice of safety protocol is determined based on the magnitude and urgency of the risk.  
To ensure the accuracy of data alignment from VR and V2X sources within VReDeepSafety, VRFormer uses spatial-temporal synchronization. This involves aligning time stamps and applying projective transformations to ensure synchronization of V2X data with VR-derived spatial data, resulting in an integrated and real-time view that facilitates the seamless merging of data from different sources.