## 3 VRDeepSafety Simulation Environment
### Mathematical Model of VRDeepSafety
#### 3.5.4 V2X Sensor Data Integration  
Combining data from multiple sensors is critical for AVs to achieve a comprehensive understanding of their environment. The state estimation process within VRDeepSafety combines measurements from various sensors--including LiDAR, cameras, and radar--using techniques such as the extended Kalman filter (EKF) and other advanced non-linear filters. The process starts with defining a measurement vector, \(z(t)\), that combines the outputs from each of the individual sensors. This measurement vector is given by the following:  
\[(\mathbf{t})= \begin{bmatrix}\mathbf{z}_{\text{LiDAR}}(t)\\ \mathbf{z}_{\text{Cameras}}(t)\\ \mathbf{z}_{\text{Radar}}(t)\end{bmatrix}  \tag{8}\]  
where \(\mathbf{z}_{\text{LiDAR}}(t)\), \(\mathbf{z}_{\text{Cameras}}(t)\), and \(\mathbf{z}_{\text{Radar}}(t)\) are the measurements from the LiDAR, camera, and radar sensors, respectively, at time \(t\). This composite vector provides a unified view of the environment. The Kalman filtering process minimizes the error between the true state \(\hat{\mathbf{x}}(\mathbf{t})\) and the estimated state \(\mathbf{x}(t)\). The EKF is applied using a prediction and an update step:  
The prediction step estimates the next state based on the current state and control inputs, accounting for uncertainty:  
\[\begin{split}\hat{\mathbf{x}}(t\mid t-1)&=f\big{(}\hat{ \mathbf{x}}(t-1),\mathbf{u}(t-1)\big{)}+\mathbf{w}(t)\\ \mathbf{\mathrm{P}}(t\mid t-1)&=\mathbf{\mathrm{F}}(\mathbf{t}) \mathbf{\mathrm{P}}(t-1)\mathbf{\mathrm{F}}\big{(}t\big{)}^{T}+  
The update step refines the predicted state using new measurements from sensors:  
\[ \begin{gathered}\mathbf{K}(t)=\mathbf{P}(t\mid t-1)\mathbf{H}\left(t \right)^{\mathbf{T}}\left(\mathbf{H}(t)\mathbf{P}(t\mid t-1)\mathbf{H}\left(t \right)^{\mathbf{T}}+\mathbf{R}(t)\right)^{-1}\\ \hat{\mathbf{x}}(t)=\hat{\mathbf{x}}(t\mid t-1)+\mathbf{K}(t)\left( z(t)-h\left(\hat{\mathbf{x}}(t\mid t-1)\right)\right)\\ \mathbf{P}(t)=(I-\mathbf{K}(t)\mathbf{H}(t))\mathbf{P}(t\mid t-1) \end{gathered}  \tag{10}\]  
where \(\mathbf{H}(t)\) is the Kalman gain, which determines the weight given to the sensor measurements, \(\mathbf{R}(t)\) is the measurement noise covariance matrix, \(\mathbf{K}(t)\) is the Kalman gain, and \(h(\hat{\mathbf{x}}(t\mid t-1))\) denotes the non-linear measurement model. The EKF's recursive structure enables real-time state estimation, suitable for both linear and non-linear sensor models, providing a coherent and accurate representation of the vehicle's environment, thus enhancing perception, localization, and decision-making. LiDAR Sensor Simulation (LiDAR(\(t\)))  
To simulate sensor measurements in VRDeepSafety within Unity, we carefully replicate each sensor type's behavior. Unity's physics and rendering engines simulate real-world sensor behaviors by mimicking the physical principles of LiDAR. In Unity, LiDAR measurements are simulated using ray casting, which sends out laser beams to measure distances to objects. We use Physics.Raycast functions to cast rays in a 360-degree pattern or within a specified field of view. The number of rays corresponds to the LiDAR sensor's resolution. \[\mathbf{z_{LiDAR}}(t)=\left\{d_{i}(t)\mid i=1,\dots,N\right\} \tag{11}\]  
where \(d_{i}(t)\) is the distance measurement for the \(i\)-th ray at time \(t\), and \(N\) is the total number of beams. To enhance realism, Gaussian noise is added to the ray-casting results. The magnitude of this noise depends on the environmental conditions. This models the real-world sensor limitations where the signal quality degrades when visibility is reduced. This noisy LiDAR measurement is defined as follows:  
\[\mathbf{z_{LiDAR}}(t)=d_{i}(t)+\boldsymbol{\epsilon_{LiDAR}}(t,e(t)) \tag{12}\]  
where \(\mathbf{z_{LiDAR}}\left(t\right)\) is the noisy LiDAR measurement at time \(t\), \(d_{i}(t)\) is the ideal, noise-free distance measured by the \(i\)-th LiDAR ray at time \(t\), and \(\boldsymbol{\epsilon_{LiDAR}}(t,e(t))\) is the Gaussian noise added to the measurement.