# Vulnerable Road User Detection for Roadside-Assisted Safety Protection: A Comprehensive Survey
## 3 VRU Detection Models
### Multi-Sensor
#### 3.2.2 Classification by Fusion Scheme  
Regarding the phase of sensor fusion [130], as depicted in Figure 4, a multi-sensor perception system can be categorized into three main classes: (1) Early fusion (EF) involves the fusion of raw data during the pre-processing phase. (2) Intermediate fusion (IF) entailsthe fusion of features during the feature extraction stage. (3) Late fusion (LF) involves the fusion of perception outcomes during the post-processing phase. Each fusion scheme has its own set of advantages and drawbacks. For instance, EF and IF typically require more computational resources and more complex model structures to efficiently extract features and achieve higher accuracy. In contrast, LF generally employs conventional rule-based methods to improve final performance with greater efficiency, making it suitable for real-time applications. Therefore, different scenarios may require different fusion schemes to meet specific objectives. Moreover, various fusion schemes may be applied simultaneously within a detection model to leverage the strengths of each approach. The following section highlights some major studies classified by fusion scheme:  
* Early Fusion Sharing the raw sensor data with other sensors to extend the perceptual range or point cloud density (multiple LiDARs) and enhance detection accuracy is a logical approach. In line with this strategy, the raw data collected from various sensors is transformed into a unified coordinate system to facilitate subsequent processing [131]. However, this kind of strategy is inevitably sensitive to some issues such as sensor calibration, data synchronization, and communication capacity, which are critical considerations in real-time implementations [132]. For example, with a limited communication bandwidth, it may be viable to transmit limited-resolution image data, but it might not be practical to transmit LiDAR point cloud data in real-time. For instance, a 64-beam Velodyne LiDAR operating at 10 Hz can produce approximately 20 MB of data per second [9]. * Intermediate Fusion The fundamental principle behind IF can be succinctly described as employing deeply extracted features for fusion at intermediate stages within the perception pipeline. IF relies on hidden features primarily derived from deep neural networks, which exhibit greater robustness compared to the raw sensor data utilized in early fusion. Moreover, feature-based fusion approaches typically employ a single detector to produce object perception outcomes, thereby eliminating the necessity of consolidating multiple proposals as required in LF. For example, MVX-Net utilizes the IF strategy to fuse features from 2D images and 3D points or voxels, and it is processed by VoxelNet [8]. [PERSON] et al. also proposed the PillarGrid to fuse the extracted features of two LiDARs to improve the detection accuracy [117]. * Late Fusion Late fusion also adopts a natural cooperative paradigm for perception, wherein perception outcomes are independently generated and subsequently fused. Unlike EF, although LF still requires a relative positioning for merging perception results, its  
Figure 4: The fusion scheme. robustness to calibration errors and synchronization issues is significantly improved. This is primarily because object-level fusion can be established based on spatial and temporal constraints. [PERSON] et al. [133] employed the Extended Kalman Filter (EKF) to coherently align the shared bounding box proposals, leveraging spatiotemporal constraints. Moreover, techniques such as Non-Maximum Suppression (NMS) [134] and other machine learning-driven proposal refining methods find extensive applications in LF methodologies for object perception [135]. Thus, different fusion schemes have their advantages and disadvantages. Early fusion and late fusion are both straightforward, but early fusion requires accurate calibration and sufficient communication bandwidth, and late fusion may result in limited detection accuracy.