# Sparsity-Robust Feature Fusion for Vulnerable Road-User Detection with 4D Radar
## 1 Introduction  
Detecting vulnerable road users (VRU), such as pedestrians, cyclists, etc., is a major challenge for autonomous vehicles. Vulnerable road users are defined in the ITS Directive of the Directorate-General for Mobility and Transport of the European Commission [1] as non-motorized road users, such as pedestrians and cyclists, as well as motorcyclists and persons with disabilities or reduced mobility and orientation. These road users are particularly challenging to detect due to their small size, especially when compared to other objects in a scene, frequent occlusion, and high vulnerability. For this reason, a wide variety of methods have been developed to detect these VRUs as robustly as possible, with such detection crucial for systems such as emergency braking assistants. These systems differ in terms of the sensor modality used. In scientific publications, the use of mono or stereo cameras is the most common [2], followed by 3D LiDAR sensors [3]. Both modalities show convincing results on scientific datasets under good weather conditions but are unfortunately limited in practical application. This is due to the strong dependence on environmental conditions [4]--night, rain, fog, etc.--and, in the case of LiDAR, to the substantial hardware costs. While traditional 3D radar sensors have been an integral part of basic driver assistance systems for a long time, the recently available high-resolution 4D (or 3 + 1D) radar systems introduce new possibilities for advanced detection tasks. In 3D radars, the three dimensions are range, azimuth, and Doppler information. Azimuth and range can be interpreted as a 2D point cloud (x- and y-coordinates). Furthermore, the Doppler information can be used tomeasure the speed of the detected object relative to the sensor. This limited spatial resolution is not sufficient to reliably detect VRUs. In certain situations, VRUs may be mistaken for other objects, making them undetectable by the sensor. In contrast, 4D radars provide four dimensions: range, azimuth, elevation, and Doppler information. The additional dimension compared to 3D radars is the elevation, which provides the ability to compute a 3D point cloud (x-, y-, and z-coordinates) that includes velocity, as shown in Figure 1. This new category of radar sensors allows 3D detection algorithms to be applied to these sensors. Compared to 3D radar sensors, 4D radars are much denser while maintaining robustness in challenging weather conditions. Based on their characteristics, these 4D radar sensors are well suited for VRU detection but are currently still significantly under-researched compared to competing camera or LiDAR approaches. This is partly because sensors have only recently become widely available and partly because large-scale datasets that include annotations, such as View-of-Delft [5], have only recently become accessible. For this reason, radar sensors are often used in the field, even though the point clouds provided are significantly sparser compared to LiDAR sensors. Radars combine the advantages of relatively low costs with great robustness against weather influences. At the same time, 4D radar sensors also present various challenges. The toughest challenge might be the limited number of 3D measurement points obtained from the sensors. In addition, these measurement points are distributed very irregularly, depending on the scene geometry. VRUs are at a natural disadvantage, as larger objects such as cars or trucks have a higher point density compared to cyclists. In addition, there are also disturbing artifacts caused by reflections and the multipath effect [6]. While classical rule-based approaches have proven to be effective at extracting geometric features such as lines [7; 8], the detection of entire objects remains challenging. Consequently, classical rule-based approaches are not suitable for detection tasks. Learning-based approaches, in particular, show their full potential in this regard.