# Survey of Deep Learning-Based Methods for FMCW Radar Odometry and Ego-Localization
## 5 Deep Learning Approaches  
Within this section, we discuss and compare various deep learning architectures that have been applied to radar-based odometry and localization. Subsequently, we delve into the manner in which radar data are seamlessly integrated with other sensor inputs within the context of deep learning approaches. The investigation of various approaches involved a thorough search on Google Scholar and Web of Science. Initially, methods related to deep learning for radar-based odometry estimation or place recognition were identified on Google Scholar using keyword searches. Additional relevant methods were then found through a backwards search of the initially identified references. To complement the Google Scholar search, a similar search was conducted on Web of Science, using the same queries as those used in the Google Scholar search. The search utilized different permutations and choices of keywordsand phrases such as \"Deep learning\", \"Learning-based\", \"(Un)-supervised\", \"Radar\", \"Ego-localization\", \"Localization\", \"Scan matching\", \"Scene flow\", \"Odometry\", \"Place recognition\", \"Point-based\", \"Sensor Fusion\", and \"SLAM\" which were connected with the conjunctions \"and\" and \"for\". Given the relatively recent introduction of deep learning in radar-based SLAM, the search specifically considered methods published from 2016 onwards. This cutoff criterion was motivated by the famous paper by [PERSON] et al. [83] using deep learning for visual odometry published in 2017. Figure 4 shows the number of papers investigated for odometry estimation and localization using deep learning and radar sensors. Here, the methods are also differentiated by radar setup, and if other modalities are also used by the method. As radars are very price-effective, they can be integrated in various systems as perception sensors. These systems can vary greatly in terms of available computational hardware and use-case scenarios. For example, autonomous cars have more space to carry a large Graphics Processing Unit (GPU) for processing, whilst a cleaning robot has very limited memory. On the other hand, the car operates in a more challenging environment, operating at potentially much larger scales and ego-velocities than the cleaning robot causing greater motion distortion. The methods introduced are thus not all comparable to one another and need to be interpreted for their respected datasets and use-cases. An overview of the investigated methods, as well as the radar representation, datasets, its task, and the usage of deep learning are shown in Table 3. \begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Method** & **Data Representation** & **Datasets** & **Task** & **Model Architecture and Usage** \\ \hline
[84] & SR cartesian scan & Oxford & Odometry & CNN for masking irrelevant detections \\ \hline
[85] & SR cartesian scan & Oxford & Odometry & CNN for masking irrelevant detections \\ \hline Fast-MbyM [86] & SR cartesian scan & Oxford & Odometry & CNN for masking irrelevant detections \\ \hline UnderTheRadar [87] & SR cartesian scan & Oxford & Odometry and localization & \begin{tabular}{c} Extraction of keypoint location, weight, and \\ descriptor using a U-Net with \\ multiple decoders. \\ \end{tabular} \\ \hline HERO [88] & SR cartesian scan & Oxford, Boreas & Odometry & \begin{tabular}{c} Extraction of keypoint location and weights \\ using a U-Net with multiple decoders. \\ \end{tabular} \\ \hline RaFlow [89] & AR 4D point cloud & In-house & \
\begin{tabular}{c} Odometry, scene flow \\ estimation \\ \end{tabular} &
\begin{tabular}{c} Multiple modules including multi-scale \\ encoder, cost-volume layer, and flow decoder \\ to predict scene flow. \\ \end{tabular} \\ \hline \hline \end{tabular}
\end{table} 
Table 3: List of deep learning methods applied for radar-based odometry and localization. SR stands for a spinning radar and AR for an automotive radar setup.