# Survey of Deep Learning-Based Methods for FMCW Radar Odometry and Ego-Localization
## 5 Deep Learning Approaches
### Automotive Radar
#### 5.4.2 Ego-Localization  
Ego-localization methods necessitate distinct evaluation criteria compared to odometry estimation. While Equation (10) can be used for predicting scan matching in detected loop closures, it is inadequate for assessing the ability of a model to recognize already visited \begin{table}
\begin{tabular}{c c c c} \hline \hline
**Method** & **KITII (\(\omega\)/\(\frac{\text{deg}}{\text{cm}}\))** &
\begin{tabular}{c} **Inference Speed** \\ **(s)** \\ \end{tabular} & **Device** \\ \hline MbyM [85] & 1.16/**3.0** & 0.0747 & Nvidia Titan Xp GPU (Santa Clara, CA, USA) \\ Fast-MbyM [86] & 2.00/6.3 & **0.0063** & Jetson GPU (Nvidia, Santa Clara, CA, USA) \\ RaL+ \(\uparrow\)[102] & **1.09**/37 & 0.243 & Nvidia Titan X GPU (Santa Clara, CA, USA) \\ UnderTheRadar [87] & 2.06/6.7 & 0.034 & \\ HERO [88] & 1.95/6.5 & 0.180 & Nvidia Tesla V100 GPU (Santa Clara, CA, USA) \\ GRAMME **[88]** & 1.98/5.1 & - & NVIDIA GTX 1080 Ti (Santa Clara, CA, USA) \\ CFEAR-3 *** [136] & 1.09/3.6 & 0.167 & i7-6700K CPU (Intel, Santa Clara, CA, USA) \\ \hline \hline \end{tabular}
\end{table} 
Table 4: List of deep learning methods that have been employed for radar-based odometry and tested on the mean Oxford robot test dataset. Performance evaluations are derived from the respective citations. Models marked with (*) indicate training involving additional LiDAR data, while those marked with (**) need additional camera data. The methods marked with (***) represent state-of-the-art classical radar odometry methods. Best performing models are highlighted with **bold**. \begin{table}
\begin{tabular}{c c c} \hline \hline
**Method** & **ATE (m/deg)** & **Inference Speed (s)** \\ \hline RaFlow [89] & 0.07/0.45 & 0.0363 \\ CMFlow * [106] & **0.05**/0.10 & 0.0304 \\
4 DRO-Net [90] & 0.08/**0.07** & **0.0108** \\ NDT ** & 0.56 / 1.52 & 0.0010 \\ \hline \hline \end{tabular}
\end{table} 
Table 5: List of deep learning methods assessed for radar-based odometry on the View-of-Delft (VoD) dataset. The methods are evaluated on VoD Seq. 4, and the data are sourced from [90]. Notably, (*) incorporates Camera, Lidar, and IMU data during training, providing a multimodal learning approach. Additionally, (**) represents a classical method based on the normal distribution transform. The evaluations are conducted on a single NVIDIA 2080 Ti GPU. Best performing models are highlighted with **bold**. locations. Typically, a scene descriptor, akin to the (Net)VLAD descriptors, is generated to encapsulate the place and facilitate place recognition. One evaluation approach involves a nearest-neighborhood (NN) search, determining the closest neighbors of the observed descriptor in a descriptor database using a defined distance metric (e.g., that specified in the NetVLAD loss of Equation (9)). Metrics like Recall@1, F1 score, and mean Average Precision (mAP) are commonly employed for assessing consistent loop closures through precision-recall curves. However, a challenge lies in defining the number of pre-defined keyposes in the dictionary, where a smaller distance between keyposes can lead to a worse metric performance as the dictionary size grows, without necessarily indicating an overall inferior model. Similar to radar odometry, place recognition models are predominantly evaluated using spinning radar data. However, unlike odometry methods, GPU requirements are less critical for these approaches, as they do not need to run continuously. Once the location on a map has been determined, an odometry network can be utilized for pose refinement. Additionally, if ego-localization is performed on a prior map, having both on the same device is computationally efficient, eliminating the need for data transfer. The choice between performing localization on an external cluster or using on-board hardware presents distinct challenges, each with its own set of considerations. In either case, the model should exhibit sufficient speed to enable re-localization in the event of global drift or when a recognized scene is detected.