# Survey of Deep Learning-Based Methods for FMCW Radar Odometry and Ego-Localization
## 5 Deep Learning Approaches
### Odometry Estimation
\begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Method** & **Data Representation** & **Datasets** & **Task** & **Model Architecture and Usage** \\ \hline
4 DRO [90] & AR dD point cloud & View-of-DelfIn-house & Odometry & Three modules consisting of feature encoding, initial pose estimation and pose warping to predict the ego-motion. \\ \hline Milli-RIO [91] & AR 3D point cloud & In-house & Odometry & LSTM layer as motion model. \\ \hline Autopace [92] & AR 3D point cloud & Nuscenes & Localization & CNN as spatial encoder and LSTM as temporal encoder. NetVLAD for local descriptor. \\ \hline Kidnapped radar [93] & SR polar scan & Oxford & Localization & VGG-16 encoder and NetVLAD descriptor. \\ \hline Look around you [94] & SR polar scans & Oxford & Localization & VGG-16 encoder and NetVLAD descriptor for sequential data. \\ \hline
[95] & SR cartesian scans & Oxford & Localization & VGG-19 or ResNet-152 as feature encoder and MLP as descriptor generator. \\ \hline Radar LCD [96] & SR cartesian scan & Oxford, MulRan & Localization & U-Net feature map and keypoint generation. PointNetVLAD for descriptors. \\ \hline Off the Radar [97] & SR cartesian scan & Oxford, MulRan & Localization & VAE for disentanglement of feature representation into uncertainty sources and invariant descriptor. \\ \hline RadarLoc [98] & SR cartesian scan & Oxford & Global pose prediction & Self-attention and DenseNet module for feature extraction. MLP for pose estimation. \\ \hline RSL-Net [99] & SR cartesian scan and satellite imagery & Oxford & Odometry & cGAN for satellite to radar. Multiple CNNs for disentanglement of rot. and trans. component. \\ \hline GRAMME [100] & SR cartesian scans and camera sequences & Oxford, RADIATE & Odometry & Four modules, including an attention module for fusion, predict ego-motion and ego-motion. \\ \hline L2R [101] & SR cartesian scan & Oxford & Localization on lidar map & Lidar-to-Radar GAN to perform classic ICP and particle filter for localization. \\ \hline RaLL [102] & SR cartesian scan and lidar map & Oxford, MulRan & Odometry & U-Net modules to calculate difference tensors between lidar embedding and radar scan. \\ \hline Pointing the Way [103] & SR cartesian scan, radar detections, (lidar) map & Boreas & Odometry & U-Net to generate mask for detection map. \\ \hline
[104] & SR polar scan and lidar map & Oxford, MulRan & Localization & Shared U-Net encoder between radar, lidar map, and positive / negative samples. FFT as final descriptor generation. \\ \hline RaLF [105] & SR cartesian scan and lidar submap & Oxford, MulRan, Boreas & Localization & RAFT feature encoder and flow field prediction. CNN for descriptor generation. \\ \hline Hidden-gems [106] & AR dD point cloud, lidar scan, camera input & View-of- Delf & Odometry and scene perception & Set conv point encoder, cost volume layer for scene flow predictions, ego-motion head, GRU for temporal updates. \\ \hline Seraloc [107] & SR cartesian map & In-house & Odometry & Semantic segmentation of points. \\ \hline \hline \end{tabular}
\end{table} 
Table 3: _Cont._where \(\hat{t}\) and \(\hat{R}\) are the predicted translation and rotation matrices in Equation (4). \(\alpha\) is a fixed or learnable parameter to adjust the error between translation and rotation weight. \(\mathbf{I}\) is the identity matrix. [PERSON] et al. [109] utilized a masking network in a U-Net encoder-decoder architecture to effectively filter out irrelevant regions within depth images, specifically for visual odometry. Drawing inspiration from this, [PERSON] et al. [84] employed a similar U-Net architecture to mask specific areas within a Cartesian radar representation image. The learned mask efficiently excludes non-relevant elements, such as noise, spurious detections, and moving objects. It is important to note, however, that this process disregards the Doppler velocity of detections. In this approach, the U-Net architecture serves as the sole component responsible for distinguishing relevant landmarks. [PERSON] et al.