# Survey of Deep Learning-Based Methods for FMCW Radar Odometry and Ego-Localization
## 5 Deep Learning Approaches
### Radar Based Loop Closure  
Deep learning methods have gained significant prominence in the domain of loop closure detection. They are commonly integrated into existing localization frameworks, often in conjunction with components such as a Vector of Locally Aggregated Descriptors (VLAD) layer. The VLAD layer, first proposed by [PERSON] et al. [113], summarizes local features from inputs into a compact representation. Local features are initially extracted from a given set of input data. These extracted features are then organized into clusters using a clustering algorithm like k-means clustering. This clustering step generates a hidden dictionary of visually meaningful words, also referred to as \"visual words\". For each local feature, a determination is made as to which visual word in the dictionary it is closest to. This assignment is made based on a chosen distance metric, such as the cosine similarity or Euclidean distance. After the assignments are made, the differences between the assigned visual words and the original features are accumulated and normalized. This aggregation process results in a compact descriptor that encodes the distribution of the differences across the visual words. The resulting dimension of the descriptor is calculated as the product of the number of visual words in the dictionary (C) and the initial dimension of the local features (\(D\)), yielding a descriptor of size \(VLAD=C\times D\). Building on this concept, the NetVLAD layer [114] extends the approach by introducing a learnable assignment stage prior to the VLAD aggregation. This involves incorporating a deep neural network that learns the encoding of the visual words and clusters. The training of the NetVLAD layer often employs a margin loss, which can be a triplet loss [115] or contrastive loss [116]. These loss functions guide the training process by encouraging meaningful representations in the learned descriptors space. \[\mathcal{L}_{\text{triplet}}(A,P,N)=\max(0,d(A,P)-d(A,N)+\eta) \tag{9}\]  
Equation (9) represents a standard triplet loss, where the variables have the following meanings: \(A\) refers to the anchor or query descriptor, \(P\) corresponds to positive descriptors that are physically close to the query location, and \(N\) denotes negative descriptors that are physically far away from the query location. \(\eta\) represents the margin. Positives are generally chosen as locations that are closer than 5 m within the query and negatives are further than 15 m away from the query location. [PERSON] et al. [92] fused a deep spatial-temporal encoder with a dynamic points removal scheme for pre-processing and a RCS based re-ranking score as post processing. The architecture is shown in Figure 6. The filtered representation is passed through a NetVLAD layer and optimized using a triplet loss to generate a unique scene descriptor. Finally, a RCS re-ranking is used to match scenes with similar RCS values, filtering out non-relevant stationary features, and reject places as potential loop closure detections if the histogram of RCS values are too dissimilar between a scan and a scan at a denoted keypose. The authors tested this method on the NuScenes dataset. [PERSON] et al. [93] employed a VGG-16 backbone with a NetVLAD layer to extract a global invariant descriptor from a RA map captured by a spinning radar. The approach capitalizes on the polar representation of the scan, incorporating circular padding, anti-aliasing blurring, and azimuth-wise max-pooling techniques to achieve rotational invariance for the local descriptor. The network is optimized using a triplet loss, enhancing the separation between different classes within the data and aiming to extract robust features for radar-based place recognition. Building on this work, [PERSON] et al. [94] created a contrast-enhanced distance matrix between embeddings from places along reference and live trajectories. Place recognition is then performed through a closest neighborhood search based on the distance matrices of different places.