# A Novel Geo-Localization Method for UAV and Satellite Images Using Cross-View Consistent Attention
## 3 Proposed Method
### Vision Transformer  
The features in cross-view images may differ greatly from various perspectives. Although many CNN-based methods can also complete feature extraction, these methods ignore the global context information, encountering many challenges in cross-view image retrieval. The network architecture based on Transformer can encode images using contextual information. This approach allows for better performance in geo-localization tasks across perspectives. As shown in Figure 2, the transferred UAV images and satellite images are cropped into \(n\) image patches with the size of \(16\times 16\), which are then embedded through linear projection as the input patch tokens. Meanwhile, the class token is a vector inputted into the Transformer block together with the input tokens. As a result, the Transformer takes \(n+1\) tokens with \(D\) dimensions as input and only the output of the class token is used for embedding the output feature vector. The whole model only outputs the embedding result of the class token, and other patch tokens do not directly output the embedding result. This architecture forces the propagation of information between patch tokens and class tokens. Since the Transformer block does not consider location information, changing the sequences of image patches has no influence on the result. Positional Encoding is introduced to represent the context information of images, which is added to the input tokens and class tokens before the first transformer block. The method of positional encoding is absolute position encoding, as it can quickly encode the location information of each image block and retain the position relationship between different blocks. Positional Encoding describes positional information, which is beneficial for the model to extract cross-view information. The input tokens and the class token combined with the Positional Encoding are then inputted to the Transformer encoder. A transformer block consists of Multi-head Self Attention layers (MSA) and Feed-Forward Network (FFN) layers. In the Multi-head Self Attention layers (MSA), the inner product of the query matrix \(Q\in\mathbb{R}^{k\times d}\) and the key matrix \(K\in\mathbb{R}^{k\times d}\) is normalized by the softmax function. The result was multiplied by the Value matrix \(V\in\mathbb{R}^{k\times d}\) to get the final output of the attention matrix:  
\[\text{Attention}(Q,K,V)=\text{Softmax}(\frac{QK^{T}}{\sqrt{d}})V \tag{1}\]  
Figure 2: The proposed method consists of four main parts: a color transfer module, a one-stage training method for image retrieving, a piecewise soft-margin triplet loss function, and a similarity calculation module for the test images. For the Multi-head attention with \(h\) heads, the outputs are concatenated to get a \(N\times dh\) sequence, and the \(N\times D\) final output is obtained through the linear transformation. The Feed-Forward Network (FFN) is added to the MSA layer, composed of two linear layers separated by a GeLu activation function. The first linear layer expands the dimension from \(1D\) to \(4D\), and the second layer reduces it back from \(4D\) back to \(1D\). The MSA and FFN layers operate as residual operators thanks to skip connections and layer normalization. The encoder output of the class token is then inputted in the MLP Head to get the embedded UAV feature vectors and satellite feature vectors, which represent the information of the cross-view images, and are then compared to get the corresponding UAV and satellite image pairs. The dimension of the embedded feature vector is decided by the structure of MLP Head, which impacts the image retrieval result shown in the experiment section. The DeiT-S [17], pre-trained on ImageNet, is introduced to our method. The DeiT-S structure in this method contains 12 Transformer layers, and the embedded dimension is 384. In each Transformer layer, there are 6 heads in the multi-head attention.