# Contrasting Dual Transformer Architectures for Multi-Modal Remote Sensing Image Retrieval
## 3 Experimental Results and Discussion
### Comparisons to State-Of-The-Art RS Methods  
In order to assess the effectiveness of our retrieval method, we compare it against different methods published recently, Tables 5-8 show the test results of the model on four datasets: TextRS, UCM, Sydney, and RSICD. Table 5 compares the proposed transformer-based model to the most recent retrieval techniques in terms of R@K for the TextRS dataset. The work proposed by [28] is based on a CNN and a Bi-LSTM for image and text encoding, respectively. The model was learned using a contrastive loss. While [21] used a bidirectional triplet network composed of an LSTM and pre-trained CNN. It is evident that the new methods significantly outperform the current techniques, while Table 6 contrasts the suggested model for the UCM dataset with recently published techniques such as VSE++ SCAN, MTFN, and AMFMN. Work [29] proposed a multi-modal feature matching model \begin{table}
\begin{tabular}{c c c c c c c} \hline \hline  & \multicolumn{3}{c}{**Text to Image**} & \multicolumn{3}{c}{**Image to Text**} \\ \cline{2-7}  & **Config. 1** & **Config. 2** & **Config. 3** & **Config. 1** & **Config 2** & **Config. 3** \\ \hline
**TextRS** & **56.93** & 52.34 & 51.56 & **56.96** & 52.74 & 51.18 \\ \hline
**Merced** & **58.25** & 49.62 & 50.22 & **49.99** & 48.91 & 47.26 \\ \hline
**Sydney** & **52.62** & 51.52 & 51.32 & **51.57** & 49.46 & 50.97 \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Average mR retrieval results for TextRS, Merced, and Sydney datasets based on three different configurations: Config. 1: models pre-trained on text–image matching; Config. 2: models pre-trained in a self-supervised mode; Config. 3: models pre-trained in a supervised mode. Figure 4: Examples for original image its corresponding attention (image and text) for dataset: (**a**) TextRS and (**b**) Merced dataset. which used a multi-scale visual self-attention module to extract visual features and a triplet loss for training. Finally, [30] proposed a model which incorporates an alignment module to address the semantic relationships between text and images with the help of attention mechanism and gate function. The obtained findings demonstrate the robustness of our model against SOTA models for valid comparison. Tables 7 and 8 states the comparison results. Table 7 shows the Sydney dataset results are reasonable compared with the most recent research. The performance of the RSCID dataset clearly demonstrates the robustness of the suggested method, which is presented in Table 8. \begin{table}
\begin{tabular}{c c c c c c c c c} \hline \hline \multirow{2}{*}{**Approach**} & \multicolumn{3}{c}{**Text Retrieval**} & \multicolumn{3}{c}{**Image Retrieval**} & \multirow{2}{*}{**mR**} \\ \cline{2-2} \cline{5-8}  & **R@1** & **R@5** & **R@10** & & **R@1** & **R@5** & **R@10** \\ \hline VSE++ [31] & 12.38 & 44.76 & 65.71 & 40.95 & 10.10 & 31.80 & 56.85 & 32.92 \\ SCAN [32] & 12.85 & 47.14 & 69.52 & 43.17 & 12.48 & 46.86 & 71.71 & 43.68 \\ MTFN [33] & 10.47 & 47.62 & 64.29 & 40.79 & 14.19 & 52.38 & 78.95 & 48.51 \\ AMFMN-soft [29] & 12.86 & 51.90 & 66.67 & 43.81 & 14.19 & 52.38 & 78.95 & 48.51 \\ AMFMN-fusion [29] & 16.67 & 45.71 & 68.57 & 43.65 & 12.86 & 53.24 & 79.43 & 48.51 \\ AMFMN-sim & 14.76 & 49.52 & 68.10 & 44.13 & 13.43 & 51.81 & 76.48 & 47.24 \\
**Ours** & 19.33 & 64.00 & 91.42 & 58.25 & 19.04 & 53.33 & 77.61 & 49.99 \\ \hline \hline \end{tabular}
\end{table} 
Table 6: Comparison between the state-of-the-art retrieval methods in the Merced dataset in terms of R@K.