# Revisiting Low-Resolution Images Retrieval with Attention Mechanism and Contrastive Learning
## 3 Results
### Quantitative Results
#### 3.3.5 The Effectiveness of Attention Mechanism  
Finally, we analyze the effectiveness of attention embeddings, which is the key proposal in our study. ViT architecture is built upon the attention mechanism, where each image patch has its own attention weight to determine which region should be focused. In this study, we also investigate the effectiveness of the attention mechanism by analyzing the number of \"decisive\" patches corresponding with the highest attention weights. Table 6 illustrates that our simple method achieves a better result using 25 decisive patches (recall@1 = 0.9492, mAP = 0.9475). The case of only one decisive patch means that only  
\begin{table}
\begin{tabular}{c c c c c c c} \hline \hline \multirow{2}{*}{**Dimensions**} & \multicolumn{5}{c}{**Recall@k**} & \multirow{2}{*}{**mAP**} \\ \cline{2-2} \cline{4-6}  & **1** & **2** & **3** & **4** & **5** \\ \hline
64 & 0.9296 & 0.9843 & 0.9843 & 0.9843 & 0.9843 & 0.9507 \\
128 & 0.9414 & 0.9648 & 0.9765 & 0.9765 & 0.9804 & 0.9438 \\
256 & 0.9453 & 0.9726 & 0.9843 & 0.9843 & 0.9882 & 0.9555 \\
512 & 0.9101 & 0.9687 & 0.9882 & 0.9882 & 0.9960 & 0.9339 \\
1024 & 0.9257 & 0.9570 & 0.9726 & 0.9726 & 0.9765 & 0.9336 \\ \hline \hline \end{tabular}
\end{table}
Table 4: Recall@k and mAP comparison using different numbers of embedding dimensions.  
\begin{table}
\begin{tabular}{c c c c c c c} \hline \hline \multirow{2}{*}{**Loss Components**} & \multicolumn{5}{c}{**Recall@k**} & \multirow{2}{*}{**mAP**} \\ \cline{2-2} \cline{4-6}  & **1** & **2** & **3** & **4** & **5** \\ \hline \(\alpha=1,\beta=0,\gamma=0\) & 0.6210 & 0.7070 & 0.7382 & 0.7890 & 0.8164 & 0.6026 \\ \(\alpha=0,\beta=0,\gamma=0\) & 0.6484 & 0.7265 & 0.7773 & 0.7890 & 0.8046 & 0.6214 \\ \(\alpha=1,\beta=0,\gamma=1\) & 0.8437 & 0.8984 & 0.9375 & 0.9492 & 0.9570 & 0.8445 \\ \(\alpha=0,\beta=1,\gamma=0\) & 0.8789 & 0.9531 & 0.9726 & 0.9726 & 0.9726 & 0.8929 \\ \(\alpha=1,\beta=1,\gamma=0\) & 0.9023 & 0.9531 & 0.9726 & 0.9765 & 0.9765 & 0.9120 \\ \(\alpha=0,\beta=1,\gamma=1\) & 0.9414 & 0.9687 & 0.9765 & 0.9765 & 0.9765 & 0.9379 \\ \(\alpha=1,\beta=1,\gamma=1\) & 0.9296 & 0.9804 & 0.9804 & 0.9843 & 0.9882 & 0.9268 \\ \(\alpha=0,\beta=0,\gamma=1\) & 0.8867 & 0.9296 & 0.9531 & 0.9531 & 0.9531 & 0.8910 \\ \hline \hline \end{tabular}
\end{table}
Table 5: Recall@k and mAP comparison between using different loss function settings of ViT model.  
the most attentive region is extracted, while the case of 128 decisive patches reflects that the average attention of all patch's embeddings is extracted. We compare our findings with a previous study [6] to address our results, as shown in the CGD row. The study from [6] is a typical study about image retrieval where a ResNet-based model is fine-tuned using combined global descriptors, and this study achieved significant performance on a broad range of datasets. We verify that our proposed framework outperforms CGD in both terms of ranking recall and mAP. Together, the present findings confirm that using attention patch's embedding is slightly more robust than class's embedding, as suggested in [12]. To conclude this section, we show that fine-tuning the ViT model with contrastive learning provides substantially better results than the direct use of features extracted from the pre-trained model, that is, an mAP of 0.54 without fine-tuning compared to an mAP of 0.6214 when fine-tuned with supervised contrastive learning. This phenomenon can be extended to other models as well.