# Gaze-Dependent Image Re-Ranking Technique for Enhancing Content-Based Image Retrieval
## 4 Experiment
### Comparison with State-of-the-Art Methods  
To establish the adaptability of the proposed method, we conducted a comparative analysis with current state-of-the-art techniques such as those presented in [55; 56]. The evaluation was performed using the HAIC datasets for both initial retrieval and re-ranking, and we present the corresponding results. Specifically, Tables 1 and 2 list the assessment results for Recall@k and NDCG@\(k\), respectively. In the evaluation process, we first calculated the initial results on the basis of a CMIR model (e.g., CLIP), and then the proposed method was employed to rearrange the initial retrieval results. The results in Tables 1 and 2 illustrate that after re-ordering using the proposed method, all re-ranking results outperform the initial retrieval results obtained from their respective baseline (BL) methods. This means that the proposed re-ranking method can improve the performance of the initial retrieval results calculated by the BL method in CMIR. These findings demonstrate the efficacy of the proposed re-ranking method in enhancing CMIR performance. Examples of retrieval results are depicted in Figure 4a,b. To better evaluate the retrieval results qualitatively, instead of using the whole desired image list as the ground truth in the quantitative evaluation, in this study, we set the first image of the desired image list as the relevant image and observed the rank of this image in the re-ranking results. In Figure 4a, the proposed method re-ordered an image from the third position in the initial retrieval to the first position, the same rank as the ground truth of the same image. Furthermore, in Figure 4b, we find that the ground truth image was ranked low in the initial retrieval (not in the top 3) but re-ordered in a higher rank after being processed by the proposed method. Finally, we demonstrate some results of the generated gaze-dependent image captions and the original images overlaid with the corresponding gaze trace in Figure 5. As shown in Figure 5, it can be seen that the gaze controls the description order of the generated caption. For example, in Figure 5a, the gaze goes through the sky, forest, and grass, and the caption also follows this order to describe the image. Based on this qualitative assessment, it is verified that the proposed method can enhance the overall retrieval performance beyond its initial state. Furthermore, the efficacy of the re-ranking technique that incorporates eye gaze data is substantiated. \begin{table}
\begin{tabular}{c c c c c c} \hline  & **Rcall@1** & **Rcall@5** & **Rcall@10** & **Rcall@50** & **Rcall@100** \\ \hline PVSE [55] & 0.027 & 0.074 & 0.118 & 0.266 & 0.336 \\
**PM (PVSE)** & **0.070** & **0.139** & **0.197** & **0.353** & **0.414** \\ \hline Ji ’19 [56] & 0.026 & 0.074 & 0.119 & 0.273 & 0.341 \\
**PM (ji ’19)** & **0.060** & **0.142** & **0.201** & **0.354** & **0.419** \\ \hline CLIP [23] & 0.080 & 0.220 & 0.298 & 0.423 & 0.479 \\
**PM (CLIP)** & **0.120** & **0.304** & **0.360** & **0.493** & **0.575** \\ \hline \end{tabular}
\end{table} 
Table 1: Comparison of Recall@k between the proposed method and state-of-the-art methods. By comparing the results of applying the proposed method to multiple advanced cross-modal image retrieval methods, the general applicability of the proposed method to improve retrieval performance can be analyzed. The bold number in the table is the best result. \begin{table}
\begin{tabular}{c c c c c} \hline  & **NDCG@1** & **NDCG@5** & **NDCG@10** & **NDCG@50** & **NDCG@100** \\ \hline PVSE [55] & 0.027 & 0.076 & 0.123 & 0.269 & 0.316 \\
**PM (PVSE)** & **0.070** & **0.143** & **0.207** & **0.341** & **0.372** \\ \hline Ji ’19 [56] & 0.026 & 0.078 & 0.125 & 0.273 & 0.317 \\
**PM (ji ’19)** & **0.060** & **0.148** & **0.216** & **0.349** & **0.382** \\ \hline CLIP [23] & 0.080 & 0.232 & 0.328 & 0.489 & 0.545 \\
**PM (CLIP)** & **0.120** & **0.317** & **0.398** & **0.549** & **0.629** \\ \hline \end{tabular}
\end{table}