# Image-Text Matching Model Based on CLIP Bimodal Encoding
## 4 Experimental Analysis
### Experimental Results and Analysis
\begin{table}
\begin{tabular}{c c c c c c c c} \hline \hline \multirow{2}{*}{**Model**} & \multicolumn{4}{c}{**Image to Text**} & \multicolumn{4}{c}{**Text to Image**} & \multirow{2}{*}{**rsum**} \\ \cline{2-2} \cline{4-7}  & **R@1** & **R@5** & **R@10** & **R@1** & **R@5** & **R@10** \\ \hline SGM [22] & 73.2 & 92.8 & 96.6 & 57.2 & 87.2 & 94.1 & 501.1 \\ NAAF [23] & 75.0 & 94.2 & 97.4 & 58.1 & 83.6 & 89.7 & 498.0 \\ CVSE [24] & 69.1 & 93.3 & 97.4 & 55.5 & 86.9 & 93.8 & 496.0 \\ \hline ours & 75.2 & 94.1 & 98.1 & 59.3 & 84.2 & 95.2 & 506.1 \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Results of ViT-Bert on Flickr30k. In the text retrieval direction, our model achieved a 2.0% higher R@1 value, and a 2.4% higher R@10 value compared to the maximum values from other models. In the image retrieval direction, our model achieved a 2.2% higher R@1 value, and a 2.1% higher R@10 value compared to the maximum values from other models. As shown in the above data table, there was a significant improvement in the recall rate when covering the top 10 retrieval results. Additionally, compared to the Flickr30k dataset, the improvement on the WuKong dataset was more significant, possibly due to the models being pre-trained on Chinese corpora. During the model training process, the changes in the model's loss were recorded. The corresponding trends are shown in Figure 5.  
From the above figures, it can be observed that within the initial few steps, the loss value rapidly increases. This is because the model is in the early stages of training and has low affinity for the relatively new dataset, resulting in significant changes in the loss value. Figure 5: Model Loss. (**a**) represents the loss trend of ViT16 under Bert-base; (**b**) represents the loss trend of ViT16 under Bert-base-fin; (**c**) represents the loss trend of ViT32 under Bert-base; (**d**) represents the loss trend of ViT32 under Bert-base-fin. \begin{table}
\begin{tabular}{c c c c c c c c} \hline \hline \multirow{2}{*}{**Model**} & \multicolumn{3}{c}{**Image to Text**} & \multicolumn{3}{c}{**Text to Image**} & \multirow{2}{*}{**rsum**} \\ \cline{2-2} \cline{4-7}  & **R@1** & **R@5** & **R@10** & **R@1** & **R@5** & **R@10** & \\ \hline SGM [22] & 71.2 & 83.1 & 90.3 & 46.5 & 63.4 & 72.6 & 427.1 \\ NAAF [23] & 67.0 & 84.5 & 91.4 & 58.3 & 79.6 & 88.4 & 469.2 \\ CVSE [24] & 70.2 & 79.4 & 88.2 & 66.7 & 71.9 & 80.2 & 456.6 \\ \hline ours & 73.4 & 81.3 & 93.5 & 68.7 & 61.8 & 90.8 & 469.5 \\ \hline \hline \end{tabular}
\end{table} 
Table 5: Comparison results of ViT–Bert on WuKong dataset. However, as the training steps increase, the overall training loss gradually decreases and eventually approaches 0, indicating that the model's parameters have gradually adapted to the entire dataset. During the initial training process, the objective function of the model fluctuated around a certain value without reaching convergence. This was due to the use of a large learning rate, which resulted in large parameter updates, preventing the model from approaching the optimal solution. Therefore, this paper employed a cosine decay strategy to dynamically adjust the learning rate, allowing the model to smoothly adjust the learning rate during training according to the cosine function. As the learning rate decreased, the magnitude of the model parameter updates also decreased, gradually improving the overall accuracy of the model. In this paper, several images and their corresponding text were randomly extracted from the test set. Different models were then used to calculate the corresponding similarity values. The model's performance was evaluated using similarity scores between images and text descriptions, which indicated how well the image matched each description. Scores closer to 1 suggested a higher degree of match, meaning the image was strongly related to the description, while scores closer to 0 indicated a weaker match and lower relevance. The results are shown in Table 6.