# Image-Text Matching Model Based on CLIP Bimodal Encoding
## 4 Experimental Analysis
### Experimental Results and Analysis  
This paper utilized two different image encoders and text encoders, resulting in a total of four combined models. The image encoders were ViT32 and ViT16, while the text encoders were Bert-base-chinese and Bert-base-fin. The text encoder employed pre-trained  
Figure 4: Learning Rate Decay. Bert weights on Chinese corpora, which made it more suitable for the WuKong dataset used in this study. To evaluate the model's performance on image-text matching tasks, we used R@n (Recall at n) as a primary metric. R@n measured the proportion of correct matches within the top-n retrieved results. Specifically, R@1, R@5, and R@10 represented the recall rates when the correct match appeared within the top 1, 5, and 10 retrieved results, respectively. This metric provided a clear indication of the model's ability to rank relevant items near the top of the retrieval list. To validate the effectiveness and feasibility of our model, we conducted comparative experiments on the Flickr30k and WuKong datasets, calculating the corresponding metrics. In these experiments, we selected SGM [22], NAAF [23], and CVSE [24] as baseline methods for comparison. These methods were representative models in the field of image-text matching: SGM (Similarity Graph Modeling) achieved efficient image-text matching by constructing a similarity graph between images and text; NAAF (Non-local Attention and Adaptive Fusion) introduced a non-local attention mechanism and adaptive fusion strategy to enhance cross-modal retrieval performance; and CVSE (Conditional Variational Semantic Embedding) utilized a conditional variational auto-encoder for semantic embedding of images and text. We chose these baseline methods due to their outstanding performance on public datasets and their diverse model architecture and feature extraction approaches, which allowed for a comprehensive evaluation of our proposed method's performance. Comparative experiments were conducted on the two datasets, and the corresponding metrics were calculated to validate the model's effectiveness and feasibility. The experiments encompassed both image retrieval and text retrieval in multimodal matching, with results on the Flickr30k dataset shown in Table 4.  
In the Image-to-Text task, the multimodal matching model proposed in this paper achieved an R@1 value 0.2% higher than NAAF and an R@10 value 0.7% higher than the comparative models NAAF and CVSE. In the Text-to-Image task, the R@1 value was 1.2% higher than the NAAF in the comparative model, and the R@10 value was 1.1% higher than SGM. According to the results in the table above, it was evident that in the Image-to-Text task, when k = 10, the coverage of the retrieval model generating relevant text information was significant. Similarly, in the Text-to-Image task, when k = 10, the coverage was also substantial. This was because in the R@k recall rate, k represented the top k returned results, and the meaning of R@k was the proportion of correct results in the top k results. As k increased, more results were included, and the proportion of correct results also increased. In addition to the Flickr30k dataset, this paper also used a WuKong dataset, which was formatted as a CSV file. The URL field in the dataset represented the web address of the image, while the Text field contained textual descriptions corresponding to the images. Therefore, it was necessary to download the images to the local environment using the URLs. This dataset consisted of one image paired with one textual description. Comparative experiments were also carried out with relevant models on this data set. The results of comparative experiments of relevant indicators are shown in Table 5.