# Image-Text Matching Model Based on CLIP Bimodal Encoding
## 1 Introduction  
In the multimodal field, image-text matching is an important task, with significant implications for cross-modal tasks such as image captioning [1] and machine translation [2]. In real life, images and text are the primary means through which humans acquire information. The task of image-text matching focuses on measuring the semantic similarity between images and text. The current mainstream approach involves mapping the features of images and text into a feature space, utilizing deep neural networks to extract image and text features, and then calculating their similarity or merging the two vectors before inputting them into a classification network for categorization. When using a sentence to describe an image, the sentence can be considered a weak annotation for the image. At a macro level, an image and a sentence correspond one-to-one; however, from a more detailed perspective, certain keywords in the text correspond to specific areas in the image, such as nouns or verbs corresponding to key pixels in the image. Therefore, image-text matching methods based on overall features also overlook, to some extent, the fine-grained correspondence between images and text [3, 4]. Image-text matching is a crucial task in the multimodal field, serving as a fundamental component for various cross-modal applications, such as image captioning and visual question answering. With the rapid development of artificial intelligence, accurately understanding and aligning visual and textual data has become increasingly significant. In real-world scenarios, images and text are the primary ways humans perceive and communicate information, making the ability to semantically match these modalities essential for building systems that can effectively interpret and utilize both types of content. The objective of image-text matching is to measure the semantic similarity between images and their corresponding textual descriptions. The mainstream approach involvesmapping image and text features into a shared representation space using deep neural networks. Typically, this involves extracting image features through vision models like the Vision Transformer (ViT), and text features through language models such as Bert. These features are then aligned through similarity computation or feature fusion, enabling tasks such as cross-modal retrieval and categorization. Despite significant advancements, current methods primarily focus on global semantic alignment, often overlooking the fine-grained correspondences between specific keywords in the text and visual regions in the image. For example, nouns or verbs in a sentence may correspond to particular objects or actions depicted in an image. Addressing this limitation requires models that not only align overall semantic features but capture intricate associations between textual and visual elements. In this work, we propose an enhanced image-text matching model inspired by Contrastive Language-Image Pretraining (CLIP). Our approach introduces a joint training framework utilizing the ViT and Bert encoders, optimized through the LiT-tuning paradigm. Furthermore, we adopt a cosine decay strategy to dynamically adjust the learning rate, enhancing model convergence and performance. We validate our model on benchmark datasets, including Flickr30k and WuKong, demonstrating significant improvements in key performance metrics and illustrating the effectiveness of our method. Our main contributions are summarized as follows:  
1. We present a novel image-text matching framework based on CLIP, integrating the ViT for image encoding and Bert for text encoding, and leveraging the LiT-tuning paradigm for enhanced training efficiency. 2. We implement a cosine decay strategy for adaptive learning rate adjustments, which improves convergence and model stability during training. 3. We conduct extensive experiments on the Flickr30k and WuKong datasets, achieving superior results compared to existing baselines and demonstrating the robustness of our model in multimodal alignment.