## 2 Related Works
### Vision-Language Models  
Visionl-Language models are mainly divided into single-stream models and two-stream models. A single-stream model combines picture and text embeddings and feeds the result into a transformer model. On the other hand, a two-stream model allows the visual and text sides of the input data to be encoded separately using two independent transformers, where attention is inserted in the intermediate layer between the two encoders to merge the multimodule information.  
The visual encoder method primarily extracts features from images and feeds them into a multimodule model. This primarily consists of three aspects: object detection (OD)-based region features, convolutional neural network (CNN)-based grid features, and vision transformer (ViT)-based patch features. OD-based region features identify the target region in the given image using a pretarget detection model and extract the representation of each region as the image side input. CNN-based grid features use CNN models such as ResNet to extract information from the original image [11], tile the final CNN input features into a sequence, and feed them into the multimodule model. ViT-based patch features extract picture information using the patch embedding approach as a reference [50].  
VisualBERT splices text and image embedding sequences and feeds them into a transformer network using a single-stream model structure [51]. The network needs to clarify whether each embedding comes from a text measurement, an image measurement, or a position embedding. Then, the OD-based region feature extraction strategy is used to identify the target area on the image side and identify the results as the inputs for the next step. The text token is masked in one stage, and then the masked text is forecasted using additional text and image information in the next stage. Finally, the approach evaluates whether the image and text are consistent. Unicode-VL describes location data more thoroughly, creating a five-dimensional vector for each region [52]. The first four dimensions show the region's position with respect to the complete image, and the fifth dimension represents the region's size in proportion to that of the original image. VL-BERT adds a visual feature embedding module to the input, using the original image to improve the text encoding [53]. ImageBERT collects images and text from a website and ranks the positive sample pairings to incorporate more weakly supervised data and boost the learning effect [54]. Uniter calculates the corresponding relationship between the image and text embeddings via optimal transport [55]. In mask tasks, a sample only masks one module at a time during mask operations to avoid blocking critical information.  
Lxmert [56], an OD-based two-stream model, employs two encoders to encode pictures and words and an interactive transformer to fuse the two data streams. Instead of using the OD technique, Pixel-BERT employs a CNN to extract the characteristics of the original picture and then splices the image and text embeddings into the model, drastically reducing the model's complexity [57]. CLIP uses comparison learning to calculate the similarities between pictures and labels by gathering 400 million image-text pairs from the Internet as pretraining data; this approach has spawned a slew of CLIP-based models [45].  
In this research, we demonstrate the effectiveness of CLIP in long-tailed data and the benefits of comparative learning compared to standard classifiers. As a result, we offer a CLIP-based single-stage network, which improves the essential information between channels using visual attention, adjusts the weight balance via the seesaw loss, and achieves a more balanced performance across all categories.