Exploring the Potential of Dense Information in Multimodal Alignment (2024)

Abstract: Despite the success of data augmentation in improving Contrastive Language-Image Pretraining (CLIP) model, existing methods that utilize large language model (LLM) or Segment Anything Model (SAM) to enrich the information in captions still suffer from several limitations, including insufficient detail and excessive hallucinations, ultimately resulting in compromised alignment and masking the true potential of dense information. This can lead to erroneous conclusions about CLIP’s ability to handle rich data, impeding the development of more effective models. To address the limitations of existing methods, we introduce a novel pipeline that generates highly detailed, factually accurate captions for images, which facilitates in-depth analysis of the potential for dense information in multimodal alignment. Contrary to previous findings, our investigation revealed that lengthening captions boosts performance across diverse benchmarks, even surpassing the effectiveness of meticulously crafted hard negative samples. Building on these insights, DELIP is introduced, demonstrably enhancing both foundational multimodal alignment and compositional reasoning abilities. Finally, we explore strategies to expand the context window of the text encoder, unlocking the potential of richer data for CLIP and paving the way for advancements in leveraging dense information for multimodal alignment.

heading: 1 Introduction
text: CLIP (Radford et al., 2021) learns alignment between text and visual modalities through contrastive learning on a massive dataset of image-text pairs. Its vision encoder has been widely applied to various downstream tasks, such as image captioning (Mokady et al., 2021; Cho et al., 2023; Hessel et al., 2022), visual grounding (Xiao et al., 2024),
*Work done during visiting student period. † Corresponding author
and visual question answering (Eslami et al., 2021; Song et al., 2022). However, recent studies (Zhao et al., 2023b; Yuksekgonul et al., 2023a; Thrush et al., 2022; Lewis et al., 2023) have shown that CLIP’s compositional reasoning ability is weak due to the use of noisy and simple text-image pair in the pre-training stage. This may lead to hallucinations when applying the vision encoder to downstream multimodal large language models (MLLMs). Hallucinations (Liu et al., 2024; Li et al., 2023c) refer to the model generating descriptions that are inconsistent with the input image. Since the architecture of MLLMs (Peng et al., 2023; Zhu et al., 2023; Li et al., 2023a; Liu et al., 2023b), is generally vision encoder, alignment module and large language model, if the representation obtained by the vision encoder is inaccurate or misleading, this error will be propagated.
Some studies have attempted to enhance the compositional reasoning ability of CLIP by improving the quality of captions (Doveh et al., 2023a), or constructing negative examples (Yuksekgonul et al., 2023b; Doveh et al., 2023a,b) related to objects, relationships, and attributes. One representative approach is Dense and Aligned Captions (DAC) (Doveh et al., 2023a), which enhances both the quality and density of captions and negative examples. However, the captions obtained by this method are shorter, lack details, and are more prone to hallucinations. Specifically, when using LLMs to enhance captions, they use the prompt "What can I see in a scene of {caption}?" to generate richer caption data. Obviously, this strategy of expanding captions from simple scenes will lead to the generation of details that do not exist in the image, i.e., hallucinations. They also proposed a method using Multiple Instance Learning (MIL) to alleviate this problem. Moreover, they only consider enhancing the density of text.
DCI (Urbanek et al., 2023) proposed a new benchmark to test the fine-grained understanding
13440
ability of the Vision Language Models (VLMs). They manually constructed detailed captions of each sub-crop of an image and the entire image, and tested the accuracy of the model matching each caption and subcrop. They also tested the negatives discrimination task and found that simply enhancing with hard negatives leads to a loss in the subcrop-caption matching task, demonstrating the limitations of previous work that enhanced performance just through negative examples.
In this paper, we propose DELIP, Density Enhanced Language Image Continual Pretraining, to analyze the impact of dense information on multimodal alignment. Specifically, we first propose a Crop-then-Merge data generation pipeline to generate detailed descriptions with fewer hallucinations, which is the basis for subsequent experiments. Then, we enhance the density of text information and image information respectively to study the impact on performance. We find that without using any tricks, we can outperform those methods that use hard negative examples on both the subcrop-caption matching task and the negatives discrimination task, which demonstrates the great potential of dense information in multimodal alignment. We then explore the strategy of expanding context windows and find that it can further improve the representation ability of the model. We also obtain some conclusions that are opposite to previous studies. Through analysis, we find that this is because the dense captions used in previous studies is either shorter or more hallucinatory. We hope that our research can help to recognize the potential of dense information and inspire future work to scale it up to a larger scale, which we call Information Density Scaling Law. Summarizing,
our key contributions are: 1.We propose a pipeline named Crop-thenMerge for generating dense information alignment data with minimal hallucinations..
2.We conduct a detailed analysis of the impact of dense information on multimodal alignment from multiple perspectives.
3.We propose a novel multimodal alignment model, DELIP, which is based on our analysis of the impact of dense information. DELIP achieves state-of-the-art results on DCI, and also demonstrates significant improvements on zero-shot classification, text-image retrieval tasks, and compositional reasoning ability benchmarks.

heading: 2 Related Work

heading: 2.1 Multimodal Alignment
text: This paper delves into the alignment of text and image modalities. Established approaches like CLIP (Radford et al., 2021) and ALIGN (Jia et al., 2021) utilize contrastive learning from scratch, bringing together related image-text pairs in terms of their global embeddings while pushing apart unrelated ones. Filip (Yao et al., 2021) and SPARC (Bica et al., 2024) refine this strategy by performing contrastive learning at the token level, enabling more granular alignment of representations. Flamingo (Alayrac et al., 2022) leverages cross-attention to establish connections between the modalities, followed by next-token prediction training to achieve alignment. BLIP2 (Li et al., 2023a) merges three distinct loss functions – imagetext matching, image-text contrastive learning, and image-grounded text generation – for comprehensive training.
Recently, numerous studies have employed align-
ment modules like linear projection layer (Zhu et al., 2023; Chen et al., 2023a), MLP (Liu et al., 2023b,a) or Qformer (Li et al., 2023a; Dai et al., 2023) to connect pre-trained vision encoders and large language models. Fine-tuning with visual instruction tuning data (Liu et al., 2023b; Zhao et al., 2023a; Gong et al., 2023) allows these multimodal large language model to process visual inputs. Notably, many utilize CLIP to generate visual representations or combine them with other visual models’ outputs. Given the extensive use of CLIP in MLLMs and the prominent role of contrastive learning in multimodal alignment, this paper investigates the influence of dense information on such methods.

heading: 2.2 Density of Alignment Dataset
text: CLIP models are trained on a massive amount of image-text pairs collected from the internet. However, some works (Thrush et al., 2022; Yuksekgonul et al., 2023a) have shown that such loose and short text-image pairs contain a lot of noise, which leads to poor compositional reasoning ability of the model.
LaCLIP (Fan et al., 2023) improves the diversity of text information by rewriting the style of the original caption, but this may introduce additional hallucinations. MLLM-augmented CLIP (Liu et al., 2023c) uses multiple MLLMs to generate caption data, which improves the diversity of expression and reduces hallucinations, but still covers little visual information. DAC (Doveh et al., 2023a) uses a two-stage approach to augment the data: first using MLLM to generate the base caption, and then using Segment Anything Model (SAM) (Kirillov et al., 2023) and LLM to augment the caption respectively, to obtain dense data. However, this approach introduces a lot of hallucinations, and in order to mitigate the impact of hallucinations, they propose to use the Multiple Instance Learning to mitigate this noise. FLIP (Li et al., 2023b) borrows the idea of MAE (He et al., 2021), due to the sparsity of image information, masks 75% of the patches of the input image, and accelerates the training process without sacrificing performance, which inspires us to explore the impact of enhancing the information density of images. All of these methods use text data lengths that are much less than 77, and FLIP even sets the max token of the text encoder to 32.
In this paper, we explore the impact of continual training of CLIP models with dense text informa-
tion and image information. In terms of text information, we analyze the impact of length, style, and negative samples. In terms of image information, we analyze the impact of the visual richness.

heading: 3 Approach

heading: 3.1 Dataset Generation Pipeline
text: We found that directly using MLLMs to generate image descriptions will miss some details, and if the model is forced to continue to output more details, the model is prone to hallucination. We think this is because of the resolution of the input image to the MLLM, which is 224*224 or 336*336, the model cannot see the specific details of the image and can only guess based on the high-level concept of the image.
We propose a Crop-then-Merge pipeline to generate detail-rich, low-hallucination data, as shown in Figure 1. Specifically, first, we generate a general but detail-poor description of the input image at low resolution. Then, we divide the input image into four parts and send each part to the GPT4V (OpenAI et al., 2024) to generate a description of each sub-image, which gives each sub-image a larger relative resolution and can capture richer and more accurate details. Next, we use the sub-image generated descriptions to enhance the general description, resulting in a description that contains richer details.
Additionally, in order to study the impact of training data with different styles, lengths, and negative samples on model performance, we perform different degrees of summary on the obtained detail enhanced global description to obtain long, middle, and short descriptions, and then rewrite the descriptions to generate different styles. Finally, multiple hard-negative samples are generated for each description.
Different from the previous method of simply replacing objects, attributes, and relationships in the description, we use LLM combined with carefully designed prompts to generate each negative sample, which is more natural and coherent, conforms to normal world commonsense, and is more difficult for the text encoder to discriminate.

heading: 3.2 Loss Function
text: We employed the InfoNCE (van den Oord et al., 2019) loss function during our analysis of the influence of various factors on model performance. The fundamental contrastive loss function used is given
by:
L = − log (
sim(Ii, Ti)
sim(Ii, Ti) + ∑N j=1 sim(Ii, Tj)
)
where Ii denotes the ith image, Ti denotes the text matching, Tj denotes all negative text samples, sim(I, T ) denotes the similarity between image I and text T .
Within each epoch of the final training phase, we randomly select one positive sample from a pool encompassing diverse lengths and styles, while incorporating all negative samples.

heading: 4 Experiment

heading: 4.1 Experimental Setup
text: In order to accommodate the amount of trainable model parameters and training data, we adopted different training configurations. In the process of searching for training hyperparameters and selecting prompts, we utilized training and evaluation sets of sizes 9500 and 500, respectively. The experiments were conducted on 8 * V100 GPUs with a per device batch size of 16, in order to keep a consistence with DCI.
We first hand-wrote a set of prompts as the sampling pool, and then used a progressive optimization approach to find the optimal prompt for each description generation stage. Our evaluation set was the aggregation of data generated by these prompts, while the training set was just the result of a single prompt. Ultimately, we selected the hyperparameters and prompts with the lowest loss on the sum eval dataset. In the first stage, we used GPT4V to generate descriptions for each sub-image. In the second stage, we used GPT-4-32k to aggregate the generated descriptions based on the global description. Since the third stage of data generation was mainly text rewriting (length, style), we found that GPT-turbo-3.5 could complete this task very
well. In order to save costs, we did not use the GPT4 interface. Since GPT-4 and GPT-3.5-turbo have been used as the target of open source models for a long time, and it is difficult to determine which open source model is the best, we used the OpenAI interface instead of deploying open source models locally.
The training process employed 8 * A100 80G servers, with a per-device batch size of 128. To mitigate catastrophic forgetting, two training methodologies were utilized: LoRA (Hu et al., 2021) and fully fine-tuning. In the LoRA training, an r value of 32, alpha of 32, and dropout rate of 0.1 were applied, specifically targeting the Q, K, and V modules. The corresponding learning rate was set to 5e-4. For full fine-tuning, a significantly lower learning rate of 5e-6 was chosen. An AdamW (Loshchilov and Hutter, 2019) optimizer with a learning rate warm-up to 3e-4 and a linear decay rate of 0.9 governed the optimization process. Training continued for 10 epochs.
During our experiments, we observed that in terms of speed, training conducted in a distributed manner showed no significant increase in speed when using LoRA compared to fully finetuning. In terms of performance, although LoRA reduced memory usage and allowed for an increased batch size, the loss associated with LoRA was significantly higher than that of fully finetuning. As a result, the test results on benchmarks were notably lower for LoRA compared to fully finetuning. Therefore, we ultimately chose to use fully finetuning for subsequent experimental analysis.

heading: 4.2 Evaluation

heading: 4.2.1 DCI
text: DCI (Urbanek et al., 2023) contains precise and reliable captions associated with specific parts of an image, which averages more than 1000 words. It evaluates the fine-grained understanding ability of VLMs through the task of matching each caption with its corresponding subcrop. To facilitate the evaluation of existing VLM models, they also released a version of sDCI that summarizes these detailed annotations to 77 tokens due to context limit.

heading: 4.2.2 ARO & VL-CheckList
text: Multiple benchmarks (Thrush et al., 2022; Zhao et al., 2023b; Yuksekgonul et al., 2023b) are proposed to evaluate the compositional reasoning ability of VLMs. In our work, to ensure consistency
with previous research, we adopted the widely used ARO (Yuksekgonul et al., 2023b) and VLCheckList (Zhao et al., 2023b). These benchmarks specifically target the assessment of models’ compositional reasoning ability of objects, relationships, attributes and order information.

heading: 4.2.3 CLIP-Benchmark
text: Prior work predominantly focused on improving the compositional reasoning ability of models and solely reported results on related benchmarks. However, we discovered that these methods can potentially affect the foundational multimodal capabilities of the model. To address this, we evaluate the model’s basic multimodal abilities using the CLIP-Benchmark. Specifically, text-image retrieval is employed to assess the model’s crossmodal alignment capacity, while zero-shot image classification serves to evaluate its generalization and semantic comprehension abilities.
Given the widespread application of CLIP’s vision encoder directly in MLLMs or in conjunction with DINO (Caron et al., 2021; Oquab et al., 2024) to provide hybrid representations (Lin et al., 2023; Jiang et al., 2024), independent evaluation of the vision encoder’s visual representation capability is of paramount importance. In image-text retrieval
and zero-shot retrieval, performance is influenced by both the vision encoder and the text encoder, making it challenging to attribute performance improvements solely to the vision encoder. In Section 4.5, we compare the results of training both the vision and text encoders with those of training only the text or vision encoder. We also evaluate the visual representation capability of the vision encoder through linear probing.

heading: 4.3 Text-rich Information
text: We used the data generation pipeline in Section 3.1 to generate data on the Laion (Schuhmann et al., 2021) dataset to analyze the impact of dense text information on model performance. We gradually increased the size of the dataset and trained the model while observing various metrics. When we found that the model’s performance was saturated, we stopped generating new data to save costs. Finally, the size of the dataset we obtained was 493k.
Our study reveals that, limited by a context window of 77, dense textual information significantly boosts the model’s performance on DCI benchmark, as shown in Table 2. This contradicts the DCI finding that improved performance via negatives-based training comes at the cost of decreased performance on subcrop-caption match-
ing. Notably, we demonstrate that dense information simultaneously enhances both subcrop-caption matching task and negatives discrimination task without resorting to extra hard-negative samples. This achievement surpasses the performance of NegCLIP (Yuksekgonul et al., 2023b) and other models. We speculate that this is because dense information contains more fine-grained alignment and more complex grammatical structures.
Similarly, our experimental results on various benchmarks also contradict Liu et al.’s (2023c) claims that using long descriptions will reduce model performance. We analyzed the differences between the data generated by simply using open source MLLMs and the data generated by our pipeline. We found that the data generated by the single MLLM has more hallucinations (Liu et al., 2024; Zhou et al., 2023; Liu et al., 2023c). Although the length is long, there are a lot of hallucinations in the latter half of the description, which may damages the potential of dense information.
Furthermore, we investigated the impact of scaling up the amount of dense text information on performance. Our findings demonstrate that using only 10k data points can significantly improve the model’s performance on the DCI, ARO, VLChecklist and CLIP-Benchmark, highlighting the efficiency of dense information.
While our study utilized dense descriptions, their length distribution diverged from shorter captions. Despite this, the DELIP model achieved enhanced performance in text-image retrieval, as shown in Table 4. We posit that this improvement stems from the inherent complexity and diversity of sentence structures within dense descriptions, coupled with their fine-grained detail. These characteristics align well with the simpler caption pattern typically
employed in text-image retrieval tasks. Table 5 reveals mixed results for the DELIP model across various datasets on zero shot classification task, exhibiting both improvements and declines. This inconsistency might be attributed to a mismatch between the prompt template for zero shot classification and the stylistic characteristics of our training data. In order to investigate the impact of different data generation styles on the model, we compared the effects of using data generated with a single prompt and data generated with a mixture of different prompts during the training phase. To improve computational efficiency, when using mixed data, a positive sample was randomly selected in each epoch, and the rest were used as in-batch negative samples. We found that using data with different styles leads to different improvements on different tasks, which is consistent with previous studies on the impact of training data distribution bias on model performance (Li et al., 2023b). However, despite the slight differences, the overall trend is upward, which demonstrates the robustness of dense information.
Although using dense information already outperforms methods that use hard-negative samples, we still investigated whether using negative samples can further improve the model’s performance. We found that using hard negative samples can significantly enhance the model’s compositional reasoning ability, but it has a certain degradation on zero-shot classification, which is consistent with the previous conclusions on using hard negative samples.

heading: 4.4 Vision-rich Information
text: We observed that the images in the Laion dataset are relatively simple and contain less visual infor-
mation. FLIP even masked out 75% of the patches to increase the batch size and accelerate training. Based on this observation, we explored the impact of using images with richer visual information on performance.
To maintain consistency, we used the same prompt and data generation pipeline to generate text descriptions. The only difference was that we used images from the SAM dataset. Due to the cost of the GPT-4V API, which is related to the pixel size of the input image, and the high resolution of the images in the SAM dataset, we only generated 10k training data based on SAM to save costs. In this paper, we mainly analyze the impact of dense visual information on performance under the same amount of data, and leave the work of scaling up data to future work.

heading: 4.5 Text-only v.s. Vision-only Tuning
text: Our findings suggest that fine-tuning either the text encoder or the vision encoder in isolation is insufficient for optimal performance. This limitation might stem from the inability of the text encoder to effectively represent dense text information during vision-only fine-tuning. The model, trained primarily on short and sparse caption data, might struggle with the richer and more complex nature of dense text, leading to degradation of the vision encoder’s performance as measured by linear probing. Additionally, fine-tuning the text encoder alone exhibits performance losses, implying that dense text goes beyond simply increasing length and sentence complexity. It introduces fine-grained alignment between the description and the image, necessitating joint training of both encoders for optimal learning.
Based on the linear probing results presented in Table ??, there appears to be minimal performance difference between our model and other
approaches. We think the reason may be 1. the capacity limitation of the vision encoder model; 2. DELIP has enhanced the ability to understand images at a fine-grained level, but the linear probe tests the model’s understanding of global information.

heading: 4.6 Scale up the Model Size
text: We found that increasing the capacity of the model can improve the performance of the model on DCI. We hope that our work can inspire future research to use dense information to scale up the model to billions or even larger scale, while current works (Dehghani et al., 2023; Sun et al., 2024; Chen et al., 2024) only scales up the size of the model, still using short text descriptions and sparse images.

heading: 5 Expand the Context Length
text: Previous works were hindered by a context window size of 77, preventing the model from effectively utilizing the information contained in longer captions. To address this limitation and harness the potential of denser text information, we explore increasing the context window to 512 tokens and analyze its impact on model performance.

heading: 5.1 Strategies
text: We use the following strategies to expand the context windows:
Train Position Embedding from Scratch: The length of the position embedding is the primary factor limiting the context window. Based on this, we reinitialize the position embedding with a length of 512 and then retrain the entire model. To investigate the impact of the length distribution of the text data, we compared two strategies: training only with long descriptions and training with a mixture
of long descriptions and original captions. Position Interpolation: We use interpolation (Chen et al., 2023b) to expand the position embedding; specifically, the position embedding is interpolated to a length of 512 as the initial embedding, and then we continue to train the model with the same data settings.
Pretrained Text Encoder: Besides expanding the position embedding based on the text encoder of CLIP, another approach is to use a pretrained language model with a longer context window as the text encoder. AltClip (Chen et al., 2023c) and NLLB-Clip (Visheratin, 2023) used a pretrained multilingual text encoder for replacement, which facilitates the performance of CLIP on low-resource languages. Different from previous work, we explore the impact of using pretrained language models on processing longer text descriptions.

heading: 5.2 Analysis
text: We found that if we only use long descriptions to train position embeddings from scratch, the model can perform well on DCI, but the performance loss is very serious on clip-benchmark; after mixing the original caption data for training, the model’s performance on zero shot classification and retrieval tasks is restored, but still lower than CLIP-base. We think this is due to the insufficient training data, we only used 493k data for training, which is lower than the data required for pre-training. Due to the limitations of API cost and computing resources (the time and memory required to train the model with a context window of 512 is much higher than 77), and the main purpose of this paper is to explore the potential of dense information rather than pre-training, we leave using dense information to train the clip model from scratch as future work.
The effect of position interpolation is not very good. We think it is because CLIP’s position embedding is obtained through training rather than rotary position embeddings (Su et al., 2023; Touvron et al., 2023). The gap between these two methods
of injecting position information leads to position interpolation not being very suitable for CLIP-style models. A potential high-efficiency solution is to use a text encoder with rotary position embedding to pre-train on short text, and then extend it to long text through position interpolation combined with fine-tuning.
While incorporating longer context via BERT (Devlin et al., 2019) and RoBERTa (Liu et al., 2019) models did improve overall results, the difference between their impacts was marginal. Interestingly, training solely on dense descriptions led to more strong performance in negative discrimination, even without hard negative samples. However, combining dense descriptions with original captions caused a performance drop in negative discrimination, while subcrop-caption matching remained unaffected. This suggests that dense descriptions have potential to enhance the model’s ability to differentiate hard negatives, a crucial aspect of its representation ability.

heading: 6 Future Work
text: Firstly, in this paper, we use GPT-4V to generate descriptions for each image or subcrop and then use GPT-4 to merge these generated descriptions into a global description with detailed details while minimizing hallucinations. However, calling the GPT-4 API is expensive. Therefore, it is necessary to explore how to use open-source MLLMs to generate high-quality descriptions and how to use open-source LLMs to merge different descriptions together.
Secondly, we have only divided a single image into four parts, which still may overlook some details if the original image’s input resolution is high and contains many details. A potential direction worth exploring is to divide the image into more parts to generate descriptions separately, which would improve the overall richness of details. However, it is essential to note how to effectively merge these individual parts without introducing addi-
tional hallucinations (e.g., if an object is incorrectly divided into multiple parts), which is a problem worth studying.
Thirdly, when merging the descriptions of each part, we explicitly marked the relative position of each part in the original image (e.g., top left, bottom left, top right, bottom right) in the prompt. However, if the image is divided into more parts, marking the position information of each part becomes difficult. A straightforward idea is to use a coordinate system for annotation. We leave this part of the exploration work for the future.

heading: 7 Conclusion
text: We conducted a systematic analysis of the impact of dense information on multimodal alignment. Analyzing the impact of increasing the density of text information and vision information on model performance, we found that data rich in detail and with less hallucination improved the model’s fine-grained alignment and enhanced its ability to distinguish negatives. Interestingly, simply using density-enhanced training data already outperformed using carefully designed hard negative samples. We also tried various strategies to further increase context windows, finding that it further improved the model’s representation ability, showcasing the great potential of dense information for enhancing multimodal alignment.
Limitations
In this paper, we utilized continual training to explore the impact of dense information on multimodal alignment. Due to the analysis of multiple variables, the total experimental time reached 21 days on 8 * A100 80G servers, despite employing base-sized models.
1. No Pretraining for Dense Information Impact Observation: While the configuration and time were sufficient for pre-training CLIP-base, our employed description length significantly exceeded the original captions in the Laion400m dataset. This resulted in limitations in batch size and training speed. 2. Data Saturation via GPT-4v Only: To optimize cost, we used GPT-4v to generate data sufficient for model saturation, without creating further variations. Scaling up the dataset alongside the model size, though not implemented due to cost constraints, was expected to improve performance. 3. English-Only Experiments: While a body of work exists on multilingual CLIP, our data con-
struction and experiments exclusively utilized the English language. Investigating how to construct dense image-text pairs in low-resource languages presents a valuable avenue for future exploration.
Ethics Statement
By enhancing information density, we have improved multimodal alignment. However, hallucination remains unavoidable, which may produce outputs that do not align with facts and could mislead users. Therefore, it is necessary to manually review the model’s outputs to ensure safe use in downstream applications.

heading: Acknowledgments
text: This work was supported by the Shenzhen Science and Technology Program (JCYJ20220818103001002), Shenzhen Doctoral Startup Funding (RCBS20221008093330065), Tianyuan Fund for Mathematics of National Natural Science Foundation of China (NSFC) (12326608), Shenzhen Key Laboratory of CrossModal Cognitive Computing (grant number ZDSYS20230626091302006), and Shenzhen Stability Science Program 2023, Shenzhen Key Lab of Multi-Modal Cognitive Computing.

heading: Future Work
text: Investigating how to construct dense image-text pairs in low-resource languages presents a valuable avenue for future exploration