Investigating and Mitigating Object Hallucinations in Pretrained Vision-Language ( CLIP ) Models (2024)

Abstract: Large Vision-Language Models (LVLMs) have achieved impressive performance, yet research has pointed out a serious issue with object hallucinations within these models. However, there is no clear conclusion as to which part of the model these hallucinations originate from. In this paper, we present an in-depth investigation into the object hallucination problem specifically within the CLIP model, which serves as the backbone for many state-of-the-art vision-language systems. We unveil that even in isolation, the CLIP model is prone to object hallucinations, suggesting that the hallucination problem is not solely due to the interaction between vision and language modalities. To address this, we propose a counterfactual data augmentation method by creating negative samples with a variety of hallucination issues. We demonstrate that our method can effectively mitigate object hallucinations for CLIP model, and we show the the enhanced model can be employed as a visual encoder, effectively alleviating the object hallucination issue in LVLMs.

## introduction current large vision-language models (lvlms) demonstrate significant potential in tasks requiring joint visual and linguistic perception, such as image captioning @xcite , visual question answering @xcite , visual grounding @xcite , and autonomous agents @xcite . despite the success of lvlms, previous studies have revealed that they commonly suffer from hallucinations in practice, including object hallucinations @xcite @xcite , spatial hallucinations @xcite , attribute hallucinations @xcite , etc. it is widely believed that hallucinations degrade model performance and reliability, and severely impair the user experience in real-world applications @xcite . in this work, we focus on investigating the causes of the highly-concerned object hallucinations, i.e., lvlms generate nonexistent objects in the image @xcite . a typical lvlm utilizes a large language model (llm) as its cognitive foundational model and employs a pre-trained image encoder as its visual perception module (mainly the clip encoder). @xcite investigated the spatial hallucination (e.g., confusing "left of" and "right of") in lvlms, and they found that various clip encoders struggle to recognize simple spatial relationships (achieving only a 55.0% accuracy on benchmarks, whereas humans are 98.8%). inspired by their findings, we hypothesize that the clip visual encoder might also be one of the causes of object hallucinations. hence, we first curate the object hallucination detection (ohd-caps) benchmark from subsets of the coco @xcite , flickr30k @xcite , and nocaps (as an out-of-domain benchmark because it comprises unseen objects) @xcite image caption datasets respectively, to more strictly measure the extent of object hallucinations present in clip encoders. we randomly select 16k/1k/1.5k (train/dev/test) samples, with each sample containing one image, one positive descriptive text, and 27 negative descriptive texts. the negative samples are perturbations of the positive sample, achieved by adding descriptions of nonexistent objects or reducing descriptions of existing objects. theoretically, a clip model without object hallucinations should accurately assign the highest clip score to the positive sample. however, taking the most commonly used "clip vit-l/14" in lvlms as an example, it only scores the highest for positive samples in 19.0% of cases. since we have observed that the clip encoder already has a serious issue with object hallucination, how can we mitigate it? in the contrastive pretraining of clip, negative samples come from text descriptions of other images within the batch, which makes the distinction between them quite straightforward. however, mitigating object hallucinations requires the clip encoder to be able to differentiate between subtle errors at the object level. we further fine-tune the clip model using the training set from ohd-caps. by incorporating a fine-grained object-level contrastive loss, we greatly reduce object hallucinations in the clip. then employing the fine-tuned clip as the visual encoder, the object hallucinations in our retrained lvlm, llava-1.5, are also diminished. in this paper, we study the object hallucinations of clip models. our main contributions are, ## the ohd-caps benchmark recent studies have found that lvlms are prone to object hallucinations @xcite . in response, researchers have developed several datasets to assess the extent of these hallucinations in such models @xcite . however, there is a relative lack of assessment work regarding the hallucinatory effects of the clip model, which is widely used as a visual encoder within lvlms. in this section, we introduce the object hallucination detection benchmark (ohd-caps) we create to evaluate the object hallucination problem in clip models and the pipeline for evaluations. figure 1 shows the object prone to hallucinations popular: person, wall, … random: boat, bird, … adversarial: road, grass, ... segmented objects mountain, tree, … a tower stands beside a road, overlooking a bunch of trees with a mountain in the background. a tower stands over a bunch of trees with a bird circling above and a mountain in the background. a tower stands over a bunch of trees with a mountain in the background. a tower towers over a bunch of trees with a mountain cityscape in the background. ## methodology we first revisit the training process of the vanilla clip model. let i be the image and t be the text, the training objective of clip is to maximize the similarity between the image and text pairs and minimize the similarity between the image and text pairs that are not matched. the loss function is defined as: with the addition of the negative samples t neg created as in the previous section, we could modify the loss l i2t as: model ohd-caps coco flickr30k nocaps avg. next, we assign different weights to the aforementioned loss terms, allowing the model to learn adaptively. consequently, the final loss function can be expressed as follows: training datasets we sample 8k images from the training set of coco and 8k images from flickr30k datasets, then generate negative samples for each image as in section 3. additionally, we randomly select ∼1k samples from the coco dataset's validation set as our dev set for the selection of hyper-parameters. detailed information about the dataset is provided in training details we utilize the clip vit/32-b and clip vit/14-l-336px implemented by huggingface @xcite as the initial models and conduct fine-tuning for 10 epochs. the training process is carried out on a single a6000 gpu, with batch sizes of 56 and 14 set for the base and large models, respectively, and the learning rate is set at 1e-6. the selection of hyper-parameters is determined by their performance on the validation set, where λ 1 and λ 2 are set as 0.1 and 0.1, τ 1 and τ 2 are set as 2. cifar-10 (2009) cifar-100 (2009) imagenet (2009) eurosat (2019) gtsrb (2012) stl10 (2011) .avg. top-1 acc. evaluation to verify the impact of our method on the model's generalization capabilities, we conducted zero-shot experiments on the following datasets: cifar-10/100 @xcite , imagenet-1k @xcite , dtd @xcite , eurosat @xcite , gt-srb @xcite and stl10 @xcite . ## conclusion our study investigates the reasons behind object hallucination in lvlms. we construct a benchmark specifically for the evaluation of hallucinations and find that the visual perception module commonly used in current lvlms, i.e., the clip model, cannot effectively discriminate hallucinated text. by designing negative samples and optimizing the contrastive loss function, we achieve a significant improvement in model performance on the hallucination detection dataset. moreover, replacing the original clip model with our improved model can effectively alleviate the issue of object hallucination in the llava model.