Negative Object Presence Evaluation ( NOPE ) to Measure Object Hallucination in Vision-Language Models (2024)

Abstract: Object hallucination poses a significant challenge in vision-language (VL) models, often leading to the generation of nonsensical or unfaithful responses with non-existent objects. However, the absence of a general measurement for evaluating object hallucination in VL models has hindered our understanding and ability to mitigate this issue. In this work, we present NOPE (Negative Object Presence Evaluation), a novel benchmark designed to assess object hallucination in VL models through visual question answering (VQA). We propose a cost-effective and scalable approach utilizing large language models to generate 29.5k synthetic negative pronoun ( NegP ) data of high quality for NOPE. We extensively investigate the performance of 10 state-of-the-art VL models in discerning the non-existence of objects in visual questions, where the ground truth answers are denoted as (e.g., “none”). Additionally, we evaluate their standard performance on visual questions on 9 other VQA datasets. Through our experiments, we demonstrate that no VL model is immune to the vulnerability of object hallucination, as all models achieve accuracy below 10% on NegP . Furthermore, we uncover that lexically diverse visual questions, question types with large scopes, and scene-relevant objects capitalize the risk of object hallucination in VL models.

## introduction in recent years, vision-language (vl) research has witnessed a proliferation of studies focusing on diverse methods, models, and learning strategies aimed at bridging the performance gap between human and model capabilities @xcite @xcite @xcite @xcite @xcite @xcite figure 1 : example of object hallucination and incorrectness in vqa. the model hallucinates a non-existent man sitting on the closest bench in the left image, while in the right image, it simply answers inaccurately. @xcite . furthermore, researchers have constructed more rigorous vl benchmarks to continually raise the performance standard @xcite @xcite @xcite . however, despite these efforts, vl models continue to grapple with the persistent issue of object hallucination, where generated responses unfaithfully contain objects non-existent in the input images @xcite @xcite . as illustrated in figure 1 , the failure of the model to faithfully ground the visual input leads to the production of unfaithful answers. these instances of object hallucination not only result in incorrect responses but also shed light on fundamental issues within vl models, such as over-reliance on unimodal priors @xcite @xcite and statistical bias @xcite @xcite . these underlying problems impede the models' ability to comprehend the concept of non-existence. despite the critical importance of addressing object hallucination in vl models, only a limited number of previous works have focused on mitigating this issue, primarily due to the challenges posed by the existing evaluation method in terms of generalization and scalability. chair @xcite has primarily concentrated on evaluating non-existent objects based on handcrafted parsing criteria as well as a predefined list of object categories and their synonyms in the context of image captioning tasks, typically utilizing 80 object categories from mscoco @xcite @xcite . however, the applicability of chair to other datasets requires the generation of a new object category list, which exhibits varying levels of granularity across different studies @xcite . in this paper, we present nope (negative object presence evaluation) to quantitatively assess object hallucination through vqa. we establish a clear distinction between object hallucination and incorrectness as follows: a) object hallucination refers to the phenomenon in vqa where a vl model's response includes a non-existent object, despite the ground truth answer being a negative indefinite pronoun (e.g., "none", "no one", "nobody", "nowhere", "neither") @xcite (negp); and b) incorrectness occurs when a vl model fails to accurately respond to a question with a ground truth answer that is anything other than negp, denoted as @xmath0, where p represents the set of all phrases. by leveraging negp, we evaluate object hallucination in nope, while others allows us to assess normative correctness across diverse corpora. our contributions are as follows: we provide an in-depth analysis of the performances and limitations of the baselines. our list-then-rewrite method produces highquality negp vqa data with 92% validity. ## nope to overcome limited negp as shown in figure 2 , there is only a minuscule amount of negp data in the existing vqa datasets. in total, there are only ∼0.4% of the existing vqa datasets are negp, which are not sufficient to assess object hallucination in vl. for this reason, we create nope through a novel negp data generation method that aims to produce questions whose ground truth answers point to the absence of appropriate existent objects. such ground truth negp answers are denoted as a @xmath1, "nothing", "nowhere", "zero", "0", "no one", "nobody", "neither"}. we automatically generate synthetic negp vqa data by leveraging the zero-shot prompting abilities of pretrained llms. to ensure the quality, we analyze the generated synthetic negp vqa data through both automatic and manual human evaluation. the resulting negp dataset is referred to as nope (negative object presence evaluation). ## experimental settings the object hallucination benchmark consists of the validation and test sets of 10 vqa corpora, including nope ( §3.4) with balanced object-scene relevance proportions. it displays the comparison between incorrectness and object hallucination over various baselines, which serves as a foundation for assessing object hallucination in addition to the standard incorrectness in 10 vl models. ## results we present the results on the test set of the benchmark in another point that we observe is, git outperforms the other baselines on both negp and others data, as well as manages to surpass much bigger models (e.g., instructblip and flamingo), showing that git is more robust against both object hallucination and general incorrectness, despite being the smallest in size ( ## conclusion we have addressed the critical issue of object hallucination in vl models, which has been lacking a general measurement. we have introduced nope to assess object hallucination in vl models, investigating the discernment of objects' non-existence in visual questions by 10 state-of-the-art vl models, alongside their standard performances. additionally, we have presented a cost-effective and scalable method for generating high-quality synthetic data with over 90% validity to overcome the severe underrepresentation of negp cases. through our comprehensive experiments, we have demonstrated that no vl model is exempt from object hallucination, highlighting their lack of understanding of negative object presence. furthermore, we have identified lexical diversity, question type, and the relevance of the object to the visual scene as influential factors impacting vl models' susceptibility to object hallucination. these findings provide valuable insights into the assessment of object hallucination in vl, thereby paving the way for the future development of enhanced vl models. object hallucination outside of negp since object hallucination refers to an effect (i.e., generating non-existent objects) and not a cause, our measurement of object hallucination is limited to negp cases, in which a vl model unfaithfully infers a supposedly non-existent object as existent in the visual context. for cases where a vl model provides an incorrect answer to others vqa, the fine line between misclassification and object hallucination has not yet been defined. performances on full others test sets in order to observe the incorrectness of vl models on others on various datasets, we compose a balanced set of ∼15k data in our dev split and ∼18k data in our test split from diverse vqa corpora. obtaining the full performance on each of the source datasets requires re-running the baselines on the full test sets of each source dataset.