Negative Object Presence Evaluation (NOPE) to Measure Object Hallucination in Vision-Language Models (2024)

Abstract: Object hallucination poses a significant challenge in vision-language (VL) models, often leading to the generation of nonsensical or unfaithful responses with non-existent objects. However, the absence of a general measurement for evaluating object hallucination in VL models has hindered our understanding and ability to mitigate this issue. In this work, we present NOPE (Negative Object Presence Evaluation), a novel benchmark designed to assess object hallucination in VL models through visual question answering (VQA). We propose a cost-effective and scalable approach utilizing large language models to generate 29.5k synthetic negative pronoun (NegP) data of high quality for NOPE. We extensively investigate the performance of 10 state-of-the-art VL models in discerning the non-existence of objects in visual questions, where the ground truth answers are denoted as NegP (e.g., "none"). Additionally, we evaluate their standard performance on visual questions on 9 other VQA datasets. Through our experiments, we demonstrate that no VL model is immune to the vulnerability of object hallucination, as all models achieve accuracy below 10% on NegP. Furthermore, we uncover that lexically diverse visual questions, question types with large scopes, and scene-relevant objects capitalize the risk of object hallucination in VL models.

heading: 1 Introduction
text: In recent years, vision-language (VL) research has witnessed a proliferation of studies focusing on diverse methods, models, and learning strategies aimed at bridging the performance gap between human and model capabilities (Yang et al., 2021; Yi et al., 2018; Zhou et al., 2020; Ray et al., 2019; Gokhale et al., 2020; Dai et al., 2021, 2022; Ishii et al., 2021; Lovenia et al., 2022; Ji et al., 2022b;
∗ The majority of the work was done when the author was studying at HKUST.
† Joint second authors.
Lovenia et al., 2023). Furthermore, researchers have constructed more rigorous VL benchmarks to continually raise the performance standard (Antol et al., 2015; Sheng et al., 2021; Li et al., 2021b; Goyal et al., 2017; Marino et al., 2019). However, despite these efforts, VL models continue to grapple with the persistent issue of object hallucination, where generated responses unfaithfully contain objects non-existent in the input images (Ji et al., 2022a; Rohrbach et al., 2018; Dai et al., 2023b; Kayhan et al., 2021). As illustrated in Figure 1, the failure of the model to faithfully ground the visual input leads to the production of unfaithful answers. These instances of object hallucination not only result in incorrect responses but also shed light on fundamental issues within VL models, such as over-reliance on unimodal priors (Jing et al., 2020; Agrawal et al., 2018; Gupta et al., 2022; Niu et al., 2021a) and statistical bias (Agrawal et al., 2016; Goyal et al., 2017; Agarwal et al., 2020). These underlying problems impede the models’ ability to comprehend the concept of non-existence.
Despite the critical importance of addressing object hallucination in VL models, only a limited number of previous works have focused on mitigating this issue, primarily due to the challenges
37
posed by the existing evaluation method in terms of generalization and scalability. CHAIR (Rohrbach et al., 2018) has primarily concentrated on evaluating non-existent objects based on handcrafted parsing criteria as well as a predefined list of object categories and their synonyms in the context of image captioning tasks, typically utilizing 80 object categories from MSCOCO (Rohrbach et al., 2018; Biten et al., 2022; Yi et al., 2018). However, the applicability of CHAIR to other datasets requires the generation of a new object category list, which exhibits varying levels of granularity across different studies (Dai et al., 2023b; Biten et al., 2022).
In this paper, we present NOPE (Negative Object Presence Evaluation) to quantitatively assess object hallucination through VQA. We establish a clear distinction between object hallucination and incorrectness as follows: a) object hallucination refers to the phenomenon in VQA where a VL model’s response includes a non-existent object, despite the ground truth answer being a negative indefinite pronoun (e.g., "none", "no one", "nobody", "nowhere", "neither") (Quirk et al., 1985) (NegP); and b) incorrectness occurs when a VL model fails to accurately respond to a question with a ground truth answer that is anything other than NegP, denoted as Others = P\NegP, where P represents the set of all phrases. By leveraging NegP, we evaluate object hallucination in NOPE, while Others allows us to assess normative correctness across diverse corpora. Our contributions are as follows:
1. By utilizing NOPE, we construct a VQA diagnostic benchmark to measure the object hallucination rate of VL models. Our experiment covers a balanced proportion of NegP and Others data with a total of ∼30k and ∼36k data in the dev and test sets, and includes 10 state-of-the-art VL baselines performances. We provide an in-depth analysis of the performances and limitations of the baselines.
2. We propose a novel automatic data generation pipeline to produce high-quality NegP VQA data from existing image captioning data by multi-turn prompting instruction-tuned large language models (LLMs). We verify and analyze our generated NegP data through automatic validation and human validation. Our list-then-rewrite method produces highquality NegP VQA data with 92% validity.
3. Through extensive analysis in NOPE, we find
that VL models tend to hallucinate more on data with higher lexical diversity, more scenerelevant objects, and larger answer scopes.

heading: 2 Related Work

heading: 2.1 Hallucination in Vision-Language
text: Only a few works study hallucination in visionlanguage, with the vast majority of them focusing on the task of image captioning. Rohrbach et al. (2018) propose CHAIR, an automatic evaluation metric to measure object hallucination in generated image captions, which is defined as a phenomenon where the models produce captions containing objects that do not exist in the input visual context. Rohrbach et al. (2018); Dai et al. (2023b); Sharma et al. (2018) also show that standard captioning metrics, e.g., CIDEr (Vedantam et al., 2015), METEOR (Banerjee and Lavie, 2005), SPICE (Niu et al., 2022), under-penalize object hallucination. These evaluations open up a way for efforts to mitigate hallucination in image captioning (Biten et al., 2022; Zhang et al., 2021; Xiao and Wang, 2021; Dai et al., 2023b). Concurrent to our work, Li et al. (2023b) propose POPE and frame the task of evaluating object hallucination as a binary-class VQA with only "yes/no" answer.

heading: 2.2 Question Generation for VQA Data
text: Most works rely on human annotators to generate visual questions with ensured quality: VQAv2.0 and VQAv1.0 (Goyal et al., 2017; Antol et al., 2015), Visual Genome (Krishna et al., 2016), Visual7W (Zhu et al., 2016), AdVQA (Sheng et al., 2021), Vizwiz (Gurari et al., 2018, 2019), TextVQA (Singh et al., 2019), R-VQA (Lu et al., 2018), VQA-Rephrasings (Shah et al., 2019), etc.
However, the cost of human annotation is expensive, thus encouraging the exploration of a more scalable option: automatic VQA data generation. Ren et al. (2015) present a simple question generation algorithm with a syntactic parser to convert image descriptions into QA forms. Johnson et al. (2017) use a functional program to generate synthetic images of objects as well as their relationships and relevant QA pairs using the groundtruth annotations. Kafle and Kanan (2017) populate multiple question templates with the image annotations (e.g., region descriptions, relationship graphs, bounding boxes) obtained from image captioning data to construct TDIUC. Changpinyo et al. (2022) annotate candidate answers by syntactically
parsing the captions, then derive questions from them. While prior studies focus on generating Others VQA data, we aim to generate NegP VQA data, which has never been done by past works.
3 NOPE to Overcome Limited NegP
As shown in Figure 2, there is only a minuscule amount of NegP data in the existing VQA datasets. In total, there are only ∼0.4% of the existing VQA datasets are NegP, which are not sufficient to assess object hallucination in VL. For this reason, we create NOPE through a novel NegP data generation method that aims to produce questions whose ground truth answers point to the absence of appropriate existent objects. Such ground truth NegP answers are denoted as ANegP = {”none”, ”nothing”, ”nowhere”, ”zero”, ”0”, ”no one”, ”nobody”, ”neither”}. We automatically generate synthetic NegP VQA data by leveraging the zero-shot prompting abilities of pretrained LLMs. To ensure the quality, we analyze the generated synthetic NegP VQA data through both automatic and manual human evaluation. The resulting NegP dataset is referred to as NOPE (Negative Object Presence Evaluation).

heading: 3.1 Prompting Methodology
text: We utilize an image captioning dataset Dcap = {(vi, ci, li}ni=1, where vi denotes a visual context, ci denotes a textual caption, and li denotes the relevant image label annotations (i.e., names of objects in vi). We rely on ci to describe the objects and the relationship between objects depicted in vi. We explore two prompting methods with varying degrees of flexibility to generate NegP questions from image captions: generate-from-scratch and list-then-rewrite. For clarity, we include all prompt templates with the examples in Appendix A
and the automatic validation methods to ensure the validity of the generated questions in Appendix C.
Generate-from-scratch In this method, we prompt an LLM to generate a question qi given three different variables: 1) an interrogative word wi ∈ {”what”, ”where”, ”how many”, ”who”, ”which”} to assert the question type needed for qi, 2) a ground truth NegP answer ai ∈ ANegP that matches wi, and 3) an image caption ci.
List-then-rewrite LLMs can infer conversational contexts and follow instructions over multiple turns (Nijkamp et al., 2023; Volum et al., 2022; Bang et al., 2023). Leveraging this multi-turn capability of LLMs, we frame our question generation task into two steps. (1) For object listing, given an image caption ci and the relevant object annotations li, we prompt an LLM to list m objects oi = {oi,j}mj=1 that are “closely related”1 but not mentioned. (2) For question rewriting, the LLM has to paraphrase a provided reference question, which is sourced from a diverse pool of humangenerated question templates with an object placeholder in Appendix B. After obtaining m listed objects from (1), we pick m random question templates from the pool and replace the object placeholders with the listed objects oi to construct the reference questions ri = {ri,j}mj=1. We prompt the LLM to paraphrase ri to qi = {qi,j}mj=1 to increase the lexical variety of the rewritten questions qi.

heading: 3.2 Human Evaluation Guidelines
text: We conduct a human evaluation to verify and analyze the quality of the generated questions obtained from §3.1, as well as measure the effectiveness
1We use “closely related” (hard) for brevity. However, this object-scene relevance can be switched to “loosely related” or “completely unrelated” in practice.
of the automatic validations performed. We employ three human annotators to perform the human evaluations. Detailed guidelines and examples are given prior to evaluation. We collect generated questions that are judged as valid and invalid by their automatic validation methods. Given a visual context, an image caption, a ground truth answer ∈ ANegP, and a generated question, the annotators are asked to determine whether: 1) the question is valid, 2) the question has a possible Others answer alternative, 3) the question does not match the answer (according to both the image caption and the image), 4) the question does not match the answer (only according to the image), or 5) the question is unclear or confusing. The examples provided for each category can be seen in Appendix E.

heading: 3.3 Results and Quality Analysis
text: Using automatic validation approaches explained in §3.1 and implementation details in Appendix D, we compare the capabilities of various instructiontuned LLMs in generating NegP VQA data. From the automatic validation results and analysis presented in Appendix F, we find that employing ChatGPT yields the highest-quality generated NegP questions by both generate-from-scratch and list-then-rewrite prompting methods, hence its use in the human evaluations. We conduct a human evaluation on randomly selected 150 generated questions from each method. For each sample, we ask 3 human experts to judge each generated question into one of the 5 options defined in §3.2.
Figure 3 shows the result of our human evaluation. For generate-from-scratch, only ±50% out
of the subset that is judged as valid by the automatic validation is actually a valid and appropriate NegP question according to the human annotators, and the rest is judged as incorrect by human annotators. The list-then-rewrite prompting approach, on the other hand, displays a significantly better question-answer generation quality with ±92% of the generated questions denoted as valid by the human annotators. This fact demonstrates that existing LLMs still fail to perform complex tasks in an end-to-end manner, while decomposing the complex tasks into several subtasks and coupling them with simple rule-based approaches can significantly improve the LLMs’ ability to perform the complex task effectively and efficiently.
A closer look at the questions generated by the generate-from-scratch method shows that while LLMs usually succeed in making questions in an end-to-end manner, 12% of the NegP generated questions include an existing object even though this information is sufficiently provided by the image caption. Moreover, 14% of the time, the generated questions also fail to include any objects and are overly generic, e.g., “What is not included in this image?”, which aligns with the observations of (Jang et al., 2023; Hosseini et al., 2021; Ettinger, 2020; Kassner and Schütze, 2020) that LMs perform poorly on negation and struggles to understand that negation changes semantics. These facts show that LLMs cannot consistently perform this implicit task breakdown. From this human evaluation result, we can conjecture that the
generate-from-scratch prompting method is not reliable and fails to elicit the LLMs’ understanding of complex tasks such as question generation. Using the list-then-rewrite method, we generate 29.5k NegP VQA data to build the NOPE dataset from OpenImagesV7 (Kuznetsova et al., 2020).

heading: 3.4 Dataset Statistics
text: NegP Question Distribution We cluster the generated questions into various types based on the starting n-grams in Figure 4. NOPE dataset exhibits a very broad lexical diversity of the generated questions, including variations in which the questions start with words other than the typical interrogative words (e.g., “what”, “where”, “how”, etc.), such as “Could you tell...”, “In what location...”, “Do you know...”, and more. This is vital to resist VL models’ notorious brittleness against linguistic variations (Shah et al., 2019; Ray et al., 2019; Kervadec et al., 2021; Whitehead et al., 2020).
Object-Scene Relevance Based on the descriptor used in the object listing step in list-then-rewrite1, the data in NOPE are divided into three categories. Figure 5 illustrates how these object-scene relevance descriptors of the generated NegP VQA data correspond to the relationship between the textual semantic similarity of the selected object and the image caption, as well as the image-text semantic similarity of the image and the QA pair. We compute the textual similarity using the SentenceTransformer library2 and the image-text similarity using CLIPScore (Hessel et al., 2021).
2https://www.sbert.net/docs/usage/semantic_ textual_similarity.html

heading: 4 Experimental Settings
text: The object hallucination benchmark consists of the validation and test sets of 10 VQA corpora, including NOPE (§3.4) with balanced object-scene relevance proportions. It displays the comparison between incorrectness and object hallucination over various baselines, which serves as a foundation for assessing object hallucination in addition to the standard incorrectness in 10 VL models.

heading: 4.1 Datasets
text: Table 1 describes the data distribution of the dev and test sets of the benchmark. Each set respectively comprises ∼30k and ∼36k data, maintaining near-balanced proportions of NegP and Others data. To ensure the quality of the visual questions in the benchmark, we also analyze the lexical diversity and the fluency of the comprising datasets, which are useful to assert a robust evaluation using questions that are linguistically diverse and coherent. In Figure 6, we show that the datasets whose data construction utilizes automatic question generation, i.e., NOPE and TDIUC, have comparable lexical diversity and fluency to the other datasets, which entirely rely on question generation by human annotators.
For lexical diversity, we employ length-agnostic lexical diversity metrics, i.e., moving average type-token ratio (MATTR) (Covington and McFall, 2010), measure of textual lexical diversity (MTLD) (McCarthy, 2005), and hypergeometric distribution diversity (HDD) (McCarthy and Jarvis, 2007, 2010), and average them. We use LexicalRichness (Shen, 2021, 2022) v0.5.03 to calculate these metrics. We also employ a large pre-trained LM GPT-Neo (Black et al., 2021) with 2.7B param-
3https://pypi.org/project/lexicalrichness/
eters to compute the perplexity of the questions, which is often used as a measure of both lexical diversity (Lewis et al., 2017; Tevet and Berant, 2021) and fluency (Fan et al., 2018; Wang et al., 2019; Cahyawijaya et al., 2021; Anonymous, 2023).

heading: 4.2 Baselines
text: For the baselines in our benchmark, we employ various vision-language model architectures on the benchmark in both zero-shot & few-shot and finetuned fashion. For the fine-tuned setting, we utilize five models: 1) OFA (Wang et al., 2022b), which unifies architectures, tasks, and modalities by formulating a unified sequence-to-sequence abstraction via handcrafted instructions to achieve task agnosticism; 2) and 3) BLIP (Li et al., 2022), which incorporates two key contributions, i.e., multimodal mixture of encoder-decoder (MED) to operate as either a unimodal encoder, an imagegrounded text encoder, or an image-grounded text decoder, and CapFilt as a new dataset bootstrapping method for learning from noisy image-text pairs; 4) ALBEF (Li et al., 2021a), which is trained using momentum distillation to improve learning from noisy web data; 5) GIT (Wang et al., 2022a), which employs an image encoder and a text decoder pretrained using a language modeling objective to map the input image to its corresponding description.
For the zero-shot setting, we employ: 1) BLIP2 (Li et al., 2023a), which utilizes a scalable multimodal pre-training method to enable any LLMs to ingest and understand images; 2) and 3) PromptCap (Hu et al., 2022), which is trained to generate captions that help downstream LMs answer visual questions; 4) InstructBLIP (Dai et al., 2023a), which is an instruction-tuned version of BLIP-2 on various tasks including VQA. We also employ 5) OpenFlamingo (Alayrac et al., 2022; Awadalla et al., 2023), which is an open-source version of
a large pre-trained VL model specialized in fewshot prompting, in the two-shot setting. Table 2 provides the model and data sizes of the baselines and Appendix H lists the model variants.

heading: 4.3 Evaluation Settings
text: For both NegP and Others, we compute accuracy and METEOR (Banerjee and Lavie, 2005) to measure the performance of vision-language models on the benchmark. While accuracy measures the performance based on an exact match between the generated answer and the ground truth answer, METEOR caters to partial (i.e., unigram) matches by computing a score for this matching using a combination of unigram-precision, unigramrecall, and alignment between the unigrams in the generated answer and ground truth answer. Additionally, for NegP, we employ a rule-based accuracy, referred to as NegP accuracy, which focuses on determining whether the generated answer is a negative indefinite pronoun (i.e., ∈ ANegP = {”none”, ”nothing”, ”nowhere”, ”zero”, ”0”, ”no one”, ”nobody”, ”neither”}) or not. All scores are computed per task and then the weighted averages according to each task size are retrieved.

heading: 5 Results
text: We present the results on the test set of the benchmark in Table 3. Examples of object hallucination are in Appendix I. While the VQA-finetuned baselines are slightly better at NegP and comparable to the zero-shot & few-shot baselines on Others, as in Figure 7, we observe that all zero-shot and VQA-finetuned baselines notably perform much worse on NegP tasks that Others with the averaged discrepancies of ±22% and ±18% accuracy,
respectively. This demonstrates that all baselines are more vulnerable and susceptible to object hallucination than standard incorrectness. In addition, less incorrectness does not entail less object hallucination. For instance, PromptCapBASE , PromptCap, and BLIP have lower scores on NegP than ALBEF despite outperforming it on Others setting. It also means that existing evaluations that solely utilize Others cases cannot effectively capture the models’ risk of object hallucination.
Another point that we observe is, GIT outperforms the other baselines on both NegP and Others data, as well as manages to surpass much bigger models (e.g., InstructBLIP and Flamingo), showing that GIT is more robust against both object hallucination and general incorrectness, despite being the smallest in size (Table 2) and having a simple architecture. This achievement could be attributed to its substantial number of pre-training images, which is an order of magnitude larger than those of the other baselines. This also aligns with (Hoffmann et al., 2022), in which for the same compute budget, a smaller model trained on more data outperforms a larger model trained on fewer data and achieves more optimal performance.

heading: 6 Analysis and Discussions

heading: 6.1 Object hallucination and lexical diversity
text: Table 3 also show that NegP performance scores on existing datasets are significantly higher than on NOPE across the metrics, indicating that ob-
ject hallucination is more likely to occur when the models attempt to solve the questions in NOPE. This is mainly due to the NOPE dataset having a relatively higher lexical diversity compared to the other NegP corpora, which are mostly composed of VQAv2 and Visual7W (see in Figure 6). This also aligns with the fact that NegP model performances have a strong negative Pearson correlation with the lexical diversity measures (r = {−0.8,−0.66,−0.65,−0.7} for METEOR and HDD, MTLD, MATTR, perplexity) and proves that corpora with higher lexical diversity (e.g., NOPE) provide more challenging NegP VQA problems to assess object hallucination.

heading: 6.2 Object hallucination and language bias
text: As shown in Figure 9, among 5 NegP question types, all VQA-finetuned VL models fail on NegP questions about color (e.g., “What is the color of...?”), object (e.g., “What is the object
beside...?”), and location (e.g., “Where is...?”), while most VL models tend to hallucinate less on NegP questions about counting (e.g., “How many...?”) and person (e.g., “Who is using...?”). A similar trend is observed for the zero-shot & fewshot baselines. We further inspect these two categories and find out that their answer scopes are of a smaller scope than the others in the training data. For instance, the answers to counting questions are often numbers ≤ 5, and the answers to the person questions are often the generic "man", "woman", "person", "people", and others which have fewer variations compared to object types, color names, or absolute and relative places. These facts suggest that existing VL models have a strong language bias (KV and Mittal, 2020; Niu et al., 2021b; Wu et al., 2022) toward certain question types, which result in acceptable NegP performances on those question types. Nevertheless, language bias does not solve object hallucination and even might make it worse, due to the VL models having weak visual grounding skills to verify the answer to the visual context, which might lead to errors on both NegP and Others questions.

heading: 6.3 Object hallucination and object-scene relevance
text: As shown in Figure 10, all VQA fine-tuned models perform lower when the object is closely related to the scene compared to when the object is loosely related or unrelated. This indicates that VL models have some degree of understanding NegP based on the relevance of the object in question with the scene. Although this helps VL models to understand about objects better in some cases, this also causes VL models to hallucinate more on objects that are relevant to the scene (Rohrbach et al., 2018; Kayhan et al., 2021; Dai et al., 2023b). On the other hand, the performance on loosely related or unrelated objects tend to be similar, which aligns with the similarity analysis provided in Figure 5. In contrast, for zero-shot & few-shot baselines, the differences between object-scene relevance are less apparent. However, in general, the NegP scores are also very low, except for BLIP-2, which suggests that most zero-shot models do not have an adequate understanding of NegP.

heading: 7 Conclusion
text: We have addressed the critical issue of object hallucination in VL models, which has been lacking a general measurement. We have introduced NOPE to assess object hallucination in VL models, investigating the discernment of objects’ non-existence in visual questions by 10 state-of-the-art VL models, alongside their standard performances. Additionally, we have presented a cost-effective and scalable method for generating high-quality synthetic data with over 90% validity to overcome the severe underrepresentation of NegP cases. Through our
comprehensive experiments, we have demonstrated that no VL model is exempt from object hallucination, highlighting their lack of understanding of negative object presence. Furthermore, we have identified lexical diversity, question type, and the relevance of the object to the visual scene as influential factors impacting VL models’ susceptibility to object hallucination. These findings provide valuable insights into the assessment of object hallucination in VL, thereby paving the way for the future development of enhanced VL models.

heading: 8 Limitation and Future Work
text: Evaluation Metrics for Object Hallucination In this work, we show three metrics to measure object hallucination and incorrectness, i.e., the exact match accuracy, METEOR, and NegP accuracy. Nevertheless, in some cases, these metrics fail to capture some equivalent answer that has the same semantic meaning. For example, given an NegP question “Where is the spoon in the picture?” with the corresponding label “Nowhere”, a system that answers with “There is no spoon in the picture” will get 0 scores on these three metrics, despite the answer is actually correct. We argue that the limitation of the existing metrics might hinder further research in alleviating object hallucination and we expect future works to focus on developing better metrics for measuring object hallucination.
Object Hallucination Outside of NegP Since object hallucination refers to an effect (i.e., generating non-existent objects) and not a cause, our measurement of object hallucination is limited to NegP cases, in which a VL model unfaithfully infers a supposedly non-existent object as existent in the visual context. For cases where a VL model provides an incorrect answer to Others VQA, the fine line between misclassification and object hallucination has not yet been defined.
Performances on Full Others Test Sets In order to observe the incorrectness of VL models on Others on various datasets, we compose a balanced set of ∼15k data in our dev split and ∼18k data in our test split from diverse VQA corpora. Obtaining the full performance on each of the source datasets requires re-running the baselines on the full test sets of each source dataset.

heading: 9 Ethics Statement
text: This research on object hallucination in visionlanguage models aims to improve the reliability and faithfulness of these models, which have significant applications in various fields such as healthcare and autonomous driving. We acknowledge the potential impact of our findings and commit to promoting responsible and ethical use of these models. We recognize that such models have the potential to perpetuate biases and stereotypes, and we have taken steps to mitigate this risk. For instance, we ensured that the synthetic data used in this study was generated in a manner that respects privacy and does not perpetuate biases or stereotypes. Furthermore, we recognize the importance of transparency and accountability in the development and use of these models. Therefore, we commit to sharing our findings and methodologies openly and making them accessible to the wider research community. We also acknowledge that these models can have unintended consequences and commit to ongoing monitoring and evaluation of their impact. Finally, we recognize that the development and use of these models must be guided by ethical principles that prioritize human well-being and social responsibility. We are committed to upholding these principles and contributing to the development of responsible and ethical practices in the field of vision-language modeling.

heading: Acknowledgements
text: This work has been partially funded by PhD Fellowship Award, the Hong Kong University of Science and Technology; PF20-43679 Hong Kong PhD Fellowship Scheme, Research Grant Council, Hong Kong; and the National Research Foundation, Singapore under its AI Singapore Programme.

heading: A Prompt Templates
text: We provide the prompt templates and examples for the generate-from-scratch and list-then-rewrite methods in Table 4 and Table 5, respectively.

heading: B Reference Question Templates
text: Table 6 presents the pool of question templates used to automatically build the reference questions for the list-then-rewrite in §3.1.
C Automatic Validation Methodologies of NegP VQA Data Generation
Generate-from-scratch To ensure the validity of qi, we use a model fine-tuned on natural language inference (NLI) to determine whether a generated question qi and answer ai pair (i.e., hypothesis) logically entails its corresponding image caption ci (i.e., premise). We also utilize a fine-tuned binary classifier to determine whether a generated question qi and answer ai pair fits a given visual context vi. If the question qi and answer ai pair is true (entailment) or undetermined (neutral) given ci as well as matches with vi, then the generated question qi is judged as valid by the automatic validation.
List-then-rewrite For the automatic validation of a listed object oi,j , we extract lemmatized noun tokens from its corresponding image caption ci and obtain the object names from li as the objects present in vi. If oi,j does not match with any of the extracted objects, then oi,j is a valid non-existent object. For the automatic validation of a generated question qi,j , if qi,j does not contradict its respective reference question ri,j , then the generated question qi,j is considered valid.
D Implementation Details of NegP VQA Data Generation
We implement §3.1 with the following LLMs that employ: 1) multi-task prompted fine-tuning, i.e., BLOOMZ (Muennighoff et al., 2022) and T0 (Sanh et al., 2022); 2) instruction meta-learning, i.e., OPT-IML (Iyer et al., 2022); 3) synthetic self-instruct, i.e., Alpaca (Wang et al., 2022c); 4) instruction (Wei et al., 2022a) and chain-of-thought fine-tuning (Wei et al., 2022b), i.e., FLAN T5 and FLAN Alpaca (Chung et al., 2022); 5) multi-task instruction pre-training, i.e., ChatGLM (Zeng et al., 2023); 6) conversation-style instruction tuning and reinforcement learning with human feedback (RLHF) (Christiano et al., 2017; Stiennon et al., 2020), i.e., ChatGPT (GPT-3.5). More details are presented in Table 7.
We utilize Open Images v7 as our image captioning dataset Dcap with respect to the provided splits. For automatic validation with NLI, we use the RoBERTa model fine-tuned on various NLI corpora that achieves the best performance on the Adversarial NLI benchmark (Nie et al., 2020).4 For automatic validation with image-QA pair classification, we build a simple CLIP-based (Radford et al., 2021) binary classifier. We provide the details in Appendix D.1. For the list-then-rewrite method, we use m = 10.
D.1 Image-QA Pair Classification
To construct a model for our image-QA pair classification, we construct a balanced image-QA corpus using NegP and Others VQA data randomly selected from 9 existing VQA datasets, i.e., VQAv2
4https://huggingface.co/ynie/roberta-large-snli_mnli_fever_anli_R1_R2_R3-nli
(Balanced Real) (Antol et al., 2015), AdVQA (Sheng et al., 2021), VizWiz (Gurari et al., 2018, 2019), TextVQA (Singh et al., 2019), R-VQA (Lu et al., 2018), Visual7W (Zhu et al., 2016), TDIUC (Kafle and Kanan, 2017), VQA-Rephrasings (Shah et al., 2019), and VQAv1 (Abstract Scenes) (Antol et al., 2015).
For the image-QA pairs from the NegP VQA data, we assign a binary label of 1 (valid), which means that the QAs correctly fit the corresponding images as valid pairs. For the Others VQA data, we replace the Others ground truth answers with NegP answers ∈ ANegP to make the invalid image-QA pairs (a binary label of 0). We split the corpus into 6k training, 2k validation, and 2k test set.
Using this corpus, we train a simple classifier with one hidden layer on top of a frozen CLIP (Radford et al., 2021). We leverage the image-text alignment learned by CLIP (Radford et al., 2021), which has been pre-trained on 400M image-text pairs using contrastive learning, to extract the image features of the images and the textual features of their question-answer counterparts. We simply concatenate both image and text features, then input them into the classifier. Our image-QA pair classifier yields an F1-score of 91.29% on the test set.

heading: E Human Evaluation Category Examples
text: We provide the human evaluation categories (§3.2) in Figure 11.
F Automatic Validation Results of NegP VQA Data Generation
Generate-from-scratch Figure 12 shows the proportions of valid generated NegP VQA data using 10 instruction-tuned LLMs listed in Appendix D over five different prompt templates, where each model generates 1k questions per template. The prompt templates are provided in Appendix A. The result shows that only ∼25% of the generated questions by the best-performing model, ChatGPT, are valid according to the automatic validation, while other models’ valid generated questions range from 6%-23%. This indicates that the task of NegP question generation is more complex and difficult than the instructions used to fine-tune the LLMs.
Next, we conduct a human evaluation on randomly selected 240 generated questions (i.e., 60 for each category in §3.2) by ChatGPT, which is the best-performing model. We ask 3 human experts to judge each generated question and answer pair into one of the five options defined in §3.2. Figure 13 demonstrates the result of our human evaluation. The result shows that automatic validation judgments do not agree with the human judgments on a considerable amount of the data, even for simple valid/invalid classification, the automatic validation judgments misclassify 27%-50% of the subsets. From this result, we can conjecture that our automatic validation approach is not effective at verifying whether the generated NegP questions are valid or invalid and that the generate-from-scratch prompting method is not reliable and fails to elicit the LLMs’ understanding of the task.
List-then-rewrite The automatic validation results on 100 generated questions (i.e., with the category proportion of 50, 35, and 15, respectively) by list-then-rewrite are provided in Table 8. The bestperforming model, ChatGPT, yields 98% valid questions with a valid non-existent object according to the automatic validation judgments, which is a huge improvement compared to generate-from-scratch. Similarly, Alpaca and ChatGLM also experience the same increase in validity (albeit not as significant), while the FLAN family models deteriorate due to their inability to handle lists inside the instructions, thus forcing them to respond with only one object instead of 10 objects (§D).
Our human evaluation on 300 generated questions by ChatGPT (presented in Figure 14) also proves that, when we omit the question generation on the wrong object, we can achieve around 90% high-quality NegP questions generated by the list-the-rewrite method. However, this method would benefit from the establishment of a more suitable penalizing method to filter out the generated questions that are inconsistent with the image captions.

heading: G Question Diversity of Existing VQA Datasets
text: We provide the illustrations of question diversity of existing VQA datasets: VQAv2 dataset (Antol et al., 2015) which utilizes a manual data generation method (presented in Figure 15a) and VQA-Rephrasings dataset (Shah et al., 2019) which utilizes an automatic data generation method (presented in Figure 15b).

heading: H Baselines in NOPE Benchmark
text: The variant details of the baselines used in NOPE are presented in Table 9.

heading: I Examples of Object Hallucination in NOPE
text: We list the examples of object hallucination from the dev set of NOPE in Table 10.

heading: Future Work
text: We argue that the limitation of the existing metrics might hinder further research in alleviating object hallucination and we expect future works to focus on developing better metrics for measuring object hallucination. Object Hallucination Outside of NegP Since object hallucination refers to an effect (i. e. , generating non-existent objects) and not a cause, our measurement of object hallucination is limited to NegP cases, in which a VL model unfaithfully infers a supposedly non-existent object as existent in the visual context. For cases where a VL model provides an incorrect answer to Others VQA, the fine line between misclassification and object hallucination has not yet been defined. Performances on Full Others Test Sets In order to observe the incorrectness of VL models on Others on various datasets, we compose a balanced set of ∼15k data in our dev split and ∼18k data in our test split from diverse VQA corpora. Obtaining the full performance on each of the source datasets requires re-running the baselines on the full test sets of each source dataset