Evaluating Object Hallucination in Large Vision-Language Models
Authors: Yifan Li
Yifan Du
Kun Zhou
Jinpeng Wang
Xin Zhao
Ji-Rong Wen
Venue: EMNLP 2023 Main (2023)
Keywords: Large Vision-Language Model
Object Hallucination

Abstract: Inspired by the superior language abilities of large language models (LLM), large vision-language models (LVLM) have been recently proposed by integrating powerful LLMs for improving the performance on complex multimodal tasks. Despite the promising progress on LVLMs, we find that they suffer from object hallucinations, i.e., they tend to generate objects inconsistent with the target images in the descriptions. To investigate it, this work presents the first systematic study on object hallucination of LVLMs. We conduct the evaluation experiments on several representative LVLMs, and show that they mostly suffer from severe object hallucination issues. We further discuss that the visual instructions may influence the hallucination, and find that: objects that frequently appear in the visual instructions or co-occur with the image objects are obviously prone to be hallucinated by LVLMs. Besides, we further design a polling-based query method called POPE for better evaluation of object hallucination. Experiment results show that our POPE can evaluate object hallucination in a more stable and flexible way.

Evaluating Object Hallucination in Large Vision-Language Models
Yifan Li Yifan Du Kun Zhou"; Jinpeng Wang',
Wayne Xin Zhao and Ji-Rong Wen
'Gaoling School of Artificial Intelligence, Renmin University of China School of Information, Renmin University of China.
BBiijing Key Laboratory of Big Data Management and Analysis Methods "Meituan Group
(liyifan0925, yifandu1999, (armmnnlly@@maall com,
beancis_kunzzoore 63. com, wangjinpeng@ tuan. com, j jrwendruc. edu.. cn
Abstract
2023a) have been proposed enhance the vision- language pre-trained model (VLPM) (Gan et al., 2022) by incorporating powerful LLMs (Touvron et al., 2023; Chiang et al., 2023), which are called large vision-language model (LVLM). Typically, ex- isting work reuses the visual encoder in VLPMs to handle image data, while replacing the original lan- guage encoder with LLMs. After vision-language pre-training (Alayrac al., 2022: Li et al., 2022b) and visual instruction tuning (Liu al., 2023), LVLMs can fulfill complex tasks according to hu- man instructions, demonstrating strong capacities in solving various vision-language tasks, e-g,, im- age captioning (Ordonez et al., 201 I; Hodosh et al., 2015; Sharma et al., 2018; Agrawal et al., 2019) and visual question answering (Antol al., 2015a; Zhang et al., 2016; Goyal al., 2017).
Inspired by the superior language abilities of large language models (LLM), large vision- language models (LVLM) have been recently proposed by integrating powerful LLMs for improving the performance on complex multi- modal tasks. Despite promising progress on LVLMs, find that they suffer from ob- ject hallucinations, Le., they tend to generate objects inconsistent with the target images in the descriptions. To investigate it, this work presents the first systematic study on object hallucination of Ls. We conduct the eval- uation experiments on several representative LVLMs, and show that they mostly suffer from severe object ballucination issues. We further discuss that the visual instructions may influ- ence the ballucination, and find that: objects that frequently appear in the visual instructions or co-occur with the image objects are obvi- ously prone be hallucinated by LVLMs. Be- sides, we further design polling-based query method called POPE for better evaluation of object hallucination. Experiment results show that our POPE can evaluate object hallucination a more stable flexible way.
Despite the success of LVLMs, previous work has revealed that their main components, i.e., LLMs and VLPMs, both suffer from hallucination. Especially, LLMs tend to hallucinate unintended text (Huang et al., 2021; Bang et al., 2023), and VLPMs might generate nonexistent objects in the image (Biten et al., 2022) (termed as object hal- lucination). It is generally believed that the hal- lucination would degrade the model performance and greatly harm the user experiences in real-world applications (MacLeod et al., 2017; Ji et al., 2022). Therefore, it is natural to ask the question: does hallucination still exist in LVLMs? In this paper, we systematically evaluate the issue of object halluci- nation in existing LVLMs, which refers to generat- ing contents that are inconsistent with ground-truth objects in the given image.
1 Introduction
Large language models (LLMs) (Zhao al., 2023) have shown remarkable abilities to solve various complex tasks by following human instructions in a zero-shot manner. The success of LLMs drives the researchers to devise more powerful multi- modal models based on the superior capacity of LLMs, enhance the understanding of visual se- mantics (Alayrac al., 2022; Li et al., 2023b), As an exemplified work, GPT-4 (OpenAI, 2023) has exhibited the exciting performance of L.Ms on multimodal tasks and scenarios.
To conduct our study, we first use the CHAIR (Caption Hallucination Assessment with Image Rel- evance) metric (Rohrbach et al., 2018), and ex- amine the hallucination degree of several repre- sentative LVLMs on the MSCOCO dataset. Our preliminary experiments (Table 1) show that most of LVLMs severely suffer from object hallucina-
Following this line of research, a surge of stud- ies (Zhu et al., 2023; Gao et al., 2023; Li et al.,
"Equal contribution. *Corresponding author.

---

Instruction-based evaluation
POPE
Rondoue vottings Is there bottle the image?!
Provide detailed description of image.
The image festures tablee with variety food displayed are two bowls food, one containing mit of vegetables, such broccoll and carrots, and the other containing The bowt vegetables placed to the front, while the meat bowl is sinuted bahind In addiiion the main disbes,
image.
There
Popular anitingy
there linifee the image!
knite image.
apple placed cable, adding a touch of frust the meal. A bottle can also be seen the table. possibly containing contliment The table neatly arranged, showcasing the difforent food items in an appetizing
Advennrinn vertings
beverage
there pear the image?
Yes, there is pear in the image.
Cases of object hallucination in LVLMs. Bold objects are ground- ground-truth objects in the annotations and red objects are hallucinated objects by LVLMs. The left case is from the traditional instruction-basee evaluation method, and the right cases are from three variants POPE.
Figure 1:
tion, and are even more prone to hallucinate than small vision-language models. Besides, we find that the existing object hallucination evaluation method may not be best suited for LVLMs and further propose a Polling-based Object Probing Evaluation (POPE) method. The basic idea is to convert the evaluation of hallucination into a bi- nary classification task by prompting LVLMs with simple Yes-or-No short questions about the prob- ing objects (e.g., Is there car in the image?). We show that such method is more stable and flexible. Besides, by using different object sampling strate- gies, we validate that existing LVLMs are prone to hallucinate objects which frequently appear or co-occur in the visual instruction dataset.
fer to the enhanced VLPMs with the integration of LLMs as Large Vision-Language Models (LVLM). Generally speaking, an LVLM consists of a vi- sion encoder, language encoder (i.e., an LLM),, and a cross-modal alignment network. The training of LVLMs is generally composed of three major steps. First, a vision encoder and a language en- coder are pre-trained on large-scale unimodal data (i.e., image and text data, respectively). Second, these two encoders aligned through image-text alignment pre-training, which enables the LLM to generate meaningful caption for given im- age. Third, the aligned model is further fine-tuned on image-text instructions, so that it can generate satisfactory answers w.r.t. to a natural language question regarding specific image. Note that in the second and third steps, we can optionally fine- tune different components instead of performing parameeerrter fine-tuning.
Our main contributions are follows: (1) We conduct an empirical study on object hallucination for several representative LVLMMs and find that they are highly affected by object hallucination. (2) We discuss the potential reasons behind this promblem, e.g., LVLMs tend to generate frequently appearing or co-occurring objects in the instruction corpora. (3) We propose an object hallucination evaluation approach called POPE, which is more stable and can be easily extended to unannotated datasets.
Once the visual encoder and the LLM are well aligned, the derived LVLM can demonstrate a su- perior visual understanding ability. It can not only grasp the visual semantics of objects in the image, but also deeply understand the linguistic seman- tics for these objects by leveraging the parametric knowledge in the LLM. Further, the LVLM per- form complex reasoning over the related concepts about these objects, thus achieving an improved performance on a variety of multimodal tasks, egg. visual question answering (VQA).
2 Background
2.1 Large Vision-Language Model
Since LLMs have been shown to be general task solvers in zero-shot/few-shot manner, number of studies devoted improving VLPM by inte- grating powerful I.LMs for more accurate language understanding and generation (Zhu et al., 2023; Liu et al., 2023; Dai et al., 2023a). In this paper, we re-
2.2 Object Hallucination
Although LVLMS are powerful in solving vision- language tasks, they also suffer from the issue of object hallucination as VLPMs. In the literature

---

of computer vision field (Rohrbach al., 2018; Biten al., 2022), object hallucination refers that the model generating descriptions or captions that contain objects which are inconsistent with or even absent from the target image. In general, object hallucination can be defined at different seman- tic levels. The most straightforward way is to define it over the object level, while more fine- grained definitions might be concerned with the attributes or characteristics of objects. In this work, we focus on searse-grained object hallucinations in the model-generated captions and leave fine- grained object hallucinations such as the number, attributes, and positions of the object for future work. We present an example of object halluci- nation in Figure 1, where the hallucinated object "meat "botttee" "beverage", "condiment" are generated by the underlying LVLMS.
Model
CHAIR,
CHAIRs
Len
OSCARBus
7.1
13.0
VinVeLaage
5.5
10.5
OFArage
4.7
8.9
BLIPPooge
4.7
8.8
mPLUG-Owl
14.8
25.4
35.8
LLLVVA
10.5
32.7
64.3
4
MultiModal-GPT
11.1
15.0
11.6
MiniGPT-4
6.7
9.5
24.7
Instruc(BLIP
2.6
3.7
8.5
mPLUG-OWl
30.2
76.8
98.5
LLaVA
18.8
62.7
90.7
12 MultiModal-GPT
18.2
36.2
45.7
MiniGPT-4
9.2
31.5
116.2
InstructBLIP
2.5
3.4
7.5
Table 1: Results of CHAIR VIPMS and LVLMs. Ii denotes "Generate short caption of the image" and Iz denotes "Provide a brief description of the given image". Len refers to the average length of generated captions. The results of VLPMs (OSCAR, VinVL, BLIP, and OFA) are collected from Dai al. (2023b). The best results in each block are denoted in bold.
The hallucination phenomenon hinders the safe use of LVLMs in real-world deployment, as it may result in unexpected consequences caused by these hallucinated objects (Macleok et al., 2017). For example, due to an incorrect understanding of the external environment, autonomous driving sys- tem would make wrong decisions when encounter- ing unexpected events, which might lead to serious safety issues. In order to mitigate these issues, this work aims to study how object hallucination exists in LVLMs from an evaluation perspective.
We select five recently released LVLMs, i.e., mPLUG-Ow/ (Ye et al., 2023), LLaVA (Liu et al., 2023), Multimdal-GPPT (Gong et al., 2023), MiniGPT-4 (Zhu et al., 2023) and Instruct- BLIP (Dai et al., 2023a) and prompt them with following instructions to generate captions about images in MSCOCO (Lin et al., 2014):
I: Generate short caption of the image.
3 Object Hallucination in LVLMs
I:: Provide a brief description of the given
image.
this section, we evaluate the object hallucina- tion problem in popular LVLMs using an existing method. We first introduce the evaluation settings and then analyze the experimental results.
Then, calculate CHAIR these captions. We leave more details about the introduction to the datasct and evaluated models in Appendix A.
3.2 Evaluation Results
3.1 Evaluation Settings
Caption Hallucination Assessment with Image Rel- evance (CHAIR) (Rohrbach et al., 2018) is pop- ular metric for evaluating object hallucination in image captioning tasks. Given the ground truth objects in the image, CHAIR calculates the propor- tion of objects that appear in the caption but not the image. Existing work commonly adopts its two variants, i.e.. CHAIR, and CHAIRs, which evalu- ate the hallucination degree at the object instance level and sentence level respectively. They can be formulated as:
Severity of Hallucinations. As the evaluation re- sults illustrated in Table 1, most instruction- tuned LVLMs suffer from the object hallucination prob- lem, even more serious than small models, e,g., LLaVA (32.7) v.s. OSCARMas (13.0) on CHAIR using Instruction J. indicates that object hallu- cination is an important problem for LVLMs and deserves to be concerned about. As comparison, InstructBLIIP hallucinates less than other LVLMMs. A possible reason is that its visual instructions are collected from wide variety of publicly avail- able datasets, which are relatively short. In con- (I) trast, other LVL.Ms mostly employ the visual in- structions generated by unimodal LLMs (Liu et al., 2023). Such synthetic visual instructions are gener-
[hallucinated objects]
CHAIR
[all mentioned objects)
ccaptions with hallucinated objects]]
CHAIRs
[all captions)|

---

MiniGPT-4
Hallucination times
600
LLaVA
500
MultiModal-GPT
400
mPLUG-OwI
300
200
100
person dining table chair
car
baok
bottle
cup
cat
horse
toilet
(a) Hallucination times of top ten frequently appearing objects, whose frequencies decrease from right to left.
MiniGPT-4
60
Hallucination times
LLavA
50
MultiModal-GPT
40
mPLUG-OWl
30
20
10
person
chair
cup
bowl
bottle
laptop
knife
book
fork
vase
(b) Hallucination times of top ten objects co-occuriing with "dining table", whose frequencies decrease from right to left.
Figure Hallucination times of frequently appearing/co-occuriigg objects MSCOCO.
ally longer and more informative, but may involve unexpected descriptive information (hallucination inherent from LLMs) that is inconsistent with the image, which could mislead LVLMs.
Influence of Instruction Data on Object Hallucination
Considering their impressive performance com- plex vision-language tasks (Chen et al., 2023; Bai al., 2023; Li et al., 2023a), is counter-intuitive that the hallucination problem of LVLMs is so severe. Since smaller VLPMs suffer less from object hallucination, is possible that the visual eustruction-tuming process of LVLMs exacerbates object hallucination. In this section, we investigate the influence of the visual instruction data. We first make two basic hypotheses in Section 4.1 and then conduct qualitative and quantitative analysis to verify them in Section 4.2 and Section 4.3.
Disadvantages of CHAIR. As Table shows, the evaluation results be affected other fac- tors, egg. instruction designs and the length of cap- tions. Specifically, although the adopted two in- structions have similar semantic meanings, LVLMs prompted by Instruction can even result dou- bled values of CHAIR metrics compared with those prompted by Instruction 1, and the performance order of some LVLMs also changes (e.g,, CHAIR, values of LLaVA and MultiModal-GPT), It indi- cates the instability of the CHAIR metric when different instructions are employed. Besides, as CHAIR requires to examine whether the mentioned objects are hallucinated in the generated caption, needs complex human-crafted parsing rules to per- form exact matching, which has not been adapted to the special generation styles of LVLMs and may lead to misclassification errors.
4.1 Hypotheses
As the visual instruction datasets of these LVLMs are mostly constructed based on MSCOCO (Lin et al., 2014), they generally share a similar unbal- anced object distribution where top frequent ob- jects occupy major part of the dataset. After be- ing fine-tuned them. LVLMs may also be prone to generate (or hallucinate) frequently appearing objects MSCOCO. Additionally, the presence of frequently co-occurring object groups (e.g., lap- top, mouse and keyboard) may also contribute to
Thus, necessary to consider a more suitable method that can stably and conveniently evaluate the object hallucination problem in LVLMs.

---

ARA
Hdiing table)
Model
@ 10
@20
@30
@ 10
@20
@30
mPLUG-OWl
0.5455
0.6591
0.7533
0.6608
0.7926
0.8253
LLaVA
0,4620
0.5911
0.6796
0.5628
0.7329
0.8595
MultiModal-GPT
0.4152
0.5399
0.6743
0.5742
0.7849
0.8961
MiniGPT-4
0.4610
0.5758
0.7207
0.5600
0.6980
0.9145
Table 2:
Results on MSCOCO that quantify the correlations hetween the appearing/co-occuring frequency of objects and the hallucination times of LVLMs.
object hallucination. LVLMs can be elicited by the existing objects in the image to hallucinate other objects that frequently co-occur with them. There- fore, we hypothesize that (1) LVLMs prone to hallucinate frequently appearing objects in the vi- sual instruction datasets: (2) LVLMs are prone to hallucinate objects that frequently co-occur with ground objects in the image. We conduct qual- itative and quantitative analyses in the following parts to verify them.
HRo@k(o) 1 Hit@k(i, Hallucinated(
(4)
where HR and HRC quantify the correlations between hallucination times and appearing and co-occurrigg frequency respectively. n is the to- tal number of images, Hallucinated(i) denotes the number of hallucinated objects in the i-th example, Hit@ I(i) denotes the number of top-k frequently appearing MSCOCO objects in Hallucinated (i). and Hit ((i, o) denotes the number of top-k fre- quently co-occurring objects with the probing ob- ject in Hallucinated( Therefore, HR @ k can reflect the proportion of top-k frequently appearing or co-occurring objects in all hallucinated objects. We present the aR, and HRo(dining table) of top 30 objects in Table and leave HRc @ (chair) and HRo@(car) in Appendix C. The HR, @ 10 and HR 10(dining table) all LVLMs are near 0.5 and 0.6, respectively. It indicates that, on average, approximately half of the hallucinated objects in each image belong to the top 10 frequently appearing COCO objects, while more than half among the top 10 frequently occurring objects with the objects already present in the image. When we broaden our observation to the top 30 objects, this proportion continues to increase. These findings further verify that LVLMs mostly hallucinate common objects in the visual instruction data and inspire to design three sam- pling strategies in our evaluation pipeline.
4.2 Qualitative Analysis
We first qualitatively analyze the correlation be- tween the appearance frequency and hallucination. For the first hypothesis, we plot a bar chart be- tween the top ten frequently appearing objects in MSCOCO and their hallucination times in the vali- dation set of MSCOCO; for the second hypothesis, we select the top ten frequently co-occurring ob- jects with "dining table" and also plot a bar chart to show their hallucination times across images that really contain "dining table". We show the re- sults of MiniGPT-4, LLaVA, MultiModal-CG GPP and mPLUG-OWl in Figure 2. Obviously, with the decreasing of the occurrence frequency of objects (from right to left), there is notable decrease in the hallucination times for all four LVLMs. It reveals that the frequently appearing and co-occurring ob- jects in the visual instruction dataset are indeed more likely to be hallucinated by LVLMs. To bet- ter support our results, we also list the full statistics of all 80 COCO objects in Appendix B.
5 POPE
4.3 Quantitative Analysis
To further consolidate the above findings, we em- ploy the top-k hit ratio (HR@k) k) to measure the consistency between the appearance frequency and hallucination times of objects, which is defined as:
In this section, we devise Polling-hased Object Probing Evaluation (POPE), simple yet effective approach for evaluating hallucination in LVLMs. We first provide an overview of POPE, and then evaluate the representative L.L.Ms with POPE. Fi- nally, we discuss the stability and scalability of our method, and also analyze the impact of hallucina-
HR @k 1
Hit@k(i) Hallucinated(3)

---

Ground-truth objects
Nonexistent objects
Human annolation
Random: dog, apple,
person, chair, umbrella, sand, sea,
Negative
Popular: table, knife, ...
Automatic annotation
sampling
Adversarial: surtboard,
Polling questions
Q: Is there person in the imaaee?
Q: Is there dog in the image?
A: Yes.
A: No.
SEEM
Segmentution
Q: Is there chair in the image?
Q: Is there table in the image?
Yes.
A: No.
Q: Is there an umbrella in the image?
Q: Is there surfboard in the image?
Yes.
N:No.
Figure 3: Overview of the POPE pipeline. Given an input image, POPE first extracts ground-trui truth objects in the image either from human annotations or with the help of automatic segmentation tools like SEEM. Then, POPE conducts negative sampling for nonexistent objects in the image under Sandond/PoullrlAddvee Finally, the ground touth objects and nonexistent objects are formulated into question templates to poll LVLMs.
settings.
tion on VQA task.
where denotes the image, g(o;) is the question template "Is there alan <object> in the image?", og is the i-th object to be probed, aj is the answer to the question ("Yes" or "No") and denotes the number of polling questions per image. o can be obtained either from annota- tions or the results of automatic segmentation tools like SEEM (Zou et al., 2023). We set the ratio be- tween ground-truth and nonexistent objects as 1:1 for label balance. After constructing the evaluation triples,
probing or based
5.1 Overview of POPE
In the empirical results of Section 3, we have re- vealed the severity of the object hallucination prob- lem in LVLMs and highlighted the limitations of the existing evaluation method, e.g., sensitive to instructions and biased to short captions. Besides, existing methods mostly rely on parsing the gener- ated captions to extract the predicted objects, which usually require human-crafted complex rules and are still inevitable to omit or misclassify objects. Therefore, we consider devising a more suitable method for the stable, fair and flexible object hal- lucination evaluation of LVLMs, namely polling- based object probing evaluation (POPE). Specif- ically, POPE formulates the evaluation of object hallucination as a binary classification task that prompts LVLMs to output "Yes" or "No", e.g., "Is there a chair in the image?". In this way, by sam- pling objects that LVLMs are prone to hallucinate, we can construct set of hard questions to poll LVLMs. As standard answers these questions are just "Yes" or "No", can easily identify them without complex parsing rules, and avoid the influ- ence of instruction designs and caption length, thus guaranteeing stability, fairness and flexibility.
directly poll LVLMs with them and collect the predicted answers.
The whole POPE pipeline is presented in Figure 3. After obtaining objects in the image, we can start to building polling questions. Ques- tions whose answers are "Yes" can be directly built using ground-trun objects, while questions with the answer "No" can be built by sampling from negative objects. Therefore, by devising differ- ent sampling strategies, we can validate whether LVLMs are prone to hallucinate specific objects. e.g., frequently appearing or co-occurring objects discussed in Section 4. Thus, we devise the follow- ing three sampling strategies:
Pipeline.
Random Sampling: we randomly sample the objects that do not exist in the image.
Definition. Given an image caption dataset, POPE focuses on constructing a set of triples, each of which consists of an image, multiple questions and their answers ("Yes" or "No"). The formulated definition of a triple can be described as:
Popular Sampling: we select the top-k most frequent objects in the whole image dastaset that do not exist in the current image, where k = Z.
Adversarial Sampling: we first rank all ob- jects according to their co-occurring frequencies with the ground-truth objects, and then select the top-k frequent ones that do not exist in the image.
{x, (g(o:), az Nmm))

---

Dataset
POPE
Model
Accuracy
Precision
Recall
Fl Score
Yes (%)
mPLUG-Owl
53,30
51.71
99.53
68.06
96.23
LLavA
54.43
52.32
99.80
68.65
95.37
Random
MultiModal-GP1
50.03
50.02
100.00
66.68
99.97
MiniGPT4
77.83
75.38
82.67
78.86
54.83
InstructBLIP
88.73
85.08
93.93
89.29
55.20
mPLUG-OWl
50.63
50.32
99.27
66.79
98.63
LLaVA
52.43
51.25
99.80
67.72
97.37
MSCOCO
Popular
MultiModal-GPT
50.00
50.00
100.00
66.67
100.00
MiniGPT-4
68.30
64.27
82.40
72.21
64.10
InstructBLIP
81.37
75.07
93.93
83.45
62.57
mPLUG-OWl
50.67
50.34
99.33
66.82
98.67
LLaVA
50.77
50.39
99.87
66.98
99.10
Adversarial
MultiModal-GPT
50.00
50.00
100.00
66.67
100.00
MiniGPT-4
66.60
62.45
83.27
71.37
66.67
InstructBLIP
74.37
67.67
93.33
78.45
68.97
Table Results LVLMs under three evaluation settings of POPE on the validation set of MSCOCO. Yes denotes the proportion of answering "Yes" the given question. The best results in each block are denoted in bold.
Under the above three settings, we can build the evaluation questions of different difficulty levels. We evaluate previously mentioned LVLMs on them with the following metrics.
mPLUG-Owl are extremely prone to answer "Yes" (near 99%). It reveals that these three LVLMs are over confident, leading to lower accuracy on ques- tions with the answer "No". Furthermore, the per- formance of LVLMs consistently decreases, from random settings, popular and adversarial. It is consistent with our findings in Section as LVLMs are prone hallucinate the frequently appearing and co-occurring objects.
Metrics. We adopt Accuracy, Precision, Recall and Fl score as the evaluation metrics. Accuracy reflects the proportion of correctly answered ques- tions. Precision and Recall reflect the ratios of correctly answering questions whose answers are "Yes" or "No", respectively. Fl score combines the results of Precision and Recall and we select it the major metric for evaluation. Besides, also report the ratio that LVLMs answer "Yes" reference analyze the model behaviors.
5.3 Advantages of POPE
As previously stated, the current approach for eval- uating object hallucination LVLMs like CHAIR instruction- based, which is hindered by LVLMs' sensitivity to prompts and requires object annota- tions and manually designed rules for evaluation. In contrast, POPE is more stable to prompt forms and can be casily extended unannotated datasets. Its probing result is also highly consistent with model's caption.
5.2 Evaluation MSCOCO
We evaluate all the LVLMs with POPE built on the validation set of MSCOCO (Lin et al., 2014). We randomly select 500 images with more than 3 ground-truth objects in the annotations and con- struct 6 questions for each image (i.e., I = 6).
Stability. Regardless of the variations in prompt templates, POPE requires LVLMs to answer sim- ple closed-ended questions, which is less likely to introduce ambiguity compared instruction-bar based methods. Such characteristic contributes its sta- bility, To validate it, we evaluate LLaVA using both POPE and CHAIR, with four different prompts for cach. The evaluation results are presented Ta- ble 4. can be observed that the standard deviation
The results are presented in Table 3, where we can obtain a similar conclusion as in Table that InstructBLIP performs the best, while LLaVA, MultiModal-GPT and mPLUG-OWl suffer more se- vere hallucination problem, whose Fl Score are be- low 70. indicates that POPE can well estimate the degree of the hallucination problem in LVLMs. Be- sides, we find that LLaVA, MultiModal-GPT and

---

POPE
CHAIR
Prompt
F1 Score
Prompt
CHAIR;
Is there <ubject> in the image?
68,65
Generate short caption of the image.
10.50
Does the image contain a <object>?
66.83
Provide a brief description of the image.
18.80
Have noticed <object> in the image?
66.67
Generate concise description for the image.
14.60
Can you see <object> the image?
67.58
Create short textual summary for the image.
11.60
Avg+Std.
67.43-00.78
13.88=3.22
Table 4: Evaluation results of LLaVA POPE and CHAIR with different prompt templates.
Dataset
POPE
Model
Accuracy
Precision
Recall
Fl Score
Fl Score (Truth)
Yes (%)
LLaVA
50.47
50.24
99.67
66.80
68.65
99.20
Random
MiniGPT-4
73.77
79.25
64.40
71.06
78.86
40.63
InstructBLIP
86.60
80.74
96.13
89.29
89.27
59.53
LLaVA
50.00
50.00
99.27
66.50
67.72
99.27
MSCOCO
Popular
MiniGPT-4
67.80
68.80
65.13
66.92
72.21
47.33
InstructB1IIP
71.27
64.20
96.13
76.99
83.45
74.87
LLaVA
99.20
66.38
66.98
99.43
Adversarial
MiniGPT-4
61.46
64.00
62.70
71.37
InstructBIII
62.53
57.50
96.13
71.96
78.45
Table 5: SEEM-based POPE results of LVLM on MSCOCO. Fl Score (Truth) are the results of POPE using ground-truth annotations, which are copied from Table 3. The best results in each block are denoted in bold.
of the Fl score is significantly lower than CHAIR,, which confirms that POPE exhibits higher stability when faced with different prompts.
tions on them.
Consistency. A potential concern for POPE is whether the Yes/No responses of LVLMs genuinely reflect their perception of objects. To validate this, measure the consistency between the POPE re- sponses and captions generated by LVLMs. Specif- ically, we examine objects that receive "No" responses seldom appear the captions, and if objects frequently mentioned in captions usually receive "Yes" answers. We collect data from In- structBLIP and MiniGPT-4, given their relatively balanced yes/no distributions. Our findings reveal that out of the 1303 and 1445 objects that are given "No" responses by InstructBLIP and MiniGPT-4, merely 0 and of those objects were referenced in captions. Moreover, out of the 664 and 1034 objects mentioned in the captions by these models, 664 and 961 respectively received "Yes" verdict. Such results underscore robust correlation be- tween objects' presence in captions and Yes/No responses in POPE questions about them, validat- ing the reliability of the POPE assessment.
Scalability. As mentioned before, with the as- sistance of automatic segmentation tools, POPE can be easily extended to datasets without annota- tions. To validate it, adopt SEEM (Zou al., 2023) to annotate images from three datasets (i.e., MSCOCO, A-OKVQA (Schwenk al., 2022) and GQA (Hudson and Manning, 2019)) and build POPE based on the segmentation results. We evalu- ate InstructBLIP, MiniGPT-4 and LLa VA on them and report the results in Table 5 and Table 11 (pre- sented in Appendix D). In Table 5, the perfor- mances of all LVLMs mostly follow the same trend annotation-basee POPE in Table 3, i.e., Ran- dom Popular > Adversarial, and InstructBLIP MiniGPT-4 > LLaVA. Such consistency indicates the reliability of the SEEM-based POPE. Whereas, we also notice the performance gap between the two settings, e.g., FI Score 71.37 v.s. 62.70 for MiniGPT-4 under the Adversarial setting. This phenomenon can be attributed the finer gran- ularity of the segmentation results generated by SEEM, which makes the POPE more challenging. summary, when combined with automated seg- mentation tools, POPE can be easily extended to unannotated datasets and conduct effective evalua-
5,4 Impact of Hallucination on Vision Tasks
Although existing LVLMs do suffer from signifi- cant object hallucination issues, it remains an open question whether these hallucinations have strong impact on other vision tasks. Therefore, we com- pare their performance on POPE with VQA and

---

Dataset
Model
POPET
VQAt
text of LVLMs, thus leading less reliable evalua- tion results. To address this issue, we proposed a polling-based query method called POPE, to pro- vide an improved evaluation approach for the object hallucination of LVLMs. Experimental results have shown that our proposed POPE can better evaluate the object hallucination issue of LVLMs.
InstructBLIP
87.20
59.68
A-OKVQA
MiniGPT-4
72.47
38.69
LLaVA
66.64
50.51
InStructBLIP
85.32
62.12
GQA
MiniGPT-4
67.13
42.24
I.LaVA
66.56
47.60
Table 6: Evaluation results of LVLMs on POPE and VQA. For VQA tasks, we report the VQA score on A- OKVQA and Accuracy on GQA. For POPE, we copy the result under the random setting from Table 11.
Limitations
Despite that we have made extensive explorations. this work still has several limitations. First, we only focus on the object hallucination problem in LVLMs, while do not consider other aspects that can reflect the capacities of LVLMs. It means that the current evaluation task cannot measure the over- all performance of LVLMs. In other words, if some model got a higher score in our evaluation sctting, it does not necessarily indicate a stronger overall capacity than the one with a lower score. Second, due to the limitation of computation re- sources, we have to evaluate all models on a part of the validation set for each dataset. The reported results might be affected by the corresponding data distribution, though have carefully set up the experiments. Third, our proposed POPE utilizes a matching-based method to determine whether LVLMs answer "Yes" or "No", while empirically, LVLMS may occasionally fail to provide answers explicitly containing these words, which may lead to inaccurate evaluation results. Fourth, when com- bined with the automatic segmentation tool, the objects would be annotated based the label set by the tool, which may be inconsistent with the collected human annotations, leading to diver- gence evaluation results. Finally, this work has only compared a small number of LVLMs, without including some recently released or closed-source ones. We leave the evaluation of more LVLMs as our future work.
image captioning tasks. For VQA tasks, we eval- uate the SEEM-based POPE and VQA scores of LVLMs on A-OKVQA and GQA datasets. Since LVLMs are prone to generate answers in an open- ended manner, we utilize ChatGPT to help parse the generated results to better evaluate the VQA performance. The details of evaluation settings are presented in Appendix E. For image caption- ing tasks, we evaluate the captions of 500 images in POPE with traditional metrics. The evaluation results are left in Appendix F.
The evaluation results are shown in Table 6. In- structBLIP performs the best under all settings, highlighting the importance of instruction-tuning on large visual instruction corpora. Note that since InstructBLIP has been trained on A-OKVQA, the result should be considered with caution. Further- more, despite MiniGPT-4 achieving higher F1 score compared to LLaVA, its performance on VQA tasks is relatively poor. A possible reason is that the instruction dataset of MiniGPT-4 only derives from image caption data, while LLaVA uses 158K visual instructions data involving complex visual questions. The results imply that the degree of hallucination may not be always consistent with the VQA performance and these two evaluation as- pects are both important and should be considered in weal-world applications.
6 Conclusion
Although we have extensively discussed the hal- lucination issues of LVLMs, it does not indicate that we hold an negative opinion on their progress. Instead, it will be a very promising direction to develop LVLMs by leveraging the powerful LLMs. These models that were evaluated in this work have been excellent demonstrations for this direc- tion. While, we do hope that our work can bring new ideas or insights to develop more reliable and numan-alignen LVLMs.
In this work, we conducted evaluation experiments on several LVLMs and examined how they suffer from the object hallucination issue. By investigat- ing the reasons for object hallucination, empir- ically revealed that the object distributions of the visual instructions would affect the object halluci- nation of L.LMMs. Besides, we also found that the existing hallucination evaluation methods might be affected by the input instructions and the generated

---

Acknowledgements
Computer Vision, WACV 2022, Waikoloa, HI, USA, January 3-8, 2022, pages 1472-2482. LEEE
This work was partially supported by National Nat- ural Science Foundation of China under Grant No. 62222215, Beijing Natural Science Foundation un- der Grant No. L233008 and 4222027, and Beijing Outstanding Young Scientist Program under Grant No. BJJWZYJHO1 101H1120191000 2019100020098. This research was also supported by Meituan.
Keqin Chen, Zhao Zhang. Weili Zeng. Richong Zhang, Feng Zhu, and Rui Zhao. 2023. Shikra: Unleashing multimodal Ilm's 's referential dialogue magic. CoRR, abs/2306.15199 15195.
Wei-L.in Chiang, Zhuohan Li, 7i Lin, Ying Sheng, Zhanghao Wu, Hao Zhang, Lianmin Zheng, Siyuan Zhuang. Yonghao Zhuang, Joseph E. Gonzalez, lon Stoica, and Eric P. Xing. 2023. Vicuna: An open- source chatbot impressing gpt-4 with 90%* chatgpt quality.
References
Harsh Agrawal, Peter Anderson, Karan Desai, Yufei Wang, Xinlei Chen, Rishabh Jain, Mark Johnson, Dhruv Batra, Devi Parikh, and Stefan Lec. 2019.
Wenliang Dai, Junnan Li, Dongxu Li, Anthony Meng Huat Tiong, Junqi Zhao, Weisheng Wang, Boyang Li, Pascale Fung, and Steven Hoi. 2023a. In- structblip: Towards ceneral-purposee vision-language models with instruction tuning. arXiv preprint arXiv:2305.0650
novel object captioning at scale. In 2019 IEEE/CVF International Conference Computer Vision, ICCV 2019, Seoul, Korea (South), October 27
nocaps:
November 2, 2019. pages 8947-8956, IEEE
Wenliang Dai, Zihan Liu, Ziwei Ji, Dan Su, and Pascale Fung. 2023b. Plausible may not be faithful: Probing object hallucination in laiguage pre-training. In Proceedings of the 17th Conference of the Ture- pean Chapter of the Association for Computational Linguistics, EACL 2023, Dubrovnik, Croatia, May 2-6, 2023, pages 2128-22 2140. Association for Com- putational Linguistics.
Banttsttt Alayrac, Jeff Donahue, Pauline Luc, Antoine Miech. lain Barr, Yana Hasson, Karel Lenc, Arthur Mensch, Katherine Millican, Malcolm Reynolds. Roman Ring. Eliza Rutherford, Serkan Cabi, Tengda Han, Zhitao Gong, Sina Samangooei. Marianne Monteiro, Jacob L. Menick, Sebastian Borgeaud, Andy Brock, Aida Nematzadeh, Sahand Sharifzadeh, Mikolaj Binkowski, Ricardo Barreira, Oriol Vinyals, Andrew Zisserman, and Karén Si- monyan. 2022. Flamingo: a visual language model for few-shot learning. In NeurIPS.
7he Gan, Linjie Li, Chunyuan Li, Lijuan Wang. Zicheng Liu, and Jianfeng Gao. 2022. Vision-language pre- training: Basics, recent advances, and future trends. Found. Trends Comput. Graph. Vis., 14(3-4):133- 352.
Stanislaw Antol, Aishwarya Agrawal, Jiasen Lu, Mar- garet Mitchell, Dhruv Batra, C. Lawrence Zitnick, and Devi Parikh. 2015a. VQA: visual question an- swering. In ICCV, pages 2425-2433. IEEE Com- puter Society.
Peng Gao. Jiaming Han, Renrui Zhang, Ziyi Lin, Shijie Geng, Aojun Zhou, Wei Zhang, Pan Lu, Conghui He, Xiangyu Yue, Hongsheng Li, and Yu Qiao. 2023, Llama-adapter V2: parameter-efficient visual instruc- tion model. CoRR, abs/2304,150011
Stanislaw Antol, Aishwarya Agrawal, Jiasen Lu, Mar- garet Mitchell, Dhruv Batra, C. Lawrence Ziticck, and Devi Parikh. 2015b. VQA: visual question an- swering. In 2015 IEEF International Conference on Computer Vision ICCV 2015, Santiago, Chile, December 7-13, 2015, pages puter Society.
Tao Gong. Chengqi Lyu, Shilong Zhang, Yudong Wang. Miao Zheng, Qian Zhao, Kuikun Liu. Wenwei Zhang. Ping Luo, and Kai Chen. 2023. Multimodal-gpt: A vision and language model for dialogue with humans. arXiv preprint 2305.2230.047999
IEEE Com-
Jinze Bai, Shuai Bai, Shusheng Yang, Shijie Wang, Sinan Tan, Peng Wang, Junyang Lin, Chang Zhou, and Jingren Zhou. 2023. Qwen-vl: A frontier large vision-language model with versatile abilities. CoRR, abs/2308.1 1966.
Yash Goyal, Tejas Khot, Douglas Summers Summers-Stay, Dhruv Batra, and Devi Parikh. 2017. Making the V in VQA matter: Elevating the role of image understanding in visual question answering. In 2017 IEEE Conference Computer Vision and Pattern Recognition, CVPR 2017, Honolula, III USA, July 2 21-26, 2017, pages 6325 6334. JEEE Computer Society.
Yejin Bang, Samuel Cahyawijaya, Nayeon Lee, Wen- liang Dai, Dan Su, Bryan Wilie, Holy Lovenia, Ziwei Ji, Tiezheng Yu, Willy Chung, Quyet V. Do, Yan Xu, and Pascale Fung. 2023. A multitask, multilingual, multimodal evaluation of chatgpt on reasoning, hal- lucination, and interactivity. CoRR, abs/2302.04022.
Micah Hodosh, Peter Young, and Julia Hockenmaier. 2015. Framing image description as a ranking task: Data, models and evaluation metries (extended ab- stract). In IJCAI, pages 4 4188-1192. AAAI Press.
Ali Furkan Biten, Lluis Gómez, and Dimosthenis Karatzas. 2022. Let there be a clock on the beach: Reducing object hallucination in image captioning. In IEEE/CVF Winter Conference on Applications of
Yi-Chong Huang, Xia-Chong Feng, Xiao-Cheng Feng, and Bing Qin. 2021. The factual inconsistency prob- lem in abstractive text summarization: A survey. CoRR, a04/2104, 14839.

---

Drew A. Hudson and Christopher D. Manning. 2019. GQA: A new dataset for real-world visual reason- ing and compositional question answering. In IEFF Conference on Computer Vision and Pattern Recogni- lion, CVPR 2019, Long Beach, CA, USA, June 16-20, 2019, pages 6700-6709. Computer Vision Founda- tion /EEE.
of the 2017 CHI Conference on Human Factors in Computing Systems, Denver, CO, USA, May 06-11, 2017, pages 9988-5999. ACM.
OpenAl. 2023. GPT-4 technical report. CoRR. abs/2303.08774.
Vicente Ordonez, Girish Kulkarni, and Tamara L. Berg. 2011. Im2text: Describing images using 1 million captioned photographs. In NIPS, pages 1143-1151.
Ziwei Ji, Nayeon Lee. Rita Frieske, Tiezheng Yu, Dan Su, Yan Xu, Etsuko Ishii, Yejin Bang, Andrea Madotto, and Pascale Fung. 2022. Survey of hal- lucination in natural language generation. CoRR, abs/2202.03629.
Anna Rohrbach, Lisa Anne Hendricks, Kaylee Burns, Trevor Darrell, and Kate Saenko. 2018. Object hallu- cination in image captioning. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, Brussels, Belgium, October 31 November 4, 2018, pages 4035-4045. Association for Computational Linguistics.
Bo Li, Yuanhan Zhang. Liangyu Chen, Jinghao Wang. Jingkang Yang, and Ziwei Liu. 2023a. Otter: multi-modal model with in-context instruction tuning. CoRR, abs/2305.00 003766.
Dongxu Li, Junnan Li, Hung Le, Guangsen Wang, Sil- vio Savarese, and Steven C. H. Hoi. 2022a. LAVIS: A library for language-vision intelligence. CoRR, 0bs//209.09019..
Dustin Schwenk, Apporv Khandelwal, Christopher Clark, Kenneth Marino, and Roozbeh Mottaghi. 2022.
A-OKVQA: A benchmark for visual question answer- ing using world knowledge. In Computer Vision ECCV 2022 17th European Conference, Tel Aviv, Israel, October 23-27, 2022, Proceedings, Part VIII, volume 13668 of Lecture Notes in Computer Science. pages 146-162. Springer.
Junnan Li, Dongxu Li, Silvio Savarese, and Steven C. H. Hoi. 2023b. BLIP-2: bootstrapping language-image pre-training with frozen image encoders and large language models. CoRR, abs/2301.12597.
Piyush Sharma, Nan Ding, Sebastian Goodman, and Radu Soricut. 2018. Conceptual captions: A cleaned, hypernymed, image alt-text dataset for automatic im- age captioning. ACL (1), pages 2556 2565. Asso- ciation for Computational Linguistics.
Jumnan Li, Dongxu Li. Caiming Xiong, and Steven C. H. Hoi. 2022b. BLIP: bootstrapping language-iman pre-training for unified vision-language understand- ing and generation. In International Conference on Machine Learning, ICML 2022, 17-23 July 2022, Bul- timore, Maryland, USA, volume 162 Proceedings of Machine Learning Research, pages 12888-12900. PMLR.
Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix. Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, Aurélien Rodriguez, Armand Joulin, Edouard Grave, and Guillaume Lample. 2023, Llama: Open and efficient foundation language models. CoRR, abs/2302. 13971.
Xiujun Li. Xi Yin, Chunyuan Li, Pengchuan Zhang, Xiaowei Hu, Lei Zhang, Lijuan Wang. Houdong Hu, Li Dong, Furu Wei, Yejin Choi, and Jianfeng Gao. 2020. Oscar: Object-semantics aligned pre- training for vision-language tasks. In Computer Vi- sion ECCV -ion 2020- 16th European Conference, Glas- gow, UK. August 23-28. 2020, Proceedings, Part XXX, volume 12375 of Lecture Notes in Computer Science, pages 121-137. Springer.
Peng Wang, An Yang. Rui Men, Junyang Lin, Shuai Bai, Zhikang Li, Jianxin Ma, Chang Zhou, Jingren Zhou, and Hongxia Yang. 2022. OFA: unifying ar- chitectures, tasks, and modalities through a simple learning framework. In Inter- national Conference on Machine Learning, ICML 2022, 17-23 July 2022, Baltimore, Maryland, USA, volume 162 of Proceedings of Machine Learning Research, pages 1--3400,,, PMIR
Tsung-Yi Yi Lin, Michael Maire, Serge J. Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C. Lawrence Zitnick. 2014. Microsoft COCO: common objects in context. In Computer Vision ECCV 2014- 13th European Conference, Zurich, Switzerland, Seplember 6-12, 2014, Proceedings, Pari V, volume 8693 of Lecture Notes in Computer Science, pages 155--55..00 Springer.
Qinghao Ye, Haiyang Xu, Guohai Xu, Jiabo Ye, Ming Yan, Yiyang Zhou, Junyang Wang, Anwen Hu, Pengcheng Shi, Yaya Shi, Chenliang Li, Yuanhong Xu, Hehong Chen, Junfeng Tian, Qian Qi. Ji Zhang, and Fei Huang. 2023. mplug-owl: Modularization empowers large language models with multimodality. CoRR, abs/2304.14177 141788
Haotian Liu, Chumyuan Li, Qingyang Wu, and Yong Jae Lee. 2023. Visual instruction tuning. CORR, abs/2304.08485.
Haley MacL.eod, Cynthia I.. Bennett, Meredith Ringel Morris, and Edward Cutrell. 2017. Understanding blind people's experiences with computer-generater captions of social media images. In Proceedings
Peng Zhang, Yash Goyal, Douglas Summers-Stay Dhruv Batra, and Devi Parikh. 2016. Yin and yang: Balancing and answering binary visual questions. CVPR, pages 5014-5022. IEEE Computer Society.

---

Pengchuan Zhang, Xiujun Li, Xiaowei Hu, Jianwei Yang, Lei Zhang, Lijuan Wang. Yejin Choi, and Jian- feng Gao. 2021. Vinvl: Revisiting visual representa- tions in vision-language models. In JEEE Conference on Computer Vision and Pattern Recognition, CVPR 2021, viriual, June 19-25, 2021, pages 5379-5588. Computer Vision Foundation IEEE.
A Details of Evaluation Settings
Dataset. MSCOCO (Lin et al., 2014) is a large- scale image recognition, segmentation, and caption- ing dataset. Here, randomly sample 2,000 im- ages with annotations about contained objects and human-labeled captions from its validation set as our evaluation dataset. For computing the CHAIR metric on MSCOCO, we follow the settings Rohrbach et al. (2018) which only considers 80 objects appearing in the MSCOCO segmentation challenge.
Wayne Xin Zhao, Kun Zhou, Junyi Li. Tianyi Tang. Xiaolei Wang. Yupeng Hou, Yingqian Min, Be- ichen Zhang, Junjie Zhang, Zican Dong, Yifan Du, Chen Yang, Yushuo Chen, Zhipeng Chen, Jinhao Jiang, Ruiyang Ren, Yifan Li, Xinyu Tang, Zikang Liu, Peiyu Liu, Jian-Yun Nie, and Ji-Rong Wen. 2023. A survey of large language models. CoRR, Abs/2303.18223.
Models. The evaluated LVLMs basically con- sist of three parts: visual encoder, an alignment model, and a large language model. All the above models have been tuned collected visual in- struction data. A detailed comparison (e.g., back- bones and trainable components) of these LVLMs shown in Table We also collect the evaluation results of smaller VLPMs, i.e., OSCAR (Li et al., 2020), VL (Zhang et al., 2021), BLIP (Li et al.,
Deyao Zhu, Jun Chen, Xiaoqian Shen, Xiang Li, and Mohamed Elhoseiny, 2023. Minigpt-4: Enhancing vision-languee language understanding with advanced large language models. CoRR, abs/2304. 10592.
Xueyan Zou, Jianwei Yang. Hao Zhang, Feng Li, Linjie Li, Jianfeng Gao, and Yong Jae Lee. 2023. Seg- ment everything everywhere once. CoRR, abs/2304.0671s
2022b) and OFA (Wang et al., 2022) from Dai et al. (2023b) as baseline results.
B Additional Qualitative Analysis Results
To better validate our hypotheses, We expand the analysis scope to all 80 objects MSCOCO and present the result in this part.
For hypothesis (1), we present the cumulative proportions of the hallucination times of all 80 COCO objects in Table 8. The table demonstrates that, across all models, the top 30 objects comprise approximately 70% all hallucinated objects. For hypothesis (2), we present the cumulative propor- tions of the hallucination times of all COCO objects that co-occur with dining table in Table 9, We also arrange these objects by their co-occurrence frequency. Similarly, the top 20 objects comprise about 80% of all hallucinated objects.
C Additional Quantitative Analysis Results
We present the HR results two other common objects, i.e., chair and car in Table 10, which show a similar trend with Table 2.
D Results of SEEM-hased POPE on A-OKVQA and GQA
We adopt SEEM (Zou et al., 2023) to annotate im- ages from A-OKVQA and GQA and build POPE

---

Pre-training
Fine-tuning
Model
VE
AN
LLM
VE
AN
LLM
VE
AN LLM
2
$
mPL.G-OOl
VIT-1/14
Attention
A7
LLaVA
VIT-L/14
Linear
13B
d
$
d
MultiModal-GPP
VIT-L/14
Attention
LLaMAyB
MiniGPT-4
VIT-G/14
Linear
Vicuna13B
InstructBLIP
VIT-G/14
Q-Former
Vicuna
Table 7: Comparison of the evaluated LVLMs. VE, AN, LLM stand for Visual Encoder, Alignment Network and
Large Language Model, respectively.
denotes frozen and & denotes trainable. The fine-tuning of LLM in
MuliModaa--PP and mPLUG-Owl implemented by LoRA.
Accumulative proportions (sorted by appearance frequency)
Model
Top
Top
Top 30
Top 40
Top
Top
Top
Top 80
mPLUG-Owl
56.89
67.75
77.34
79.93
90.41
95.33
98.75
100.00
LLaVA
48.47
60.29
69.74
72.80
81.46
90.26
96.69
100.00
Multimodal-Gne -PT
43.07
56.
68.85
72.78
81.59
90.07
97.07
100.00
MiniGPT-4
44.96
55.19
70.68
78.12
84.36
93.23
97.59
100.00
Table The accumulated proportions of hallucination times of 80 COCO objects. arrange objects by
their frequency of occurrence.
Accumulative proportions (sorted co-occurrence frequency)
Model
Top 10
Top 20
Top 30
Top 40
Top 50
TTop 60
mPLUG-Owl
71.78
82.82
86.81
92.94
99.08
100.00
LLaVA
60.69
73.79
85.52
93.10
97.24
100.00
MultimodalGGP
53.50
75.16
85,99
96.82
98.09
100.00
MiniGPT-4
64.00
80.00
92.00
94.00
96.00
100.00
Table 9: The accumulated proportions of the hallucination times all objects that co-occur with dining table, We arrange all objects by their co-occurrence frequency with dining table.
based segmentation results. We evaluate In- structBLIP, MiniGPT-4 and LLaVA. We also eval- uate full-data supervised-tuned smaller model, BLIP (Li et al., 2022b) to better reflect the degree of hallucination. The evaluation results are pre- sented Table 11.
judge whether the student's answer matches of the 10 correct answers. If matches, please output the correct answer directly (must be an element in the list, if it matches multiple correct answers, please output the most frequent occurrence the list); if not, please output <NAN> directly. Do NOT output anything else! correct answers: student answer:"
ChatGPT-assisted VQA Evaluation.
We employ the VQA score (Antol al., 2015b) to assess VQA tasks with the help of ChatGPT. Considering that LVLMs generally produce open- ended responses, enlist ChatGPT assess whether the model's reply aligns with potential answers. The prompt we provide to ChatGPT is as follows:
F Results of Image Captioning
The MSCOCO captioning results of LVLMs are showcased in Table 12. Generally, their captioning performance aligns with the POPE assessments. suggesting that object hallucination influences the efficacy of LVLMs in other vision tasks.
" "You are an examiner who can judge whether a student's 's answer matches the correct answers. Next, will provide you with 10 correct answers in the form of a list and a student's ss answer. Please

---

(ch(ch
(Ra(car)
Model
@11
@00
@30
@e20
@30
mPLUG-Owl
0.5926
0.7186
0.8201
0.7587
0,9136
0.9707
LLaVA
0.5206
0.6830
0.8152
0.6870
0.8886
0.9188
MultMModal-GPT
0.5732
0.7576
0.8811
0,6031
0,8482
0.8623
MiniGPT-4
0.5701
0.7746
0.8581
0.6444
0.8278
0.9417
Table 10: The HRc result of chair and car.
Dataset
POPE
Model
Accuracy
Precision
Recall
FI Score
Yes (%)
BLIP
91.00
92.24
89.53
90.87
48.53
LLaVA
50.16
50.08
99.53
66.64
99.37
Random
MiniGPT-4
74.47
78.63
67.20
72.47
42.73
InstructBLIP
85.77
79.21
97.00
87.20
61.23
BLIP
88.40
87.55
89.53
88.53
51.13
A-OKVQA
Popular
LLaVA
50.03
50.02
99.67
66.61
99.63
MiniGPT-4
69.93
70.40
68.80
69.59
48.87
InstructBLIP
75.03
67.39
97.00
79.53
71.97
BLIP
82.37
78.31
89.53
83.55
57.17
LLaVA
50.13
50.07
99.67
66.65
99.53
Adversarial
MiniGPT-4
64.33
63.62
66.93
65.24
52.60
InstructLIP
65.46
59.48
97.00
73.75
81.53
BLIP
89.93
91.08
88.53
89.79
48.60
LLaVA
50.17
50.08
99.20
66.56
99.03
Random
MiniGPT-4
71.33
78.67
58.53
67.13
37.20
InstructBLIIP
83.90
78.39
93.60
85.32
59.70
BLIP
86.63
85.34
88.47
86.87
51.83
GQA
Popular
LLaVA
50.03
50.02
99.47
66.56
99.43
MiniGPT-4
68.26
72.99
58.00
64.64
39.73
InstructLIP
71.87
65.24
93.60
76.89
71.73
BLIP
82.40
78.89
88.47
83,41
56.07
LLaVA
49.77
49.88
99.20
66.38
99.43
Adversarial
MiniGPT-4
64.23
66.29
57.93
61.83
43.70
InstructBL.IP
64.30
59.02
93.60
72.39
79.30
Table 11: SEEM-based POPE results of LVLMs and BLIP on A-OKVQA and GQA. The probing objects are selected from the segmentation results of SEEM on these datasets. The best results each block except BLIP are denoted in bold. We employ the BLIP fine-tuned VQAv2 from LAVIS (Li et al., 2022a).
Model
POPE
BLEU-1
BLEU-2
METEOR
ROUGE-L
InstructBLIP
89.29
59,5
45.2
22.6
42.3
LLaVA
68.65
22.0
13.9
19.8
22.4
MiniGPT-4
78.86
41.1
28.8
25.6
44.7
Table 12: MSCOCO caption results.


Reviews:
[{"review_id": "6UJm8ChVQF", "reviewer": "Reviewer_tq5W", "summary": "", "questions": "1. The large model is unlikely to only reply with yes/no. How to handle subsequent replies and whether there will be inconsistencies between the previous and the current responses remains to be seen.\n2. Is Table 6 based on a subset or the entire VQA dataset, and does the evaluation for VQA task also use the Yes/No scoring method?", "limitations": "", "rating": 3, "confidence": 5, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for the careful reading and helpful feedback. We respond below to your questions and concerns:\n\n> Is object hallucination the most important problem of multimodal LLM? Others include knowledge, object spatial relationships, fine-grained attributes, etc.\n\nThanks for your insightful comment. Actually, object hallucination is one of the most fundamental issues that degrade the performance of LVLMs. It is because the semantic understanding of an image is often directly related to objects that it contains, those (object spatial relationship, fine-grained attributes) as you mentioned. Furthermore, a number of visual-text tasks are also defined over objects, *e.g.,* VQA and visual reasoning. Evaluating object hallucination essentially reflects whether the model can accurately identify the main objects in an image, which is the basic ability of LVLMs to understand images, providing the foundation for addressing complex tasks. However, as the reviewer suggested, a future direction is to extend the study of object hallucination to a more fine-grained hallucination (e.g., attribute-level hallucination). \n\nFollowing the suggestion of the reviewer, we conduct statistical analyses on finer-grained hallucination issues of LVLMs. Concretely, we select 10 captions from each of the 5 LVLMs and manually annotate their hallucination frequency on object, knowledge, spatial ralationships and fine-grained attributes. The results are presented below.\n\n|                | object | knowledge | spatial ralationships | fine-grained attributes |\n| -------------- | -----: | --------: | --------------------: | ----------------------: |\n| mPLUG-Owl      |     10 |         1 |                     2 |                       7 |\n| LLaVA          |      8 |         0 |                     3 |                       3 |\n| MultiModal-GPT |      6 |         0 |                     0 |                       6 |\n| MiniGPT-4      |      6 |         0 |                     0 |                       3 |\n| Sum            |     30 |         1 |                     5 |                      19 |\n\nThe above statistics show that object hallucination is the most prevalent one, while knowledge and spatial hallucination only occupy a small part. We also notice that LVLMs often make mistakes on fine-grained attributes of objects, such as number, color or textual content. However, it is challenging to enumerate all fine-grained attributes, and their annotation and evaluation are also hard to be efficiently performed. As a comparison, the object halluciation issue is much easier to be evaluated, and their annotations can be acquired from the caption data or through a segmentation model (*e.g.,* SEED).\n\n\n\n> Is it sufficient to measure object hallucination through only yes/no responses? Yes response does not necessarily indicate that the model comprehends the presence of the object in the image, as it may still produce incorrect objects when undertaking other tasks.\n\nWe believe that the yes/no responses sufficiently reflect LVLMs' understanding of the presence of objects in the image. To verify this, we design two extra experiments to measure the consistency between the yes/no responses and captions. We find that objects which get \"No\" responses from models are hardly appear in models' captions, and models prefer to answer \"Yes\" to objects mentioned in captions. \n\n1. Consistency between \"No\" responses and captions\n\n   We first collect objects and corresponding images from POPE_random which get a \"No\" answer from models. Then, we ask models to generate captions on these images and count the occurence of these objects. We collect data from InstructBLIP and MiniGPT-4, since their yes/no proportions are relatively more balanced. The results are presented in the table below.\n\n   | Model        | Number of No responses | Number of objects in captions | Ratio |\n   | :----------- | ---------------------: | ----------------------------: | ----: |\n   | InstructBLIP |                   1303 |                             0 | 0.00% |\n   | MiniGPT-4    |                   1445 |                             5 | 0.35% |\n\n2. Consistency between \"Yes\" responses and captions\n\n   We first collect 500 captions from each model and extract COCO objects from them. Then, we form them into POPE questions using the template \"Is there a \\<object\\> in the image?\". We ask the models about these questions and count the number of \"Yes\" responses. For the same reason, we only test InstructBLIP and MiniGPT-4. The results are presented in the table below.\n\n   | Model        | Number of objects in captions | Number of Yes responses | Ratio |\n   | :----------- | ----------------------------: | ----------------------: | ----: |\n   | InstructBLIP |                           664 |                     664 |  100% |\n   | MiniGPT-4    |                          1034 |                     961 | 92.9% |\n\nThe above experiments show that the yes/no responses and captions of LVLMs are highly consistent. It also indicates the effectiveness of the yes/no responses in POPE to evaluate the object hallucination of LVLMs as existing methods.\n\n\n\n\n\n> The large model is unlikely to only reply with yes/no. How to handle subsequent replies and whether there will be inconsistencies between the previous and the current responses remains to be seen.\n\nWe evaluate the consistency between the yes/no and susequent replies of models with the assistance of ChatGPT. The results show that most answers of LVLMs are consistent.\n\nConcretely, we select 200 answers of POPE questions from each model and ask ChatGPT to judge whether the two part of the answer have consistent semantic. The prompt we provide to ChatGPT is as follows:\n\n- *I will provide you with a dialogue consisting of a question and an answer. The question will be a simple closed-ended question, such as \"Is there a truck in the image?\" The answer will be composed of two parts: the first part will be a single 'Yes' or 'No,' and the second part will provide additional explanation or description for the reasons behind the given answer. Your task is to determine whether the attitude of both parts of the answer is consistent.*\n\n  *Here are some examples to illustrate the task:*\n\n  *Question: Is there a car in the image? Answer: Yes, there is a car in the image. Judgment: Consistent*\n\n  *Question: Is there a boat in the image? Answer: Yes, there is a car in the image. Judgment: Inconsistent*\n\n  *Question: Is there a boat in the image? Answer: No, there is a boat in the image. Judgment: Inconsistent*\n\n  *Please provide your judgment for the following dialogue:*\n\nTo avoid misjudgement, we also manually check the result. The final results are presented below.\n\n| Model            | InstructBLIP |   LLaVA | MiniGPT-4 | Mulitmodal-GPT | mPLUG-Owl |\n| ---------------- | -----------: | ------: | --------: | -------------: | --------: |\n| Consistent ratio |      200/200 | 189/200 |   199/200 |        199/200 |   199/200 |\n\nThe results show that most reply of LVLMs are consistent, so we believe it is reasonable to judge model's attitude with the yes/no part. \n\n\n\n> Is Table 6 based on a subset or the entire VQA dataset, and does the evaluation for VQA task also use the Yes/No scoring method?\n\n1. We use the entire A-OKVQA and a subset of GQA (500 examples).\n\n2. No, we use the VQA score [1] to evaluate VQA tasks with the assistance of ChatGPT. Due to page limits, we have left this part in Appendix C. \n\n   Since the reply of LVLMs are usually open-ended,  conventional evaluation for VQA tasks may not effectively measure the quality of answers. Therefore, we use ChatGPT to judge whether the model's answer matches candidate answers. The prompt we provide to ChatGPT is as follows:\n\n   *You are an examiner who can judge whether a student's answer matches the correct answers. Next, I will provide you with 10 correct answers in the form of a list and a student's answer. Please judge whether the student's answer matches one of the 10 correct answers. If it matches, please output the correct answer directly (must be an element in the list, if it matches multiple correct answers, please output the most frequent occurrence in the list); if not, please output \\<NAN\\> directly. Do NOT output anything else!*\n\n   *correct answers:* \n\n   *student answer:* \n\n   Then, we calculate the VQA score using the original equation $\\text{score}(ans) = \\min(\\frac{\\text{humans that said} \\ ans}{3}, 1)$. \n\n​\t\n\n[1] Antol S, et al. Vqa: Visual question answering[C]//Proceedings of the IEEE international conference on computer vision. 2015: 2425-2433.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. The paper is well written and easy to follow.\n2. The work conducts an empirical study on object hallucination for several representative LVLMs, and are highly affected by object hallucination.\n3. The work proposes an object hallucination evaluation method, which can be easily extended to unannotated datasets.", "reasons_to_reject": "1. Is object hallucination the most important problem of multimodal LLM? Others include knowledge, object spatial relationships, fine-grained attributes, etc.\n2. Is it sufficient to measure object hallucination through only yes/no responses? Yes response does not necessarily indicate that the model comprehends the presence of the object in the image, as it may still produce incorrect objects when undertaking other tasks.", "questions_to_authors": "1. The large model is unlikely to only reply with yes/no. How to handle subsequent replies and whether there will be inconsistencies between the previous and the current responses remains to be seen.\n2. Is Table 6 based on a subset or the entire VQA dataset, and does the evaluation for VQA task also use the Yes/No scoring method?", "paper_topic_and_main_contributions": "The work presents the first systematic study on object hallucination of LVLMs. The authors conduct the evaluation experiments on several representative LVLMs. They also design a polling-based query method called POPE for better evaluation of object hallucination. ", "excitement": 3, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "gS32nqMFQh", "reviewer": "Reviewer_Dmv1", "summary": "", "questions": "- On Figure 2a, given some objects (cat and horse) are rarely hallucinated despite having high frequency, have authors evaluated the trend of the object hallucination for the other objects (let say top-20 or top-30)? Does the trend hold in general? \n- On section 4.2, do author also evaluate the co-occuring on different objects other than dining table? Does it produce a consistent trend for different objects?\n- On section 4.3, what is the probing object used in HR_C@10?\n- On section 4.3, since the top-10 objects only cover 50% of all the hallucinated objects, what about the remaining objects? Does it produce a downward trend depending on the frequency of the objects in MSCOCO?\n- On section 5.4, why don't author compare POPE with other VL tasks such as image captioning or image classification? Is there any specific reason why authors choose VQA? It will be better if authors can also compare with performance on other tasks before making the conclusion on Line 512.", "limitations": "", "rating": 4, "confidence": 4, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for the careful reading and helpful feedback. We respond below to your questions and concerns:\n\n> On Figure 2a, given some objects (cat and horse) are rarely hallucinated despite having high frequency, have authors evaluated the trend of the object hallucination for the other objects (let say top-20 or top-30)? Does the trend hold in general?\n>\n> For section 4.3, it would be better if authors can show the cumulative hallucination count of all objects in MSCOCO sorted by frequency\n\nYes, the same trend holds for other objects as well. Due to space limitations, we have only listed the trend for the top 10 objects in our figure. Here, we present the complete statistics for all 80 COCO objects.. We organize these objects based on their appearance frequency in the COCO validation set and calculate their hallucination times in groups of 10. The results are displayed in the table below.\n\n|                | Top 1-10 | Top 11-20 | Top 21-30 | Top 31-40 | Top 41-50 | Top 51-60 | Top 61-70 | Top 71-80 |  Sum |\n| -------------- | -------: | --------: | --------: | --------: | --------: | --------: | --------: | --------: | ---: |\n| LLaVA          |     1349 |       329 |       263 |        85 |       241 |       245 |       179 |        92 | 2783 |\n| MiniGPT-4      |      598 |       136 |       206 |        99 |        83 |       118 |        58 |        32 | 1330 |\n| Multimodal-GPT |      690 |       209 |       204 |        63 |       141 |       136 |       112 |        47 | 1602 |\n| mPLUG-Owl      |     2860 |       546 |       482 |       130 |       527 |       247 |       172 |        63 | 5027 |\n\nWe also list the cumulative proportions of the hallucinated objects below.\n\n|                | Top 10 | Top 20 | Top 30 | Top 40 | Top 50 | Top 60 | Top 70 | Top 80 |\n| -------------- | -----: | -----: | -----: | -----: | -----: | -----: | -----: | -----: |\n| LLaVA          |  48.47 |  60.29 |  69.74 |  72.80 |  81.46 |  90.26 |  96.69 |    100 |\n| MiniGPT-4      |  44.96 |  55.19 |  70.68 |  78.12 |  84.36 |  93.23 |  97.59 |    100 |\n| Multimodal-GPT |  43.07 |  56.12 |  68.85 |  72.78 |  81.59 |  90.07 |  97.07 |    100 |\n| mPLUG-Owl      |  56.89 |  67.75 |  77.34 |  79.93 |  90.41 |  95.33 |  98.75 |    100 |\n\nFrom the table, we can see that the hallucination times decrease across different groups (except for top 31-40), and the top-10 objects consistently exhibit the highest hallucination times.  Additionally, across all models,  the top-30 objects occupies approximately 70% of the hallucinated objects. Therefore, it is reasonable to conclude that LVLMs tends to hallucinate frequently appearing objects in the instruction corpus.\n\n\n\n> On section 4.2, do author also evaluate the co-occurring on different objects other than dining table? Does it produce a consistent trend for different objects?\n\nYes, the trend hold for other objects too. Due to space limitations, we only list the result of dining table. Here, we provide the full statistics of dining table, chair and car.\n\n| dining table   | Top 1-10 | Top 11-20 | Top 21-30 | Top 31-40 | Top 41-50 | Top 51-60 |  >60 |  Sum |\n| -------------- | -------: | --------: | --------: | --------: | --------: | --------: | ---: | ---: |\n| LLaVA          |       88 |        19 |        17 |        11 |         6 |         4 |    0 |  145 |\n| MiniGPT-4      |       32 |         8 |         6 |         1 |         2 |         2 |    0 |   50 |\n| Multimodal-GPT |       84 |        34 |        17 |        17 |         2 |         3 |    0 |  157 |\n| mPLUG-Owl      |      234 |        36 |        13 |        20 |        20 |         3 |    0 |  326 |\n\n| chair          | Top 1-10 | Top 11-20 | Top 21-30 | Top 31-40 | Top 41-50 | Top 51-60 |  >60 |  Sum |\n| -------------- | -------: | --------: | --------: | --------: | --------: | --------: | ---: | ---: |\n| LLaVA          |       90 |        30 |        20 |        25 |         7 |         3 |    7 |  182 |\n| MiniGPT-4      |       12 |         4 |         0 |         2 |         1 |         0 |    0 |   19 |\n| Multimodal-GPT |       48 |        28 |        11 |         6 |         2 |         0 |    3 |   98 |\n| mPLUG-Owl      |      427 |        65 |        83 |       117 |        11 |         3 |    9 |  715 |\n\n| car            | Top 1-10 | Top 11-20 | Top 21-30 | Top 31-40 | Top 41-50 | Top 51-60 |  >60 |  Sum |\n| -------------- | -------: | --------: | --------: | --------: | --------: | --------: | ---: | ---: |\n| LLaVA          |       90 |        17 |        10 |         7 |         1 |         0 |    2 |  127 |\n| MiniGPT-4      |       18 |         1 |         2 |         1 |         0 |         0 |    1 |   23 |\n| Multimodal-GPT |       19 |         4 |         0 |         0 |         1 |         0 |    0 |   24 |\n| mPLUG-Owl      |      201 |        44 |        20 |         1 |         5 |         2 |    2 |  275 |\n\nFrom the tables above, we can observe that the descending trends remain consistent for all three objects.Therefore, it is reasonable to conclude that LVLMs tends to hallucinate frequently co-occurring objects in the instruction corpus.\n\n\n\n> On section 4.3, what is the probing object used in HR_C@10?\n\nThe probing objects for HR_C@10 is dining table. We will rectify this in the following revisions. \n\n\n\n> On section 4.3, since the top-10 objects only cover 50% of all the hallucinated objects, what about the remaining objects? Does it produce a downward trend depending on the frequency of the objects in MSCOCO?\n\nYes, this downward trend also applies to less frequent objects. To confirm this, we add supplementary experiments for HR_A and HR_C with the top 20 and top 30 objects. Additionally, for HR_C, we've included the results for two more probing objects (*i.e.,* chair and car). The results are displayed in the tables below. The data in parentheses indicate the difference in HR compared to the preceding column, corresponding to the HR within current interval (*e.g.,* top 11-20 or top 21-30).\n\n|                | HR_A@10        | HR_A@20        | HR_A@30        |\n| -------------- | -------------- | -------------- | -------------- |\n| mPLUG-Owl      | 0.5455(0.5455) | 0.6591(0.1136) | 0.7533(0.0942) |\n| LLaVA          | 0.4620(0.4620) | 0.5911(0.1291) | 0.6796(0.0855) |\n| MultiModal-GPT | 0.4152(0.4152) | 0.5399(0.1247) | 0.6743(0.1344) |\n| MiniGPT-4      | 0.4610(0.4610) | 0.5758(0.1148) | 0.7207(0.1449) |\n\n| dining table   | HR_C@10        | HR_C@20        | HR_C@30        |\n| -------------- | -------------- | -------------- | -------------- |\n| mPLUG-Owl      | 0.6608(0.6608) | 0.7926(0.1318) | 0.8253(0.0597) |\n| LLaVA          | 0.5628(0.5628) | 0.7329(0.1701) | 0.8595(0.1266) |\n| MultiModal-GPT | 0.5742(0.5742) | 0.7849(0.2107) | 0.8961(0.1112) |\n| MiniGPT-4      | 0.5600(0.5600) | 0.6980(0.1380) | 0.9145(0.2165) |\n\n| chair          | HR_C@10        | HR_C@20        | HR_C@30        |\n| -------------- | -------------- | -------------- | -------------- |\n| mPLUG-Owl      | 0.5926(0.5926) | 0.7186(0.1260) | 0.8201(0.1015) |\n| LLaVA          | 0.5206(0.5206) | 0.6830(0.1624) | 0.8152(0.1322) |\n| MultiModal-GPT | 0.5732(0.5732) | 0.7576(0.1844) | 0.8811(0.1235) |\n| MiniGPT-4      | 0.5701(0.5701) | 0.7746(0.2045) | 0.8581(0.0835) |\n\n| car            | HR_C@10        | HR_C@20        | HR_C@30        |\n| -------------- | -------------- | -------------- | -------------- |\n| mPLUG-Owl      | 0.7587(0.7587) | 0.9136(0.1549) | 0.9707(0.0571) |\n| LLaVA          | 0.6870(0.6870) | 0.8886(0.2016) | 0.9188(0.0302) |\n| MultiModal-GPT | 0.6031(0.6031) | 0.8482(0.2451) | 0.8623(0.0141) |\n| MiniGPT-4      | 0.6444(0.6444) | 0.8278(0.1834) | 0.9417(0.1139) |\n\nThe above results show that HR_A and HR_C in most interval keep descending along with the co-occurring frequency of objects, and all of them are siginificantly smaller than HR_A/HR_C@10. Hence, it's reasonable to conclude that LVLMs are more susceptible to hallucinating objects that appear frequently or co-occur in the instruction corpus.\n\n\n\n> On section 5.4, why don't author compare POPE with other VL tasks such as image captioning or image classification? Is there any specific reason why authors choose VQA? It will be better if authors can also compare with performance on other tasks before making the conclusion on Line 512.\n\nWe select VQA tasks as they offer a comprehensive assessment of the advanced capabilities of LVLMs. VQA tasks involve posing questions about images, demanding a profound understanding of both visual and language content. Compared to tasks like image classification and image captioning, challenging VQA tasks such as A-OKVQA and GQA can better showcase the formidable image comprehension and reasoning capabilities of LVLMs. Following GPT-4 [1], recent LVLMs [2, 3] also demonstrate their abilities by providing case studies on VQA-like examples, such as explaining jokes or story generation given an image. \n\nFollowing your advice, we also add experiments on image captioning tasks. Specifically, we ask InstructBLIP, MiniGPT-4, and LLaVA to generate captions for 500 images utilized in POPE. Subsequently, we evaluated these captions using BLEU-1, BLEU-2, METEOR, and ROUGE-L. The evaluation results are presented below:\n\n|              | BLEU-1 | BLEU-2 | METEOR | ROUGE-L |\n| ------------ | -----: | -----: | -----: | ------: |\n| InstructBLIP |   59.5 |   45.2 |   22.6 |    42.3 |\n| MiniGPT-4    |   41.1 |   28.8 |   25.6 |    44.7 |\n| LLaVA        |   22.0 |   13.9 |   19.8 |    22.4 |\n\n The results show that their performance on image captioning are basically consistent with POPE.\n\n\n\n> Missing References\n\nThank you for your suggestions. We will include these reference in the subsequent versions.\n\n\n\n[1] OpenAI. GPT-4 Technical Report. arXiv preprint arXiv: 2303.08774, 2023.\n\n[2] Ye Q, et al. mplug-owl: Modularization empowers large language models with multimodality[J]. arXiv preprint arXiv:2304.14178, 2023.\n\n[3] Zhu D, et al. Minigpt-4: Enhancing vision-language understanding with advanced large language models[J]. arXiv preprint arXiv:2304.10592, 2023.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "- The paper clearly explained the problem with the existing object hallucination metric, CHAIR, which is unstable over different prompt and requires complex human-crafted parsing rules to be used\n- The paper proposes a new metric, POPE, which solves the instability and complexity problems of CHAIR, by framing object hallucination into as a binary classification task. The stability and scalability of POPE is clearly evaluated through a series of experiment over multiple datasets.", "reasons_to_reject": "- The qualitative and quantitative analysis (Section 4.2 and 4.3) only shows the top-10 objects in MSCOCO, it is not enough to support the two hypotheses, it would be better to show the cumulative hallucinated count over all objects in MSCOCO for the first hypothesis, especially when there are some objects in the top-10 list (cat, horse, and toilet) while for the second hypothesis, taking only a single object (e.g., \"dining table\") for analysis is inadequate to provide a robust supporting evidence to conclude the hypothesis.", "questions_to_authors": "- On Figure 2a, given some objects (cat and horse) are rarely hallucinated despite having high frequency, have authors evaluated the trend of the object hallucination for the other objects (let say top-20 or top-30)? Does the trend hold in general? \n- On section 4.2, do author also evaluate the co-occuring on different objects other than dining table? Does it produce a consistent trend for different objects?\n- On section 4.3, what is the probing object used in HR_C@10?\n- On section 4.3, since the top-10 objects only cover 50% of all the hallucinated objects, what about the remaining objects? Does it produce a downward trend depending on the frequency of the objects in MSCOCO?\n- On section 5.4, why don't author compare POPE with other VL tasks such as image captioning or image classification? Is there any specific reason why authors choose VQA? It will be better if authors can also compare with performance on other tasks before making the conclusion on Line 512.", "paper_topic_and_main_contributions": "The paper presents a systematic study on object hallucination of large vision language models (LVLMs), providing both qualitative and quantitative analyses to better understand object hallucination in various LVLMs.  The paper showcases the limitation of the existing object hallucination metric, CHAIR, which is unstable and requires complex parsing rules to be used. To this end, the paper proposes a new object evaluation metric, POPE, which evaluates object hallucination through a binary classification task. The paper shows the effectiveness of POPE, demonstrating better stability and scalability to various datasets. The paper also displays that object hallucination is not always aligned with the task-based evaluation performance on the VQA task, showing the importance of considering both evaluation aspects for real-world applications.", "excitement": 4, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Justification_for_Ethical_Concerns": "-"}, "extra_text": {"Missing_References": "- Line 53: the reference to the image captioning task is quite recent, in my opinion, for the image captioning task, it would be better to also refer to Im2text (Ordonez et. al., 2011), FLICKR 8k (Hodosh et. al., 2013), FLICKR-30k (Young et. al., 2014), MSCOCO (Lin et. al., 2014), Pinterest40M (Mao et. al., 2016), and Conceptual Captions (Sharma et. al., 2018)\n- Line 54: the reference to the vqa task is quite recent, in my opinion, for the VQA task, it would be better to also refer to \"VQA: Visual Question Answering\" (Antol et. al., 2015) and \"Yin and Yang: Balancing and Answering Binary Visual Questions\" (Zhang et. al., 2016)", "Typos_Grammar_Style_and_Presentation_Improvements": "- For section 4.3, it would be better if authors can show the cumulative hallucination count of all objects in MSCOCO sorted by frequency\n- For section 5.4, it will be better if authors can also compare with performance on tasks other than VQA"}}, {"review_id": "z07YpPHPF2", "reviewer": "Reviewer_gyaq", "summary": "", "questions": "Please check \"Reasons To Reject\", where my questions are included.", "limitations": "", "rating": 4, "confidence": 4, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for the careful reading and helpful feedback. We respond below to your questions and concerns:\n\n> Where do you get the list of non-existent objects for negative sampling? Are they also from the predefined object list as used in CHAIR computation or a larger pool?\n\nWe utilize the same set of 80 COCO objects as used in CHAIR computation. Our analysis in Section 4 reveals that LVLMs have a tendency to hallucinate objects that are commonly seen and co-occurring. Therefore, we propose three different negative sampling strategies, which focus on random, frequently appearing, and frequently co-occurring objects respectively.\n\n\n\n> Evaluate models with more diverse prompts and answer candidates. For example, utilizing synonyms of \"yes\" and \"no\" might help to improve evaluation accuracy.\n\nThank you for your advice. We count the number of answers in POPE_random that start with \"yes\" or \"no\". The results are presented below.\n\n| InstructBLIP | MiniGPT-4 | LLaVA     | Multimodal-GPT | mPLUG-Owl |\n| :----------- | :-------- | :-------- | :------------- | :-------- |\n| 3000/3000    | 2974/3000 | 3000/3000 | 3000/3000      | 2991/3000 |\n\nThe findings indicate that a majority of answers start with either \"yes\" or \"no\", making them identifiable through a straightforward matching strategy. In our approach, we classify answers that incorporate synonyms of \"no\",  such as \"not\" and \"cannot\" as negative responses, while treating the rest as positive. Therefore, we believe the accuracy of our current methods is acceptable and leave further improvement such as using large language model as assistance to future work.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. The paper is well written and structured.\n2. The proposed pipepline could serve as a better evaluation method than CHAIR, to evaluate LVLMs for the object hallucination issue.\n3. The experiments are solid and exhaustive.\n", "reasons_to_reject": "I don't see explicit reasons to reject this paper. However, some parts could be made more clear or improved:\n\n* Where do you get the list of non-existent objects for negative sampling? Are they also from the predefined object list as used in CHAIR computation or a larger pool?\n* Evaluate models with more diverse prompts and answer candidates. For example, utilizing synonyms of \"yes\" and \"no\" might help to improve evaluation accuracy.", "questions_to_authors": "Please check \"Reasons To Reject\", where my questions are included.", "paper_topic_and_main_contributions": "This paper studies the object hallucination problem in large vision-language (VL) models. It first utilize CHAIR to evaluate modern LVLMs and discuss the disadvantages of CHAIR. Then, this paper proposes a new evaluation framework, named Polling-based Object Probing Evaluation (POPE). Results indicate that POPE is more stable and scalable, which could serve as a better evaluation tool for the object hallucination issue of LVLMs.", "excitement": 4, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Missing_References": "  "}, "extra_text": {}}]

Author rebuttal:

Meta-review: Three reviewers provided feedback for this work and were in consensus. They found the paper to be well written, the empirical study to be well done and well presented and the new metric to be sound. There were very few complaints for this paper. One complaint was questioning if object hallucinations were an important problem in todays VLMs. A second complaint was about using yes/no responses to measure this problem. And a third complaint was about expanding to all COCO categories. The rebuttals provided by the authors were very thorough and in my opinion have addressed all these concerns.
Decision: Accept-Main