Analyzing and Mitigating Object Hallucination in Large Vision-Language Models
Authors: Yiyang Zhou
Chenhang Cui
Jaehong Yoon
Linjun Zhang
Zhun Deng
Chelsea Finn
Mohit Bansal
Huaxiu Yao
Venue: ICLR 2024 poster (2024)
Keywords: Hallucination
large vision-language model
multimodal learning
Primary area: representation learning for computer vision, audio, language, and other modalities
TL;DR: A lightweight method to rectify object hallucination in large vision-language models

Abstract: Large vision-language models (LVLMs) have shown remarkable abilities in understanding visual information with human languages. However, LVLMs still suffer from object hallucination, which is the problem of generating descriptions that include objects that do not actually exist in the images. This can negatively impact many vision-language tasks, such as visual summarization and reasoning. To address this issue, we propose a simple yet powerful algorithm, LVLM Hallucination Revisor (LURE), to post-hoc rectify object hallucination in LVLMs by reconstructing less hallucinatory descriptions. LURE is grounded in a rigorous statistical analysis of the key factors underlying object hallucination, including co-occurrence (the frequent appearance of certain objects alongside others in images), uncertainty (objects with higher uncertainty during LVLM decoding), and object position (hallucination often appears in the later part of the generated text). LURE can also be seamlessly integrated with any LVLMs. We evaluate LURE on six open-source LVLMs and found it outperforms the previous best approach in both general object hallucination evaluation metrics, GPT, and human evaluations.

Published as a conference paper at ICLR 2024
ANALYZING AND MITIGATING OBJECT HALLUCINA- TION IN LARGE VISION-LANGUAGE MODELS
Yiyang Zhou' Chenhang Cui
Jaehong Yoon'
Linjun Zhang?
Zhun Deng
Chelsea Finn* Mohit Bansal'
Huaxiu Yao'
!hNC-Chapel Hill, Rutgers University, $Columbia University, $Stanford University
zhouyiyangai 1 abê gmailmmalll il com, osallymalcnee@mee com, huaxiuecs.unc. wnc.
ABSTRACT
Large vision-language models (LVLMs) have shown remarkable abilities in un- derstanding visual information with human languages. However, LVLMs still suffer from object hallucination, which is the problem of generating descriptions that include objects that do not actually exist in the images. This can negatively impact many vision-language tasks, such as visual summarization and reasoning. To address this issue, we propose simple yet powerful algorithm, LVLM Hal- lucination Revisor (LURE). to post-hoc rectify object hallucination in LVLMs by reconstructing less hallucinatory descriptions. LURE is grounded in rigorous statistical analysis of the key factors underlying object hallucination, including co- occurrence (the frequent appearance of certain objects alongside others in images), uncertainty (objects with higher uncertainty during LVLM decoding), and object position (hallucination often appears in the later part of the generated text). LURE can also be seamlessly integrated with any LVLMs. We evaluate LURE on six open-source LVLMs and found it outperforms the previous best approach in both general object hallucination evaluation metrics, GPT, and human evaluations. Our data and code are available peetts://qithann htw.hh...ooo////
INTRODUCTION
Large Vision-Language Models (LVLMs) have made significant progress in understanding real- world images, showing potential towards achieving general artificial intelligence (Liu et al., 2023d] Zhu et al.) 2023 Ye et al., 2023 Li et al.] 2023a| Maaz et al., 2023 Gong al. 2023). Although LVLMs have demonstrated their versatility and linguistic fluency. they often suffer from object hal- lucination in their generated text outputs (Wang et al. 2023a Liu al., 2023a Gunjal al. 2023). Object ballucination refers to the phenomenon of generating inaccurate descriptions for given im- age, including non-existent objects or omitting essential features. The issue with hallucinatory text generation in LVLMs is that it can mislead and deceive users in downstream applications that depend on these captions or descriptions, ultimately resulting in negative impact on various fields that em- ploy LVLMs. including robotics (Mai et al., (2033] Liu et al., (0236), medical imaging (Wang et al., 2023b
Hu et al. 2023). and human-computer interaction (Olson al., 1994 Brie et al. 2023).
Early works have attempted to address the problem of object hallucinations in small-scale mul- timodal pre-trained models by performing either fine-grained alignment across different modali- ties (Biten al. (2022) reducing object co-occurrence patterns with data augmentation (Rohrbach) cct al. 2018 Kim et al. 2023). However, the auto-regressive architecture of LVLMs differs signifi- cantly from small-scale multimodal pre-trained models, making their direct utilization impractical. A few recent works (Li et al., 2023c Liu et al., 2023a d) have studied to reduce object hallucina- tions in LVLMs by enhancing the quality of datasets used for fine-tuning. Yet, acquiring substantial number of high-quality examples for fine-tuning can be time-consuming and labor-intensive, requir- ing human expertise and effort. Instead, we aim propose lightweight method to post-hoc handle object hallucination by introducing LURE: LVLM hallcUination REvisor.
Concretely, LURE is grounded in rigorous statistical analysis that elucidates the underlying causal- ities of object hallucinations in LVLMs. This analysis delves into the relationship between the pre-training data and their corresponding textual responses from LVLMs that exhibit hallucinatory
"qqul contribution. Work was done during Yiyang Zhou and Chenhang Cui's remote internship at UNC.

---

Published as a conference paper at ICLR 2024
contents (Ordonez al. 2011 Lin al. 2014 Changpinyo al. 2021 Liu et al. 2023d). Both halluciaatt
our empirical and theoretical findings reveal that object hallucinations can be attributed to three key factors: co-oceurrence, uncertainty, and object position. First, if the training data contains spuri- ous co-occurring patterns between objects, language models may generate outputs based on these learned spurious associations, thus resulting in hallucinatory descriptions. Second, hallucinations occur more frequently on objects characterized by high uncertainty during generation. Lastly, posi- tional factors also play role, as more object hallucinations tend appear in the latter portions of the generated description due to the accumulation of misinterpretation.
Based on our statistical analysis, LURE develops a object hallucination revisor. This revisor takes potentially hallucinatory descriptions as input and converts them into accurate ones. To create the revisor, we first generate a hallucinatory dataset using GPT-3.5 by making two modifications to the original correct captions: (1) Insert additional object texts into the description that are likely to co- occur with the objects contained the initial description. This modification allows LURE to learn to disentangle such co-occurrence patterns effectively; (2) Replace uncertain objects or those at the end of descriptions with a placeholder tag, encouraging the revisor to re-evaluate these objects. In the end, train our hallucination revisor leveraging the acquired hallucinatory dataset. Once trained, the revisor can seamlessly integrate with any LVLM to correct potential hallucinatory descriptions.
Our primary contribution is LURE, lightweight and compatible post-hoe approach for rectifying object hallucination LVL This approach is grounded in our rigorous statistical analyses of object hallucinatory phenomena in LVLMs. Our experiments thoroughly evaluate LURE multiple existing open-source LVLMs. Compared to the best prior method, the results demonstrate that LURE can significantly reduce object hallucination under general object hallucination evaluation metrics (e.g., CHAIR (Rohrbach et al. 2018)), GPT evaluation, and human evaluation.
2 WHY Do LARGE VISION-LANGUAGE MODELS EXPERIENCE OBJECT HALLUCINNATION?
This section scrutinizes the root causes of object hallucinations in vision-language models via com- prehensive statistical analyses from three critical viewpoints: co-occurrence, uncertainty, and po- sition, recognized as the primary factors contributing to object hallucination. We further provide a rigorous theoretical explanation that complements our empirical findings on object hallucinations.
Notations.
Large Vision-Language Models (LVLMs) typically generate sentences in a free-form and auto-regressive manner, predicting the probability distribution of the next token progressively. In this context, we denote the input as 2, the correct answer y, and the generated sequence with a length of N₂ as {2I.... ........ NN... }. For a given LVLM, the probability of generating 2, as the i-th token can be described as p(zi|s<i, 2) (where 1 < S N.), and sęi refers to the previously generated tokens (=i,..., 1 Given description 8, we additionally define the complete object set, which is arranged in the order of appearance, as O, {o,,1...... tn, (o...........tt:. Here, n,,, and n. represent the number of hallucinatory and non-hallucinatory objects, respectively.
2.1 CO-OCCURRENCE AND SPURIOUS CORRELATION AMONG OBJECTS
In the realm of multi-modal models, "co-occurrence" denotes the frequent appearance of specific objects. When the training data includes spurious co-occurring patterns among objects, language models can generate outputs based on these learned associations. However, these associations may not hold true for test examples, resulting in hallucinatory outputs. For example, "grass" and "sky" frequently co-occur in the training data. The model falsely associates them and tends to generate "grass" and "sky" together even when only "grass" is present in the context.
In order to assess the influence of co-occurrence on object hallucination, we draw inspiration from Biten et al. (2022)and introduce a Co-occurrence Score denoted as CoScore. For each im- age description s, the corresponding co-occurrence score CoScore, is computed as the summation of co-occurrence degrees across all hallucinatory objects (o.,1..... 09, ). which defined as:
Bhtt Σ
(S(o,)) n S(o,,))
CoScore, = Σ
(1)
[8(o,)) 11 + [8(0,,))
(=1 j= #ors
2

---

Published as a conference paper at ICLR 2024
1.50
10:
Non-hall ucinatory Caption
loucitorly Obj
Non-hallucinatory Obj
Hallucinatory Caption
1.25
Hallucinatory Obj
2.5
Hallucinatory Obj
Frequency
Pequency
1.00
1.5
2-
0.1 0.2
0.3
0.4
0.5
2 Uncertainty Score
0.2
0.4
0.6
0.8
1.0
Co-occurrence Score
Position Score
(a) Co-occurrence
(b) Uncertainty
(c) Object Position
Figure 1: Comparison between hallucinatory and non-hallucinatory captions under different factors.
Here, S(.) denotes the set of all descriptions that mention specific object, and SS()) represents the cardinality this set.
Based on the definition of CoScore, compare the distribution of co-occurrence scores between hallucinatory and non-hallucinatory captions (please refer to Appendix A. for our experimental setting), As shown in Figure Ia hallucinatory captions tend to exhibit higher co-occurrence scores, which suggests a stronger association between object hallucination and co-occurrence.
2.2 OBJECT UNCERTAINTY
In language modeling. beam search (Holtzman al. 2019; Freitag & Al-Onaizan 2017) is ployed to predict words iteratively, introducing inherent uncertainty into the search process (Please refer illustrative examples in Appendix [D.I). This uncertainty is used measure of the model's confidence in generating the next token, and can be related to the hallucination problem, as objects with higher uncertainty are more likely be inaccurate. Here, aim to quantitatively investigate the potential relationship between the uncertainty associated with objects at each prediction step and the hallucinations.
Concretely, we represent the probability of autoregressive decoding for each object token as p(o,.]s<<.22 2). where denotes the positional index of object Os,i- For each object OsHis the cor- responding Uncertainty Score is defined as:
UnScore - log.(o... x),
where higher value of the uncertainty score indicates greater uncertainty. In Figure 1b we perform a statistical analysis examining the connection between hallucination and object uncertainty (refer to Appendix A for experimental details), Similar to the analysis of co-occurrence, hallucinatory objects are predominantly observed in the high-uncertainty range, while non-hallucinatory objects more frequently generated in the certain range.
2.3 OBJECT POSITION IN GENERATED DESCRIPTIONS
We also find a significant correlation between the object position in the generated descriptions and hallucination, where dominant hallucinations occur in the latter part of the descriptions. To validate it, we introduce the Positioning Score denoted as PoScore for each object o,,; as follows:
PoScore,, Index(o...)
N
where Index(o,.;) signifies the position index of object os,i within the entire description.
Based on the definition of PoScore, conduct a analysis of the positions of hallucination in the descriptions, illustrated in Figure Id (refer to Appendix A. for experimental details and Ap- pendix C.1.T for more analysis). These findings indicate that high-density areas of hallucinatory objects predominantly appear towards the end of the sequence. This pattern corroborates our ob- servation that object hallucination frequently occurs the latter segments of generated text. One plausible explanation for this observed trend is rooted in the autoregressive text generation process. In the initial stages, the model closely adheres to the semantic information of its input image, re- sulting in coherent beginnings. However, the generation progresses, the accumulation of past hallucinatory information and emerging uncertainties may steer the model off-course, ultimately leading to pronounced emergence of object hallucination.
3

---

Published as a conference paper at ICLR 2024
2.4 THEORETICAL EXPLANATION
After examining these empirical correlations, we proceed to offer theoretical insights to explain them (all proofs be found in Appendix B Specifically, focus on predicting the i-th token, denoted as 24. and introduce a predictive function denoted as f. For each object k within a set of objects represented as [ the function fa(s<;, signifies the predicted score associated with the k-th object. Here, K is defined as the total number of objects under consideration, and we use = 1 to denote the presence of the k--th object in an image and yk
otherwise. Furthermore, we make an assumption that [ 2) can be expressed as (ob 2), 9x2), ox (*<2, ye (yk Még Ta) and Pr(y = 1] = Pr(yu
-1) 1/2. For a training set DD the optimizer for the k-th class parameter 3x trained on D is defined as: 3x [D] Elecis (t) Vs.K W.. (s x), where Vi E {-1, 1 represents whether object & will occur at position Such model and optimizer are commonly used in the theoretical analysis of deep learning models (Carmon et al. 2019 Zhang et al. (022a)).
Co-occurrence. Based on this definition, we first consider co-occurrence. Without loss of general- ity, we assume that K = 2, and the first and second classes are frequently observed together, i.e., we observe (Ø1 ($<s, 2), 02 (s x)) among a fraction Pu E (0, 1) of samples when both y11 and y/2 are equal to 1. Here, to simplify the autoregressive process while maintaining sequential prediction man- ner, we consider using fi = (di( x), B.) for the prediction of the first object, and the second prediction, we model the information passed from the first information by (#1(s<s, 27), (1)) and con- sider f2 (dr (5 (1) (d2(se., z), 32). The model outputs the second object if f2(8<s; 2) > O.
Under this setting, we consider two sampling schemes: (1) Each class is sampled according to the original training distribution; (2) Each class is sampled by setting p Ppo These two sampling schemes result in two subset of samples D(1) DD2) with the same size. Denote the classifiers trained ) on ( and (22) by {f% (1,2) and ( (.E(1.2) respectively. Theorem 2.1 reflect reducing co-occurrence issue can lead smaller test misclassification error Err(-)-
Theorem 2.1 Suppose || < d, d/ID(()) + for k (1, and universal constants 0. We have
Err(f? S { Err(
Uncertainty.
We then our attention object uncertainty. Here, we consider the two following sampling schemes: (1) Each class is sampled with equal probability 1/K; (2) Each class is sampled the uncertainty score, defined
log( is above a certain threshold 7 > O. Here, ps is calculated as follows: Pa प Eoe.....) (s Bu)), where Der represents the training set. These two schemes result in two subsets of samples ((1) and D(2) with the same size. Given and 8ei, we make prediction about whether the k-th object is present in the image using ft. Theorem 2.2 illustrats that sampling more certain objects can lead to reduction test error.
Theorem 2.2 Suppose || 1 probability at least o(1),
p. d/(D(k) K for > and [K].
We will have with
K
K K
Σ K Err Eir(
Object Position. The effect of object position on object hallucination is closely tied to error or pre- diction uncertainty accumulation in autoregressive models. This topic has been extensively studied in time series analysis, and several theoretical models have been established to investigate it (Hannan
ct al. 1989 Ing 2007 Ding et al. 2017).
LVLM HALLUCINATION REVISOR
After thoroughly investigating the root causes of hallucinations, this section formally introduces our remedy, LURE, that mitigates object hallucinations in large vision-language models. Inspired by denoising autoencoders (Vincent et al. 2008), which is designed to reconstruct clean data from corrupted input, we employ a hallucination revisor in our approach that aims to transform potentially LVLM- generated hallucinatory descriptions into accurate ones. The framework of LURE is depicted in Figure 2 In the subsequent sections, we will delve into the training and deployment processes of the hallucination revisor.
4

---

Published as a conference paper at ICLR 2024
the sky and bright
sky:
Genernted doaription
Image
EVEMs
Deseille
CGAGPT
ChaGGP
image perenn wilking along beach their juntismn They appear to be looking out the ocean and the waves The bench is sandy and there are some rocks in the water. There
LVLMS
Co-oceurrence
Uncertainty
LURE
and the beach
Emage
Hallusinatury
blue the
description
there
picture depices serene lakeiide
looks like besutiful day
waters
Pesition unscrtainty
surounded by towering The lake's surfice reflects the sky and bright moun, crceting atmosphere of traquility and
+
Under revision!
LURE
image person strolling
Rechified deseription
Standard description
Hallucination correction
Sevenil indiviaull can the beach, they are enjoying water activities
Data preparation
LURE Training Phase
Training
LURE Inference Phase
The orange-shaded section shows the training paradigm of LURE, where the black-bordered part represents the hallucinatory data generation phase, including introducing co-occurring objects and replacing either uncertain objects or objects in later positions in the descriptions. The purple-bordered part indicates the revisor training process, with the masking process that can be referenced Alg. The orange-shaded section illustrates an example in the inference phase of LURE.
Figure An illustration of LURE Framework:
3.1 TRAINING HALLUCINATION REVISOR
In LURE, to train the hallucination revisor, we first curate training dataset. Each example in this dataset consists of an image accompanied by hallucinatory description, with the correct description serving as the output target. A significant challenge encountered during dataset curation lies in the generation naturally-occuriing hallucinatory descriptions. To overcome this challenge, LURE generates hallucinatory descriptions by modifying the accurate descriptions using GPT-3.5. These adjustments are guided by factors related to object hallucination, including co-occurrence, object uncertainty, and object position. In the following, detail these modifications:
Introducing Potential Co-Occurrence Objects. To create a more naturally occurring co- occurrence scenario, rather than relying counting co-occurrence frequencies from any specific datasets that may contain bias co-occurrence records, LURE leverages GPT-3.5 to deduce and in- corporate objects that are most likely to co-occur the scene into the original description.
Reconsidering Uncertain Objects & Objects in Later Position in the Descriptions. Hallucina- tion is more prone occur in objects with greater uncertainty and objects exist later in the descrip- tion. In this context, we anticipate that the revisor should place greater emphasis on and reevaluate these objects. To achieve this, we utilize string matching to replace objects with significant uncer- tainty and those located at the end of the description with the placeholder tag "[IDK]".
Here, quantify object uncertainty in descriptions, use the uncertainty values of noun tokens as proxy. Token uncertainty is expressed as the entropy of each token, denoted as log p (..,,r)) We classify tokens uncertain objects their corresponding uncertainty exceeds a threshold y, and if they are identified nouns. Like uncertainty, we determine the later object's position using the condition Index(2;; 2 1i Length(s) and the thresold 7). This approach enables the model to reassess and either replace "[IDK]" with more appropriate object based on the image remove entirely. Using these modification strategies, for every accurate description, we provide GPT-3.5 with list of potential co-occurrence objects, and list of uncertain objects.
We then prompt GPT-3.5 to generate the corresponding hallucinatory description using the prompts listed in Appendix A.3 Finally, leverage the constructed hallucination dataset to fine-tune LVLM and use it as revisor. Some cases of hallucinatory descriptions are in Appendix [.2] The training pipeline is illustrated in Alg.
5

---

Published as a conference paper at ICLR 2024
Algorithm Training LVLM Hallucination Revisor in LURE
Require:
training immae set X; groundtruth descriptions }: LVLM M(.): uncertainty threshold n; hallucina- tion revisor R=(-) with parameters 0: position threshold
1: Use GPT-3.5 construct hallucinatory description set Hota (see Appendix A.3 for more details)
2: Initialize the revisor's s parameter and an empty set Hnew + {
3: while not converged do
4: for each image X' and the correpsonding hallucinatory description h E Hold do
5: Generate description = MMx) with object O
object 0s,: E O. do if o..; in and log p(o,.s|M,, a) 2 then Add placeholder tag "[IDK]" to i... S.... Mask(h, os.i) if Os.: in and (n.ex 2 * Length(h) then Add placeholder tag "[IDK]" h, i.c. h Mask(h, Os.)) Put into Hnew
7:
8:
9:
10:
11:
Update parameter 0 with autoregressive loss [(Ro (Hnew):))
3.2
DEPLOYING HALLUCINATION REVISOR
In the inference stage, the trained revisor is employed to rectify the generated descriptions. Specif- ically, similar to the process of constructing hallucinated descriptions during the training phase, in the testing phase, we similarly integrate the placeholder tag "[IDK]" into the generated descriptions. This integration serves the purpose of enforcing the Revisor to reevaluate objects exhibiting high uncertainty or appearing later in the generated text. The inference pipeline is detailed in Alg. 2
Algorithm Inference Pipline of LURE
Require: test image æ;; LVLM M();; trained hallucination revisor Rī(-); uncertainty threshold 7, position threshold
I: Generate description
.((r)) with object set O.,
2: for object Oug,: O., do log (obojectt,MM.a x) then 4: Add placeholder tag "[[DK]" &t, i.c., 85 +- Mask(8t, 0ata)
5: if Index((sz.) 2 Length(s:) then
6: Add placeholder tag "[IDK]" 8t, i.c., +- Mask(st, omys)
7:
return (x))s)
4 EXPERIMENTS
In this section, we evaluate the performance of LURE aiming to answer the following questions: (1) Can LURE effectively reduce object hallucination in LVLMs compared to other baselines? (2) Can the key factors we' 'e identified related to hallucinations in LVLMs benefit the training process of the revisor?
(3) Is LURE sensitive to the revisor's backbone?
Datasets. MSCOCO (Lin et al. 2014) is comprehensive dataset used for image recognition, segmentation, and captioning. comprises over 300,000 images spanning more than 80 object categories, each with detailed annotations. Following (Li al. [023d Liu et al.
(202a)), al. 5,000 unique images from the COCO 2014 training dataset to evaluate performance. To train the hallucination revisor, we randomly selected 5000 image-text pairs from I.L (Liu et al.) 2023ch, ensuring that these images were different from the ones used in testing. In addition, we also evaluate the performance on other datasets, as discussed in Appendices B.4 and B.5
selected
Evaluation Metric. Caption Hallucination Assessment with Image Relevance (CHAIR) (Rohrbach al., 2018) is a widely-usei metric for evaluating object hallucination image captioning tasks. CHAIR assesses the quality of image captions by comparing them to the ground truth objects present the corresponding images. It calculates the proportion of objects mentioned in the caption that are not actually present in the image. There are two common variants of CHAIR: CHAIR, and CHAIRs. Both variants evaluate the degree of object hallucination, but at different levels: the object instance level and the sentence level, respectively. The two variants are formulated as follows:
[hallucinated objects)| CHAIR,
CHAIRs [captions with hallucinated ohjects)|
(4)
(all mentioned objects]
(all captions)

---

Published as a conference paper at ICLR 2024
Baselines. The comparison methods include: Original, which directly the generated descrip- tions from LVLMs; Teacher (Saha et al., 2023). which leverages blip2 (Li et al. 2023b) to generate short image descriptions and employs them contextual guidance for generating long-form descrip- tions; Chain-of-Thouse
(CoT)) (Wei et al. 2022), which involves the model initially listing objects and subsequently describing the image; Srrred--eccoing..... method that abstains from using sam- pling strategy and aims to make the model output the most certain tokens; EPT-Ensemble, which initially employs GPT-3.5 aggregate the commonly generated descriptions from multiple LVLMs, excluding the one under evaluation. Subsequently, GPT-3.5 utilizes these summarized common de- scriptions as guidance to rewrite the originally generated description from the evaluated model; GPT-Teucher, where GPT-3.5 is tasked with rewriting the original long-form description based on the blip2 generated short descriptions. Detailed descriptions about baselines are in Appendix [A.4
Evaluated LVL.s. We performed experiments utilizing six of the most recent LVLMMs, with their corresponding language models specified in parentheses: MiniGPTT- (Vicuna 13B) (Zhu et al.)
2023 LL (LLaMA 13B) (Liu al. 2023d). MMGPT (LLaMA 7B) (Gong et al. 2023)
LLaMA-Adapte (LLaMA 7B) (Zhang et al. 2003bb) mPLUG-OWl (LLaMA 7B) Ye et al.
2023
and InstructBLIP (Vicuna 7B) (Dai et al. 2023)).
Hyperparameter Settings. Unless specified, all experiments in the paper are using MiniGPT-4 as the backbone of the revisor, along with the training parameter settings provided in Appendix A.2 All hyperparameters are selected via cross-validation.
4.1 EVALUATION STRATEGIES AND RESULTS
Automated Object Hallucination Evaluation. We follow the guidelines presented in (Rohrbach) et al. 2018) to perform an automated calculation of CHAIR metrics for the MSCOCO dataset, where 80 objects involved in this automated evaluation process. In addition, we extend our evaluation to include other widely used metrics such as BLEU and CLIP score, which are commonly adopted in assessing the quality of image captioning. Detailed descriptions and results for these additional metrics can be found in Appendix B.3
Human and GPT Evaluations. Although automated evaluation strategies are efficient, they cannot encompass all objects present in the evaluated images. To overcome this limitation, we conducted a comprehensive human evaluation involving several native speakers. Please refer to Appendix [A.5] for the evaluation interface. In this human evaluation, participants are assigned the task of annotat- ing hallucinatory objects and we rank different methods based on human feedback. In addition to human evaluation, inspired from (Zheng et al.) (023), we also prompt GPT-3.5 to compare differ- ent descriptions. In this GPT evaluation, we provide the annotated information, including detection boxes and captions, and anticipate that GPT-3.5 can provide ranking for the descriptions from various methods. For GPT evaluation, we use the prompts referenced in Table Pale in the Appendix.
Results. In Table D and Table 2 report the results of automated evaluations and human and GPT evaluations under different LVLMs, respectively (see more analysis about the effectiveness of LURE on Appendices C.2 and C.1.3). Here, taking cost into account, only compare LURE with the four strongest methods human and GPT evaluations. Although Teacher, CoT, and GPT- Teacher can improve the performance compared to the original descriptions in most cases, LURE significantly enhances performance over these strong baselines, which effectively reduces object hallucination generated descriptions. One potential reason for this is that all of these baselines experience eттor propagation to some extent. For instance, CoT's 's linear guidance can lead to errors if the object listing step is incorrect. In contrast, LURE directly corrects hallucinatory descriptions using guidance from potential factors that can trigger hallucinations.
4.2 ANALYSIS OF LURE
Table 3: Compared LURE fine-tuning method using the training data of revisor.
Are the Performance Gains of LURE from Us- ing Constructed Hallucination Datasets? To ver- ify that the performance gains of our method are not from using additional data to train the revisor, we fine-tuned the original LVLMs with the additional dataset. The results on MiniGPT-4 are shown in Ta- ble 3 where "Original" represents the descriptions
Model
CHAIRs 4
CHAIR, J
Original
26.8
7.3
FT (add'l data)
31.0
7.2
LURE (Ours)
19.7
4.9
7

---

Published as a conference paper at ICLR 2024
Table 1: Automated hallucination evaluation performed under six LVLMs using CHAIRs (Cs) and CHAIR, (C1), where smaller values indicate less object hallucination. For additional metrics, please refer to Appendix B.3
MiniGPT-4
LLaVa Cs 4
MMGPT
LLaMA-Adapter
mPLUG-OWl
InstructBLIP
Cs 4
Cr 4
C₁
Cs 4
Cr
Cs 4
Cr
Cs
C
Cs 4
Cr +
Original
26.8
7.3
54.0
11.3
56.6
11.0
58.8
13.7
71.2
16.5
40.0
8.2
Teacher
24.0
5.7
49.9
9.3
53.4
7.5
40.8
9.4
62.4
13.0
36.4
7.5
CoT
31.6
9.4
47.6
9.0
48.8
17.5
43.3
9,4
56.9
13,4
35.7
7.8
Greedy-Decodigg Decoding
25.1
6.6
50.9
10.0
50.6
8,4
55.9
13.7
55.1
12.8
35,5
7.8
GPT-Ensembbl
41.0
10.6
43.0
10.7
51.0
11.1
47.1
13.0
52.0
15.2
51.0
13.0
GeT-Tracher
25.3
7.6
38.0
7.8
26.7
9.3
49.0
12.4
22.0
9.0
32.0
7.8
LURE (ours)
19.7
4.9
27.1
22.2
5.6
35.3
18.8
5.4
21.0
5.1
Table 2: We conducted evaluations for description ranking, comparing the four strongest baselines in both human ('H') and GPT ("G') evaluations. Metrics represent the average rankings within the top positions, with lower rankings indicating less ballucination.
MiniGPT-4 G
LLaVa G
MMGPT
LLaMA-Adapte
mPLUG-Owl
InstructBLIP
1
H4
G
I
G
H4
G
14
G
II
Original
3,97
3.10
4.55
4.62
3.66
3,25
4.79
4.45
4.25
3,98
4.29
4.77
Teacher
3,36
3.83
3.30
3.07
3.09
3.20
3.00
3.13
3,25
3.66
3.34
3.53
CoT
2.44
2.83
3.05
2.52
4.38
4.07
2.63
2.10
3.75
3.13
2.78
2.21
GeT-Teachhr
3.56
3.28
2.45
2.96
2.16
2.90
2.68
3.24
2.50
2.44
3.12
2.56
LURE (ours)
1.67
1.96
1.65
1.83
1.61
1.58
1.90
2.08
1.25
1.79
1.47
1.93
of MiniGPT-4. According to Table 3, LURE outperforms the fine-tuned LVLMs, which indicates that method indeed reduces object hallucination by post-hoe rectifying potential hallucinatory descriptions rather than using additional data.
Ablation Study Do the Hallucination Factors Contribute Performance Gains? To demonstrate the impact of considering co-occurrence, uncer- tainty, and object position in reducing hallucination, we conducted ablation experiments and report the results in Table 4 where "Original" represents the descriptions of MiniGPT-4. the ablation experi- ments, we trained and deployed the revisor without each of the three factors, one at time. The results show that all three factors contribute to training strong hallucination revisor to reduce object hallucination. Furthermore, we have also conducted an analysis of the changes in these three factors before and after applying the revisor, presented in Appendix I.I.I This analysis demonstrates that LURE can effectively reduce instances of halluci- nation caused by these factors.
Table 4: Ablation studies on three hallucination factors.
Model
CHAIRs 1 CHAIR;
Original
26.8
7.3
w/o Co-oceurrence
22.6
4,9
wfo Uncertainty
21.2
5.4
w/o Position
22.3
5.8
LURE (Ours)
19.7
4.9
Robustness Analysis of the Hallucination Revi- sor. We further analyze the robustness of the revi- sor with respect to different backbones. Specifically, trained the revisor the same dataset using different backbones: MiniGPT-4 LLaMA- -ddaterr, and mPLUG-OWl. The results are reported in Table 5] where "Original" represents the descriptions of MiniGPT-4 We can observe that despite the varying performance of each backbone, LURE consistently improve the performance compared to the original description, which further indicate the effective- ness of LURE. Additionally. we analyze the results of LURE with respect to various uncertainty thresholds in Appendix C.1.3] The findings demonstrate that LURE exhibits strong performance across a wide range of uncertainty thresholds.
Table 5: Performance under different hallucina- tion revisor backbones.
Backbone
CHAIRs
CHAIR,
Original
26.8
7.3
MiniGPT-4
19.7
4.9
LLaMA-adapter
21.3
5.2
mPLUG-OWl
22.1
5.4
Case Analysis. We select several strong baselines and presented case with rectified descriptions in Figure Compared with other approaches, LURE excels in providing accurate image
8

---

Published as a conference paper at ICLR 2024
the mege fecten mnmomee
cach di poc MENEEEAI of cumsi sala mure momile by alis did und aad lod fwoe Bil aquaaat
Teacher
plom oas das sile the som du
do pole tode lealy kita ln hecigrond, Boos ar pupple rime arat soiil pe mg pni
budal lime vn
plantie
Body hebeiging people pravert
sackuc vfiicc
LUBE
Alsss one whl sai locied molkeved lome up
while wine
Teacher
accend on of underr right andtive
Figure 3: A case study comparing the levels of hallucination among various baselines.
description. In the case, LURE accurately depicts the primary elements (e.g., sandwich, chair, plate) while avoiding hallucinatory objects like the fork and handbag. Although other baselines partially reduce hallucination, they still exhibit object hallucinations in their descriptions. Additionally, we also mitigate logical errors to some extent, including object orientation and actions. Further case
analyses can be found in Appendices] D.3 and D.4
5 RELATED WORK
Vision-Language Models. Vision-language pre-trained models, as exemplified by (Li et al. 2021 Zeng et al. 2021), demonstrate substantial capabilities in modeling interactions between visual and textual information, especially when fine-tuned for specific tasks. Recently, autoregressive large- scale language models (LLMs) (Brown et al. 2020: Chowdhery et al. 2022, Touvron et al. 2023 Zhang et al., 20226 Chiang et al.) 2023] Taori al. 2023) have ushered
new era of vision-
language models. These models, known as LVLMs, integrate L.MSs with visual modality and show- case impressive visual understanding through end-to-end training techniques that directly decode vi- sual and text tokens in unified manner (Liu al.] [023d] Zhu et al. 2023 Ye et al. [023 Li et al.) 2023a). However, similar to VLMs, LVLMs also face the challenge of object hallucination (Wang al., 2023a; Rohrbach et al. 2018). This form of object hallucination is more pronounced and widespread in the long-form descriptions produced by LVLMs compared to the shorter descriptions generated by VLMs (Zhang al., 2023a).
Hallucination in VLMS and LVLMs. In VLMs, hallucination typically refers to scenarios where the generated descriptions contain information that does not exist in the visual modality (Rohrbachi et al. 2018: Biten et al.) 2022 Wang et al.) 2023a}. Addressing object hallucination in VLMs is primarily achieved through techniques such as fine-grained contrastive learning (Zeng al. 2021
ROI feature fusion (Biten et al. 2022). and eliminating co-occurrence patterns through data aug- mentation (Kim et al. (023). However, the training paradigms between traditional VLMs and re- cent LVLMs differ, and the new autoregressive training paradigm in LVLMs makes it challenging to directly apply hallucination mitigation methods used in VLMs to LVLMs. Recent research has begun to address the issue of object hallucination in LVLMss including hallucination evaluation and detection (Wang et al. 2023a; Liu et al.] [003aa Li et al. 2023d). as well as the construction of higher-quality datasets for fine-tuning (Gunjal al. 2023 Li et al.) 2023c Liu et al. 2023a d) Nevertheless, acquiring substantial number of high-quality examples can be time-consuming and labor-intensive. Instead, grounded in statistical analysis of hallucination, we propose conceptually different approach, LURE, to post-hoc rectify object hallucination. We have already demonstrated its effectiveness in reducing hallucination and its compatibility with various LVLMs.
6 CONCLUSION
In this paper, our objective is to address the challenge of object hallucination in LVLMs. We in- troduce a lightweight post-hoc method, named LVLM Hallucination Revisor (LURE), designed to rectify object ballucination in the generated descriptions produced by LVLMs. LURE is grounded in three key factors known to contribute to object hallucination: co- no-occurrence, uncertainty, and object position. These factors have been demonstrated induce hallucination both empirically and theo- retically. Our experiments, conducted on six open-source LVLMs, demonstrate the effectiveness of LURE in mitigating object hallucination in M-gnnerae descriptions.

---

Published as a conference paper at ICLR 2024
REPRODUCIBILITY STATEMENT
For our theoretical results, we present complete proofs for all our claims in Appendix and engage in thorough discussion of the assumptions. As for empirical results, delve into the details of the experimental setup, introduce additional metrics, and provide a comprehensive overview of
baseline details, all of which be found in Appendices a B.3 Additionally, Appendix [ we offer detailed case demonstrations and comparisons. Furthermore, we include template prompis used during these analytical processes within the A..3 and A.4 is worth noting that we are com- mitted to open-sourcing the code related to our research after publication.
ACKNOWLEDGEMENT
This work was partially supported by Juniper, Networks ONR grant N00014-22-1- 2621, NSF-AI Engage Institute DRL-21 12635,, DARPA ECOLE Program No. HR001 12390060, and DARPA Machine Commonsense (MCS) Grant N66001-19-2-4031. We also thank Center for AI Safety and Google Cloud Research Credits program for supporting our computing needs. The views contained in this article are those of the authors and not of the funding agency.
REFERENCES
Peter Anderson, Basura Fernando, Mark Johnson, and Stephen Gould. Spice: Semantic propo- sitional image caption evaluation. In Computer Vision-ECCV 2016: 14th European Confer- ence, Amsterdam, The Netherlands, October 11-14, 2016, Proceedings, Part V 14, pp. 382-398. Springer, 2016.
Satanjeev Banerjee and Alon Lavie. Meteor:
An automatic metric for mt evaluation with improved correlation with human judgments. In Proceedings of the acl workshop on infrinsic and extrinsic evaluation measures for machine translation and/or summarization, pp. 65-72, 2005.
Ali Furkan Biten, Lluis Gómez, and Dimosthenis Karatzas. Let there be clock on the beach: Reducing object hallucination in image captioning. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1381-1390, 2022.
Paul Brie, Nicolas Burny, Arthur Sluýters, and Jean Vanderdonckt. Evaluating large language model on searching for gui layouts. Proceedings of the ACM on Human- Interaction, 7 133 2023.
Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal,
Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners. Advances in neurul information processing systems, 13:1877-1901, 2020.
Yair Carmon, Aditi Raghunathan, Ludwig Schmidt, John C Duchi, and Percy Liang. Unlabeled data improves adversarial robustness. Advances in neural information processing systems, 32, 2019.
Soravit Changpinyo, Piyush Sharma, Nan Ding, and Radu Soricut. Conceptual 12m: Pushing web-scale image-text pre-training to recognize long-tail visual concepts. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3558-3568, 2021.
Wei-Lin Chiang, Zhuohan Li, Zi Lin, Ying Sheng, Zhanghao Wu, Hao Zhang, Lianmin Zheng, Siyuan Zhuang, Yonghao Zhuang, Joseph E. Gonzalez, Ion Stoica, and Eric P. Xing. Vicuma: An open-source chatbot impressing gpt-4 with 90%* chatgpt quality, March 2023. URL https:
Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung, Charles Sutton, Sebastian Gehrmann, et al. Palm: Scaling language modeling with pathways. arXiv preprint arXiv:2204.023: 2022.
Wenliang Dai. Junnan Li, Dongxu Li, Anthony Meng Huat Tiong, Junqi Zhao, Weisheng Wang, Boyang Li. Pascale Fung, and Steven Hoi. Instructblip: Towards general-purpose purpose vision-language models with instruction tuning, 2023.
10

---

Published as a conference paper at ICLR 2024
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A scage-scale hi- erarchical image database. In 2009 JEEE conference on computer vision and pattern recognition, pp. 248-255. Icee, 2009.
Jie Ding, Vahid Tarokh, and Yuhong Yang. Bridging aic bic: a new criterion for autoregression. IEEE Transactions Information Theory, 64(6))4024-44)) 4043, 2017.
Markus Freitag and Yaser Al-Onaizan. Beam search strategies for neural machine translation. arXiv preprint 00t.:01702.010000 2017.
Chaoyou Fu, Peixian Chen, Yunhang Shen, Yulei Qin, Mengdan Zhang, Xu Lin, Zhenyu Qiu, Wei Lin, Jinrui Yang, Xiawu Zheng, et al. Mme: A comprehensive evaluation benchmark for multi- modal large language models. arXiv preprini arXiv:2306. 13394, 2023.
Tao Gong, Chengqi Lyu, Shilong Zhang, Yudong Wang, Miao Zheng, Qian Zhao, Kuikun Liu, Wenwei Zhang, Ping Luo, and Kai Chen. Multimodal-gpt: A vision and language model for dialogue with humans, 2023.
Anisha Gunjal, Jihan Yin, and Erhan Bas. Detecting and preventing hallucinations in large vision language models. arXiv preprint arXiv:2308. 06394, 2023.
Edward James Hannan, AJ McDougall, and Don Stephen Poskitt. Recursive estimation of autore- gressions. Journal of the Royal Statistical Society: Series B (Methadological), 51(2):117-233 1989.
Ari Holtzman, Jan Buys, Li Du, Maxwell Forbes, and Yejin Choi. The curious case of neural text degeneration. arXiv preprint arXiv: 1904.097// 2019.
Mingzhe Hu, Shaoyan Pan, Yuheng Li, and Xiaofeng Yang. Advancing medical imaging with language models: journey from n-grams to chatgpt. arXiv preprint wwXiv:2304.049200 2023.
Ching-Kang Ing. Accumulated prediction errors, information criteria and optimal forecasting for autoregressive time series. The Annals of Statistics, 35(3):1288-1277, 2007.
Jac Myung Kim, A Koepke, Cordelia Schmid, and Zeynep Akata. Exposing and mitigating spurious correlations for cross- retrieval. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 2584-2594, 2023.
Bo Li, Yuanhan Zhang, Liangyu Chen, Jinghao Wang, Jingkang Yang, and Ziwei Liu. Otter: A multi-modal model with in-context instruction tuning. arXiv preprint arXiv:2335.03376. 2023a.
Junnan Li, Ramprasaath Selvaraju, Akhilesh Gotmare, Shafiq Joty, Caiming Xiong, and Steven Chu Hong Hoi. Align before fuse: Vision and language representation learning with momentum distillation. Advances in neural information processing systems, 34:9694-9705, 2021.
Junnan Li, Dongxu Li. Silvio Savarese, and Steven Hoi. Blip-2: Bootstrapping language- image pre-training with frozen image encoders and large language models. arXiv preprint arXiv:2301. 12597, 2023b.
Lei Li, Yuwei Yin, Shicheng Li, Liang Chen, Peiyi Wang, Shuhuai Ren, Mukai Li, Yazheng Yang, Jingjing Xu, Xu Sun, et al. Mti: A large- scale dataset towards multi-moda multilingual instruc- tion tuning. arXiv preprint arXiv:2306. 30t.:2306604888, 2023c.
Yifan Li, Yifan Du, Kun Zhou, Jinpeng Wang, Wayne Xin Zhao and Ji-Rong Wen. Evaluating object hallucination in large vision-language models. arXiv preprint arXiv:2305. 10355, 2023d.
Chin-Yer Yew Lin. Rouge: A package for automatic evaluation of summaries. In Text summarization branches oul, pp. 74-81, 2004.
Tsung- Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitncck. Microsoft coco: Common objects in context. In Computer Vision Vision-ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part V 13, pp. 740-755. Springer, 2014.

---

Published as a conference paper at ICLR 2024
Fuxiao Liu, Kevin Lin, Linjie Li, Jianfeng Wang, Yaser Yacoob, and Lijuan Wang. Aligning large multi-modal model with robust instruction tuning. arXiv preprint arXiv:2306.14563 2023a.
Haokun Liu, Yaonan Zhu, Kenji Kato, Izumi Kondo, Tadayoshi Aoyama, and Yasuhisa Hasegawa. Lim-based human-robot collaboration framework for manipulation tasks. arXiv preprint arXiv:2308. 14972, 2023b.
Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning, 2023c.
Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning. arXiv preprint arXiv:2304.0844 2023d.
Muhammad Maaz, Hanoona Rasheed, Salman Khan, and Fahad Shahhaz Khan. Video-chatgpt: Towards detailed video understanding via large vision and language models. arXiv preprint arXiv:2306.0553 23040044444 2023.
Jinjie Mai, Jun Chen, Bing Li, Guocheng Qian, Mohamed Elhoseiny, and Bernard Ghanem. Llm as a robotic brain: Unifying egocentric memory and control. arXiv preprint arXiv:2304.09349, 2023.
Gary M Olson, James D Herbsleb, and Henry H Reuter. Characterizing the sequential structure of interactive behaviors through statistical and grammatical techniques. Coman-Coomptee Interac- fion, ((3-4):427-472 1994.
Vicente Ordonez, Girish Kulkarni, and Tamara Berg. Im2text: Describing images using million captioned photographs. Advances in neural information processing systems, 24, 2011.
Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. Bleu: method for automatic evaluation of machine translation. In Proceedings of the 40th annual meeting of the Association for Computational Linguistics, pp. 311-318, 2002.
Alec Radford, Jong Wook Kim, Chris Hallacy. Aditya Ramesh, Gabriel Goh, Sandhini Agar- wal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. Learning transferable visual models from natural language supervision, 2021.
Anna Rohrbach, Lisa Anne Hendricks, Kaylee Burns, Trevor Darrell, and Kate Saenko. Object hallucination in image captioning. In Proceedings of the 2018 Conference Empirical Methods in Natural Language Processing, pp. 4035-4045 2018
Swarnadeep Saha, Peter Hase, and Mohit Bansal. Can language models teach weaker agents? teacher explanations improve students via theory mind. arXiv preprint urXiv:2306.09299 2023.
Piyush Sharma, Nan Ding, Sebastian Goodman, and Radu Soricut. Conceptual captions: A cleaned, hypernymed, image alt-text dataset for automatic image captioning. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume Long Papers), pp- 2556-2560 2565,, 2018
Rohan Taori, Ishaan Gulrajani, Tianyi Zhang, Yann Dubois, Xuechen Li, Carlos Guestrin, Percy Liang, and Tatsunori B. Hashimoto. Stanford alpaca: An instruction-following llama model. ftttps://githueee www.tcooooooooc...coooooooooo.ot httr://ttuuuutttt lab/ aaabbstaafood_appccaaa 2023.
Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.1397// 2023.
Ramakrishna Vedantam, C Lawrence Zitnick, and Devi Parikh. Cider: Consensus-based image description evaluation. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 4566-4575, 2015.
Pascal Vincent, Hugo Larochelle, Yoshua Bengio, and Pierre-Antoine Manzagol. Extracting and composing robust features with denoising autoencoders. In Proceedings of the 25th international conference on Machine learning, pp. 1096-1100 103, 2008
12

---

Published as a conference paper at ICLR 2024
Junyang Wang, Yiyang Zhou, Guohai Xu, Pengcheng Shi, Chenlin Zhao, Haiyang Xu. Qinghao Ye, Ming Yan, Ji Zhang, Jihua Zhu, et al. Evaluation and analysis of hallucination in large vision- language models. arXiv preprint 23X1:23008.1116, 2023a.
Sheng Wang, Zihao Zhao, Xi Ouyang, Qian Wang, and Dinggang Shen. Chatcad: Interac- tive computer-aided diagnosis on medical image using large language models. arXiv preprint 0.Xiv:2302.07257, 2023b.
Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoe Le, Denny Zhou, et al. Chain-of-thought prompting elicits reasoning in large language models. Advances in Neural Information Processing Systems, 35:24824-24837, 2022.
Qinghao Ye, Haiyang Xu, Guohai Xu, Jiabo Ye, Ming Yan, Yiyang Zhou, Junyang Wang, Anwen Hu, Pengcheng Shi, Yaya Shi, et al. mplug-owl: Modularization empowers large language models with multimodality. arXiv preprint arXiv:2304. 14/78, 2023.
Yan Zeng, Xinsong Zhang, and Hang Li. Multi-grained vision language pre-training: Aligning texts with visual concepts. arXiv preprint arXiv:2111.08276 2021.
Linjun Zhang, Zhun Deng, Kenji Kawaguchi, and James Zou. When and how mixup improves calibration. In International Conference Machine Learning, pp. 26135-26160. PMLR, 2022a.
Muru Zhang, Ofir Press, William Merrill, Alisa Liu, and Noah A Smith. How language model hallucinations can snowball. arXiv preprint arXiv:2305. 13534, 2023a.
Renrui Zhang. Jiaming Han, Aojun Zhou, Xiangfei Hu, Shilin Yan, Pan Lu, Hongsheng Li, Peng Gao, and Yu Qiao. Llama-adapter: Efficient tine-tuning of language models with zero-init atten- tion. arXiv preprint 0rXiv:2303.16199 2023b.
Susan Zhang, Stephen Roller, Naman Goyal, Mikel Artetxe, Moya Chen, Shuohui Chen, Christo- pher Dewan, Mona Diab, Xian Li. Xi Victoria Lin, et al. Opt: Open pre-trained transformer language models. arXiv preprint arXiv:2205.01065 2022b..
2022b.
Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q Weinberger, and Yoav Artzi. Bertscore: Evaluat- ing text generation with bert. arXiv preprint arXiv:1904. 09675, 2019.
Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric Xing, et al. Judging llm-as-a-judge with mt-bench and chatbot arena. arXiv preprint 230.:2300005685, 2023.
Deyao Zhu, Jun Chen, Xiaoqian Shen, Xiang Li, and Mohamed Elhoseiny. Minigpt-4: En- hancing vasion-language understanding with advanced large language models. arXiv preprint arXiv:2304. 10592, 2023.
A EXPERIMENTAL DETAILS
A.1 EXPERIMENTAL SETTING FOR THE HALLUCINATION ANALYSIS
Experimental Setting for Co-occurrence Analysis. The objects this experiment are based on the 80 object labels annotated in (Rohrbach et al. (018)) from the COCO dataset, and the image descriptions are generated by MiniGPT-4 based on inference results from 5000 images in the COCO 2014 train dataset.
Experimental Setting for the Uncertainty Analysis. Because uncertainty and position analysis relatively independent from co-occurrence, in order to avoid conducting statistical analysis on the training set distribution, the statistical data for uncertainty analysis is derived from MiniGPT-4's deseriptions of 200 images from the COCO 2014 test dataset. The computation of uncertainty is performed using - log pl (2)85))))))
Experimental Setting for the Analysis of Position of Hallucinated Objects. Similar the uncer- tainty analysis, we used the manually annotated descriptions of MiniGPT-4 for 200 images from the COCO 2014 4 test dataset, due to the need for precise positioning.
13

---

Published as a conference paper at ICLR 2024
A.2 TRAINING SETTINGS FOR REVISOR
The overall revisor training setting is similar to MiniGPT-4. Here, we only need one A100 80G GPU for training, which takes approximately 10 minutes. We present hyperparameter settings of the LURE during the training phase, as shown in T
Table 6: Training hyperparameters.
Hyperparameters
Training steps
410
Warmup steps
50
Max length
512
Batch size of multi-modal instruction data
12
Optimizer
AdamW
Learning rate
3e-5
Learning rate decay
Cosine
Adam W e
1e-6
AdamW B
(0.9, 0.999)
Weight decay
0.05
A.3 PROMPTS FOR TRAINING DATASET
We leverage the in-context few-shot learning capability of GPT-3.5 to generate hallucinatory data automatically for revising. Initially, we prompt GPT-3.5 to provide a list of objects that are highly likely to co-Occur with the objects mentioned in the given description. Next, as MiniGPT-4) to generate descriptions for the training set of 5000 images. During this process, will save nouns with log p(2;) x));, greater than the uncertain threshold y in the decoding process to the list of uncertain objects corresponding to each image. Subsequently, we direct the model take the original description and incorporate randomly chosen word from the "co-occur objects" list, as well as another randomly chosen word from the "uncertain objects" list, into it. Detailed prompts are listed in Table 7
LVLMs (such
and few examples are presented in T9
A.4 DETAILS ABOUT BASELINE
In this section, will provide detailed explanation of the settings used for the baseline in Table I including some parameter settings and prompt configurations. The detailed prompt for baselines be seen in Table g
Teacher: The "Teacher" approach involves generating short descriptions for the images via blip2 (Li et al. 2023b) and using them as context to guide the model in generating descriptions. By providing these descriptions additional information, the model can benefit from the guidance and produce more accurate or relevant descriptions.
CoT: The "CoT" method asks the model to first list the objects identifies in the image and then describe the image based those objects. It draws inspiration from the concept of chain thought (Wei et al. 2022) and aims to guide the model in generating accurate descriptions by focusing object recognition.
Greedy-Decoding: The difference between the "Greedy-Decodine strategy and the "Original" strategy that in the GGreedy-Decoding strategy, the model uses greedy decoding instead of sampling during the generation of image descriptions produce the most deterministic output. This approach used to explore the potential connection between the generation of illusions and the use of sampling.
GPT-Ensemble:: In "GPT-Ensemble." we utilize GPT-3.5 to summarize the common elements in the descriptions generated by multiple LVLMs, excluding the one being evaluated. Subsequently, employ GPT-3.5 rewrite the description of the evaluated LVLM, using the identified com- mon elements from the descriptions the other models to correct any dissimilar parts in the evaluated model's description.
14

---

Published as a conference paper at ICLR 2024
Table 7: The prompt for GPT-3.5 API generate required hallucination dataset. "Instruction used to ask ChatGPT to provide a list of co-occurring objects based on the description, while "Instruction 2" used to integrate the objects obtained from the co-occurring object list and the objects from the list of uncertain objects into the given description.
Instruction 1:
List three other objects that you think are most likely to appear with the objects in the scene described below:
[description}
Output in strict accordance with the following format:
Object one
Object two
Object three
Instruction 2:
Input caption: {description}
co objects list: {co objects list)
uncertain _ojiitt__oettssseeeee list: (uncertain_obije) list)
Select object from "co objects list" and "uncertain objects list" respectively and add it to "Input caption" to get "Output caption". (Try not to change the format)
Output caption:
"GPT-Teacher" represents the process of providing the GPT-3.5 API with con- textual references and descriptions from the model's output, allowing it to revise the inaccurate description generated by the model into more accurate version based on the contextual informa- tion.
GPT-Teacher:
A.5 DETAILS ABOUT MANUAL ANNOTATION EVALUATIONS
The manual evaluation annotation interface provides a user-friendly interface for performing manual annotations and capturing evaluation feedback. The interface is hosted the Amazon Web Services (AWS) platform, which offers scalability, reliability, and security for handling annotation tasks. As shown in Figure the images. We then provided a binary label (0/1) to indicate whether each description contained hallucinations. Based on the fine-grained annotation results, similar GPT evaluation, we sorted the results from different baselines.
annotated all objects and hallucinated objects in the descriptions based on
B DETAILED PROOF
B. PROOF OF THEOREM
2.1
Let us denote N [D(1 = |D(2)), For the detection rule of the first object, we have
3 [D()
Σ
Vi.k (x(2))))))
As (sch, Wi. N(v. ph. 1). we write
Vo.k (x(S),))) pi + Ci
Now, suppose among all samples, a fraction po E (0, 1) of samples have both y and y/2 are equal to 1. We can then write
Pg-N
Pg
(8())310)) = (Pol4 + N
Σ
G,1 Po12
Σ G.2)
N
15

---

Published as a conference paper at ICLR 2024
Table 8: Prompts for baselines.
Teacher:
Reference caption:
(blip2 )
Please refer to reference caption and describe this picture:
CoT:
Human:
Please list the main objects in the picture and strictly follow the following format: (objectl, object2, object3......)
AI:
(objects list}
Human:
Describe this image
Al:
(description}
GPT-Ensemble:
Reference captions l:(description of model }}
Reference captions 2: description of model 2}
Reference captions 3:/description of model 3)
Reference captions 4: desecription of model 4]
Reference captions 5: description of model 5}
Original Descriptioo::(eeerrppiooo
Synthesizing the commonalities of Reference captions 1-5, and then removing the parts in the Original Description that do not align with the commonalities, while preserving the original format. Answer:
GPT-Teachhr::
Reference caption:
(blip2 }aption}
Original description:
{description)
Rewrite the original description align it with the reference caption, delete some objects that you think are hallucinations, and keep the original format. Answer:
Use (---) to denote the cumulative distribution function of a standard normal distribution. Then for the prediction function f2 = (s1),, ((1) + ()z(s<.,)), B()) we have
(rr()(2))
P(( (8 (((b)( 2), 8(1)) (02(8 4), 88') <0 =
P((0i 2), ((1)) + (d2 2), 2( 0 1 1)
+ (pi, B1) + (B2,
V 113 112 + 11
Poll: 112 + Pa 22
=
08 + |P 1| endd + aovd + o(1).
16

---

Published as a conference paper at ICLR 2024
Instruction::
11) the utsect lst mentored the desciption
helucinatryy stjecte merricned the description objects only)
descrition yeu,
Format Requirements:
following "'by"
objects description:
2n) Please 5il the "aalucinatory onjects" the to tollowing former ("holt" retors to the illsorr onjects that exkst the desoription but are not present the image::
1,. hotg?, hobj
Modil description: Thene ane four childhen on benchh awarig bassball uniforma. Teo them are boye and the other two are girts. All them are smillng and looking at the camera bench located a park playground, there vabble the backggrund. There is building the right sice image. children weening helmees, and
Submit
Figure 4: Human evaluation annotation interface.
Table 9: The prompt for ChaiGPT3.5 evaluation.
Instruction:
Suppose are hallucination annotator who judges the degree of hallucination based on objects, and you have the following image information. Referenee captions: five captions from COCO)
Bounding box: bounding boxes}
Please just provide the ranks for the below descriptions without any explanation, where the caption ranks first with the most hallucinations. The output format: [caption.x.... Descriptions:
caption I: [description 1}
caption 2: {description 2}
caption_3: (description.3 3}
caption_4: {description.44
caption 5: {description 5}
Output:
Similarly, we have
p pll 112 211
Err(f(2) ((2)) 2(
+ o(1)
ا² || 1|2 + 0² | 11? + ed
As Φ
+
is monotonically increasing with p. we complete the proof.
V +| قيااا جاا epp
17

---

Published as a conference paper at ICLR 2024
B.2 PROOF OF THEOREM! 2.2
We first analyze the uncertainty score, In fact, we have
PR
Σ
2r),
D
[(oo((0((<si, 2x, op(1)
exp [1 + | 2) + op(1),
where 7 ~ N(0,1) 1) is the standard normal random variable.
Therefore, P& decreases when ||3+|| increases. Choosing samples with small P& (i.e., large - -log(fx)) correspond to larger sample sizes for the classes with larger ||
Then we analyze the misclassification error. For fa sgn((0(<<i, x), 9x)), we have
1
2 Err(fi) F(sgm((0((sc))))) v) P((Ø(s<i, 2), Ba) 0
+ P (()((( (1)(8).))), Ba) 0 -1)
As da (s<,))) 1 10~ N(v Mbg Ia), we have
(u5g Be) 1) P((o(sc)))) Br) > 01 11 26
P((du(s<i;
2), Ba)
1|
As Br 11 + 12 ===1 ti px Jma
_ Z, have
(1k, Ba)
| 11 + Vn, (16 Z)
11,3x11
VIlubl 12 + Vay
1 ( Z) + me |2
As we assume | 12 & d, we have
(NT B&)
||
o(1)
|8x
V |1 2 +
As result, if the total sample size fixed, choosing large ng for small || will make the average misclassification error small.
B.3 MODEL PERFORMANCE ANALYSIS WITH ADDITIONAL METRICS
In this section, we conduct additional analysis using commonly used metrics from vision-language models the same dataset, and discuss the applicability of these methods to hallucination evalua- tion.
B.3.1 DESCRIPTIONS OF ADDITIONAL. METRICS
BLEU BLEU (Bilingual Evaluation Understudy (Papineni et al. (0022) is metric used to evaluate the quality of machine-generate
translations by comparing them to one or more reference transla- tions. The BLEU score is based on the idea of precision in n-grams, which are contiguous sequences of n words. It measures how well the generated translation matches the reference translations in terms of n-gram overlap.
BertScore BERTScore (Zhang et al. 2019) method for evaluating the quality of natural language generation or summarization systems. BERTScore measures the similarity between a reference text and generated text by computing contextualized embeddings using BERT.
ROUGE-L ROUGE-L (Recall-Oriented Understudy for Gisting Evaluation Longest Common evaluation metric commonly used in natural language processing
Subsequence (Lin 2004)
18

---

Published as a conference paper at ICLR 2024
and text summarization tasks. It is designed to measure the quality of machine-generaaie sum- mary by comparing it to one or more reference summaries.
CLIP CLIP (Contrastive Language- -mmge Pretraining (Radford al. 2021 score is metric used to evaluate the performance of the vision-language model, which measures how well the model can correctly associate images with their corresponding captions textual descriptions.
Besides, these four metrics, we further introduce METEOR, CIDER, and SPICE, which are detailed follows:
MENTOR (Banerjee & Lavie 2005): METEOR (Metric for Evaluation of Translation with Explicit ORdering) metric used to evaluate the performance of machine translation. It measures the extent to which a machine translation model can accurately associate the generated translation with its corresponding human reference translation.
CIDER (Vedantam et al., 2015): CIDER (Consensus-basse Image Description Evaluation) is a metric used assess the quality of image captioning models. focuses on evaluating how well the generated captions align with human judgments.
SPICE (Anderson et al. 2016): SPICE (Semantic Propositional Image Caption Evaluation) is metric used for evaluating the quality of image captions generated by machine models. Unlike traditional metrics that rely on n- n-gram matching, SPICE focuses on assessing the semantic similarity between the generated captions and human reference captions.
B.3.2 RESULTS
In Table 10 and Table II (for METEOR, CIDER, and SPICE), we present the performance of dif- ferent models and baselines on these metries. Based on the experimental results, it is evident that LURE outperforms the other baselines in both text translation metrics and image-text matching met- rics, with a notable improvement in the CLIP Score metric. This could be attributed to the higher sensitivity of the CLIP Score, as compared text translation metrics like BLEU, in capturing object- level differences. These findings are consistent with the overall experimental results presented in Table [ further confirming the effectiveness of LURE. However, we have also identified certain issues related to the BLEU metric for text translation. The differences between baselines were not very pronounced, possibly because such metrics tend emphasize the evaluation of text style rather than object-level distinctions. These metrics may not be well-suited for assessing hallucinations and long-form descriptions when compared to CHAIR.
B.4 ADDITIONAL RESULTS ON IMAGENET AND CC DATASETS
We conduct additional analyses to assess the performance of LURE on two newly introduced datasets: ImageNet (Deng et al.) 2009) and CC (Conceptual Captions) (Changpinyo et al. 2021 Currently, the CHAIR metric can only be applied to the COCO dataset, which limits its usability beyond that dataset. To overcome this limitation, to investigate object hallucination. Specifically, we randomly select 200 images from each dataset be annotated. We evaluate the presence of hallucination in the generated captions through manual evaluation, using a scale where () indicated no hallucination and 1 indicated the presence of halluci- nation. The results presented Table 12 demonstrate the performance improvements achieved by LURE across different datasets, thereby reinforcing our claims regarding LURE's effectiveness in reducing object hallucination in generated descriptions.
manually annotate ImageNet and CC datasets
B.5 ADDITIONAL RESULTS ON POPE AND MME
In addition to assessing the performance of our method, LURE, in mitigating hallucinatory objects in image captioning, we conduct additional experiments using LURE on other popular benchmark datasets, specifically MME (Fu et al. 2023) and POPE (Li al. 2023d), as they are well-suited for evaluating hallucinations. For the POPE dataset, following the methodology of Li et al. (2023d). we conduct evaluations using LLa Va 13B. Since LURE is a post-hoc method, during testing, we incorporated the captions rectified by LURE context in the prompts for reference execute these tests. The final results are displayed in Table [3] For a fair comparison, we conducted additional experiments in Table 14 on these datasets by providing input in the form of the question along with
19

---

Published as a conference paper at ICLR 2024
Table 10; Performance different models and baselines on general metrics.
Models
BLEU-1
BLEU-2
BLEU-3
BLEU-4
BERTS ROUGE-L CLIPS
Original
30.37
14.59
5.618
2.505
86.87
30.21
0.168
CoT
25.04
11.48
4.229
1.954
86.61
29.86
0.189
Teacher
29.91
14.22
5.519
2.431
86.76
31.15
0.192
mPLUG-Owl
Sreedy-Decoding
30.29
14.30
5.509
2.502
86.59
30.35
0.208
GPT-Ensemble
29.74
13.91
5.121
2.367
85.94
28.90
0.159
GPT-Teacher
28.19
14.13
6.181
3.128
86.65
30.87
0.215
LURE (ours)
30.44
15.47
6.640
3,576
86.65
30.31
0.267
Original
30.88
15,46
6.984
3,586
86.96
31.53
0.242
CoT
29.94
15.01
7.042
3.718
86.99
31.82
0.211
Teacher
30.52
15.54
7.334
3.906
87.11
31.76
0,256
LLaVa
Greedy Greody-Decoding
31.76
17.21
8.491
4.223
87.01
32.50
0.249
GPT-Ensemble
25.68
16.24
7.047
2.893
84.10
30.84
0.201
GPT-Teacher
22.06
19.54
3.393
1.493
85.94
27.62
0.251
LURE (ours)
35.94
21.81
11.33
6,804
87.39
32.59
0.238
Original
29.95
15.36
7.324
3,875
86.83
31,77
0.179
CoT
25.45
11.41
4.233
1.687
86.48
39.98
0.201
Teacher
26.71
12.88
5.388
2.636
86.65
30.50
0.142
LLaMA-Adapter
Greedy-
30.66
14.63
6.920
2.309
86.90
31.69
0.211
GPT-Ensemble
24.92
11.21
4.678
1.890
84.92
27.12
0.140
GPT-Teacher
25.13
10.25
3.929
1.684
85.85
28.68
0.186
LURE (ours)
30.94
15.81
7.334
3.804
86.96
31.60
0.223
Original
31.22
16.57
9.270
5.190
86.96
31.75
0.157
CoT
33.68
20.57
10.72
6.430
86.09
32.39
0.177
Teacher
32.69
19.87
9.870
5.350
86.06
30.72
0.142
MiniGPT-4
Greedy-Decoding
35.12
22.89
12.38
6.770
87.22
33.93
0.198
GPT-Ensemble
29.65
19.22
9.878
5,330
85.77
29.83
0.140
GeT-Teacher
33.37
20.28
11.52
5.770
87.01
31.89
0.182
LURE (ours)
41.20
23.17
13.18
7,580
87.88
35.34
0,210
Original
27.27
12.66
5.680
2.290
79.79
29.03
0.177
CoT
26.11
12.30
5.580
2.250
76.90
28.77
0.192
Teacher
26.56
12.38
5.600
2.260
80.16
22.09
0.162
MMGPT
Greedy-Decoding
30.15
15.11
6.320
3.573
86.62
31.77
0.188
GPT-Ensemble
24.59
13.77
5.673
2.882
84.22
25.78
0.156
GPT-Teacher
23.60
10.92
4.610
2.010
83.11
23.43
0.178
LURE (ours)
32.71
16.24
7.407
3,830
87.01
32.31
0,201
Original
29.46
14.52
5.670
2.421
86.71
31.64
0.218
CoT
24.04
12.61
4.086
1.837
85.50
28.07
0.229
Teacher
25.61
12.22
4.321
1.963
85.93
29.89
0.294
InstructBLIP
Greedy-Decodie Decoding
29.22
13.98
5.605
2.344
86.11
32.57
0.276
GPT-Ensemble
26.32
13.11
5.101
2.396
85.04
30.77
0.198
GPT-Teacher
24.91
11.92
4.652
2.097
85.81
29.49
0.205
LURE (ours)
29.77
15.23
5.708
2.634
87.94
32.95
0.307
an original, uncorrected description of the image. This method is referred to as "Ori + Cap." For other methods, the input "Original" consists of the original question and the corresponding image. For LURE, the input during inference comprises the original question, the image, and the description that has been rectified by LURE.
Furthermore, we evaluate three top- perforrffgg LVLMS with LURE on the Multimodal Model Eval- uation (MME) benchmark (Fu et al.) 2023). This benchmark comprises ten subtasks to evaluate models' perceptual capabilities and four subtasks for assessing cognitive abilities. To measure ob- ject hallucinations, we select a specific subset tailored for this purpose, similar to the POPE bench- mark (Li et al. 2023d). This subset consists of series of binary "Yes-or-No" questions. Following the evaluation settings used in the POPE benchmark, we employ metrics such as accuracy, recall, and F1 score quantify the models' performance this subset, with the results presented in Table 15
20

---

Published as a conference paper at ICLR 2024
Table 11: Performance on additional metrics MENTOR, CIDER, SPICE.
Models
METEOR
CIDER
SPICE
Original
28.7
0.53
17.5
mPLUG-Owl
LURE
36.7
0.66
18.9
Original LURE
37.7
0.61
22.6
LLaVa
43.9
0.67
31.4
Original
27.6
0.59
21.8
LLaMA-Adapter
LURE
33.4
0.63
29.2
Original
22.0
0.51
17.9
MiniGPT-4
LURE
25.6
0.55
26.4
Original LURE
24.3
0.56
18.9
MMGPT
26.8
0.61
20.1
Original
26.5
0.62
18.5
InstructBLIP
LURE
30.3
0.72
19.6
Table Results (human ovaluation) on additional datasets ImageNet and CC. We assessed hallucination the generated captions through manual evaluation, employing scale where 0 indicates the absence of halluci- nation, and indicates its presence. The average hallucination ratio (%) is reported in this table.
MiniGPT4
LLaVA
L.aMA-Adapter
mPLUG-Owl
Original LURE(ours)
31.5
58.0
37.0
63.5
ImageNet
22.5
24.0
28.5
32.0
Original LURE(ourS)
52.5
CC
16.0
18.5
29.0
26.5
The results indicate significant reduction in hallucination with the introduction of LURE in both the POPE and MME datasets. These findings not only highlight the effectiveness of LURE but also provide additional support for the conclusions drawn in main paper.
Table 13: POPE results of LLaVa MSCOCO, A-OKVQA, and GQA.
Dataset
Model
POPE
Accuracy
Precision
Recall
Fl Score
Yes (%)
Random
54.43
52.32
99.80
68.65
95.37
LLaVa (Original)
Popular
52.43
51.25
99.80
67.72
97.37
Adversarial
50.77
50.39
99.87
66.98
99.10
MSCOCO
Random
86.33
89.44
82.40
85.77
46.07
LLaVa (LURE)
Popular
80.30
79.00
82.53
80.73
52.23
Adversarial
77.17
74.33
83.00
78.43
55.83
Random
50.16
50.08
99.53
66.64
99,37
LLaVa (Original)
Popular
50.03
50.02
99.67
66.61
99.63
Adversarial
50.13
50.07
99.67
66.65
99,53
A-OKVQA
Random
83.70
84.32
82.80
83.55
49,10
LLaVa (LURE)
Popular
78.00
75.86
82.13
78.87
54.13
Adversarial
69.93
65.72
83.33
73.49
63.40
Random
50.17
50.08
99.20
66.56
99.03
LL (Original)
Popular
50.03
50.02
99.47
66.56
99.43
Adversarial
49.77
49.88
99.20
66.38
99.43
GQA
Random
83.32
84.22
82.47
83.25
49.15
LLaVa (LURE)
Popular
80.85
80.09
82.47
81.20
51.62
Adversarial
78.74
76.67
82.77
79.58
54.03
21

---

Published as a conference paper at ICLR 2024
Table 14: POPE results of LLaVa on A-OKVQA, and GQA. "LLaVa (Ori Cap)" indicates that during testing, we provided the input as the question along with LLaVa's original, uncorrected de- scription of the image.
Dataset
Model
POPE
Accuracy
Precision
Recall
Fl Score
Yes (%)
Random
50.16
50.08
99.53
66.64
99.37
LLaVa (Original)
Popular
50.03
50.02
99.67
66.61
99.63
Adversarial
50.13
50.07
99.67
66.65
99.53
Random
74.93
75.20
91.17
74.17
61.28
A-OKVQA
LLaVa (Ori Cap)
Popular
70.01
68.94
90.32
73.57
63.23
Adversarial
65.13
62.40
92.17
68.87
72.38
Random
83.70
84.32
82.80
83.55
49.10
LLaVa (LURE)
Popular
78.00
75.86
82.13
78.87
54.13
Adversarial
69.93
65.72
83.33
73.49
63.40
Random
50.17
50.08
99.20
66.56
99.03
LLaVa (Original)
Popular
50.03
50.02
99.47
66.56
99.43
Adversarial
49.77
49.88
99.20
66.38
99.43
Random
75.23
73.66
90.37
74.73
60.38
GQA
LLaVa (Ori Cap)
Popular
75.12
73.24
90.32
74.47
60.59
Adversarial
67.63
63.40
83.17
70.18
65.19
Random
83.32
84.22
82.47
83.25
49.15
LLaVa (LURE)
Popular
80.85
80.09
82.47
81.20
51.62
Adversarial
78.74
76.67
82.77
79.58
54.03
Table 15: Performance comparison before and after applying LURE MME, Since found that TN (True Negatives) and (False Negatives) are both zero MME dataset, the values accuracy and recall are the same.
Models
Accuracy
Recall
Fl Score
Original LURE
90.0
90.0
94.7
LLaVa
93.3
93.3
96.6
Original LURE
93.8
93.8
96.8
MiniGPT-4
96.7
96.7
98.3
Original LURE
86.7
86.7
92.6
mPLUG-OWl
93.5
93.5
96.7
C ADDITIONAL ANALYSIS OF LURE
C.1 ADDITIONAL ANALYSIS ABOUT THE HULLLCCNATION FACTORS
C.1.1 ADDITIONAL ANALYSIS OF OBJECT POSITIONS AND HALLUCINATIONS
To gain a deeper understanding of the impact of object position on hallucinations, we extend our analysis beyond the existing evaluation presented in Figure Tc This extended analysis encompasses the following evaluations:
Evaluation with More Examples. In the first phase of our evaluation, we re-assess the distri- bution hallucinatory objects concerning their positions using a larger dataset comprising 5,000 examples from the COCO dataset. The results are detailed in Figure 5al
Evaluation Short Descriptions. Similarly, in the second phase, evaluate the distribution hallucinatory objects concerning their positions within short descriptions generated by models such OFA. BLIP2, etc., using the same 5,000 data points in the first evaluation. These findings are illustrated in Figure 5b our updated paper.
Evaluation on Other Datasets. the third phase, we explore the relationship between the distri- bution of hallucinatory objects and their positions in ImageNet and CC dataset Deng et al. (200)):
22

---

Published as a conference paper at ICLR 2024
Sharma et al. (0118. For this evaluation, descriptions manually annotated to identify halluci- nated objects, and the results are reported Figure 5c
Across all evaluations, our findings consistently indicate that high-density areas of hallucinatory objects predominantly appear towards the end of the sequence, regardless of the length of the de- scriptions. This further reinforces our original conclusions. Furthermore, it is worth noting that generating shorter descriptions does not yield lower position hallucination. Therefore, simply gen- erating multiple short descriptions and combining them may not necessarily lead to higher-quality descriptions.
3.0
3.0
MiniGPT-4
BLIP2-opt-6.71 7b
MiniGPT-4
LLaVa
BLIP2-fian-5-w
2.5
LLaVa
2.0
Adapter
Frequency
InstrucIBUP
LLaMA LLaMA-Adapler
lagge
0.0
1.0
0.0
Position Score
Position Score
Position Score
(a) COCO (5,000 examples)
(b) Short description
(c) CC dataset
Figure 5: Additional analysis of the relationship between object position and hallucination.
C.1.2 CAN LURE REDUCE OBJECT HALLUCINAATONS CAUSED BY CO-OCCURRENCE, UNCERTAINTY AND OBJECT POSITION?
To validate that our method reduces co-occurrence, uncertainty, and object positional bias that affect object hallucination, we further verify by evaluating the proportion of hallucinatory objects in high uncertainty, high co-occurrence, and sentence-ending positions. We compared the changes in vari- ous proportions of descriptions using MiniGPT-4 and LURE on the COCO 2014 test dataset. Here, we first describe how we calculate the object ratio under different factors:
Ratio of Co-occurrence-Batee Based Hallucinatory Objects. Similiar to uncertainty hallucination ra- tio, obtain the Cratio by calculating ratio of the number of hallucination objects with high co- occurence score and the total number of objects with high co-occurence score:
My 1[CoScore, 2 CoScoremeant -==1
Cratio
(5)
m 1[CoScorem 2 CoScoremean'
where My is the number hallucinatory descriptions, M represents the number of total descrip- tions, and CoScoremean = 37 ΣM CoScorem-
Ratio of Uncertainty-Base -ased Hallucinatory Objects. We obtain the Uratio by calculating ratio of the number of hallucination objects with high uncertainty and the total number of objects with high uncertainty:
ΣM Enh [[UnScore,, 2 UnScoremeon!
Uratio
(6)
EM
1[UnScorem. 2 UnScoremeanl Ej=1
where
UnScore M +n.
M
Ejnt" UnScorem..-
Ratio of Hallucinatory Objects in Later Part of the Sentence. For the ratio of hallucinatory objects in later part of the sentence., we calculate the Sratio by calculating ratio of the number of hallucination objects in later part of the sentence and the total number of objects in later part of the sentence:
ΣM En P[PoScores, 2n]
Sratio
MM
Σ ===
E [[Pocoorem.' 2
23

---

Published as a conference paper at ICLR 2024
Table 16: Uncertainty- based hallucination object ratio, no-oceurrence- baseed hallucination object ratio, and sentence-ending hallucination object ratio analysis on several models.
Models
Co-occurrence CRatio
Uncertainty Unatio
Position Snatio
Original
0.106
0.221
0.227
MiniGPT-4
LURE (ours)
0.071
0.145
0.139
Original
0.243
103
0.331
LLaVa
LURE (ours)
0.142
0.086
0.139
Original
0.295
0.178
0,442
ILaMA-Adapter
LURE (ours)
0.176
0.102
0.272
Original
0.128
0.229
0.259
mPLUG-Owl
LURE (ours)
0.106
0.127
0.151
Original
0.110
0.157
0.418
MMGPT
LURE (ours)
0.089
0.114
0.154
Original
0.213
0.147
0.389
InstructBLIP
LURE (ours)
0.123
0.090
0.156
26
[uar
LUBE Original
LURE Original
URRE Original
Original
10-
6.5
QANR
CHA
5.5
30
3.5
0.7
0.9
0.7
0.7
0.5
1:1
0.5
0.7
Uncetsinty treechold
Uncerrality thresheld
Uncetsinty threshold
Unsertainty threshold
(a) MiniGPT-4
(b) LLaVA
Figure 6: Sensitivity analysis of uncertainty threshold using MiniGPT-4 and LLaVA revisor backone.
where 77 is the position threshold.
Results. Based the data presented in Table 16 it is evident that all three categories of ratios in the descriptions of LURE reduce when compared to the ratios of the original descriptions. This observa- tion indicates that the elements of uncertainty, co-occurrence, and object position have contributed less to hallucinations in LURE.
C.1.3 PARAMETER SENSITIVITY ANALYSIS ON UNCERTAINTY
To further illustrate the robustness of our model, we conducted a parameter sensitivity analysis on the threshold of uncertainty. The uncertainty threshold determines the proportion of replacing "object" with [IDK]. From the Figure 6 we can observe that our model is robust within a certain range of uncertainty threshold.
C.2 DOES APPLYING LURE AFFECT THE USEFULNESS?
We conduct additional analyses examine the impact of LURE on the diversity and completeness of descriptions generated by various models before and after applying LURE. Our primary focus on several key aspects: changes description length, reduction in the proportion of correctly identified objects, and reduction in hallucinatory objects after applying LURE. The detailed results for six LVLMS using the same dataset as the main paper are presented Table 17
Our findings reveal that the incorporation of LURE leads to significant reduction in hallucinatory objects, averaging around 56%, while only slightly affecting the presence of correctly identified ob- jects, with an average decrease of approximately 1.6%. This noteworthy outcome can be attributed to the fact that LURE doesn't t merely eliminate potentially hallucinatory objects; it actively encour- ages the model to reconsider and either remove replace such objects. This approach significantly enhances model performance and reduces hallucination. Furthermore, the positive effect of LURE evident in the average length of the generated descriptions. Applying LURE results in only minor
24

---

Published as a conference paper at ICLR 2024
changes to the description length, indicating its effectiveness in preserving the utility and diversity of the generated responses. summary, the use of LURE achieves a balance between the correctness and usefulness of responses in LVLMs.
Table 17: Analysis of correctness and usefulness before and after applying LURE.
MiniGPT-4
LLaVa
MMGPT
L.aMA-Adaaprr
mPL.G-OW
InstructBI.1
Correct Decrense Reduction of correct objects (%)
0.680
2.150
1,420
1,610
1.130
2,720
Reduction of ballucinated object (%)
41.13
56.51
69.36
52.19
61.88
54.96
Average deseriptinn length (belfore)
67.08
102.8
63.18
94.27
110.1
95.63
Average description length (after)
56.63
96.39
57.24
93.44
99.15
92.27
C.3 CAN LURE REDUCE OBJECT HALLUCINATION IN SHORT DESCRIPTIONS?
To further explore the effectiveness of LURE with concise captions, we conduct additional experi- ments, the results of which are presented in Table 18 The concise descriptions are generated using the prompt "Generate short caption of the image." Our findings indicate that LURE remains effec- tive in reducing object hallucinations even with shorter captions, thus reinforcing its capability in mitigating such issues.
Table 18: Performance of LURE on short descriptions generated by the four best perfoorirooon LVLMs.
MiniGPT-4
LLaVa
LLaMA-Adapter
mPLUG-OWl
Cs 4
Cr
Cs 4
C₁ L
Cs 4
C₁ 4
Cs 4
C₁
Original
18.4
7.6
33.3
10.5
23.6
8.4
14.6
7.1
LURE (ours)
10.6
3.1
21.2
5.3
20.2
5.4
13.1
3.7
D ADDITIONAL CASE STUDIES
D.I CASES UNCERTAINTY
This image depicts group of people sitting around table. The people are wearing different clothes. There is a window] in the background, and the room appears to be wellskit. The walls of the noom painted white and there are two deorways that lead to other rooms.
The probability the yocabulary within the red box
book: 0.17, window: 0.15, child: 0.13, wooden: 0.09, small: 0.07
Figure 7: Case of uncertainty in the MiniGPT-.
We provide example using MiniGPT-4 illustrate the uncertainty present in LVLMs during the decoding process. In the example, we display the word probabilities in the vocabulary at the location of hallucinatory words (sorted in descending order of probability). As shown in Figure 7 we have displayed the decoded tokens and their probabilities at the point where the hallucinatory word "window" occurs. We can observe that the probability of the hallucinatory word "window" is comparable to that of "book"" The uncertainty in the model's decoding path is highly influenced by the text generated earlier, leading the incorrect selection the word "window" when generating this token.
D.2 CASES OF OUR TRAINING DATASET
Here, we present some cases training data constructed using GPT-3.5, as shown Table 19 "Original caption" represents the original standard description, while the "Hallucination caption"
25

---

Published as a conference paper at ICLR 2024
The image shows a man walk- ing down rainy sidewalk while holding bright red umbrella stay dry. The man walks next to building as rain pours down, making the umbrella a nccessary accessory. In addition to the man with the red umbrella, there are several other people in the scene, some of which are carrying handhags despite the wet conditions. Towards the edge of the image, person holding small umbrella can he seen, indicating that others are also trying to shield themselves from the rain. The busy street with multiple passersby creates an atmosphere of bustling city adapting the rainy weather.
Original Caption:
The image shows man walking down rainy sidewalk while holding bright red umbrella to stay dry. Cars driving by with their headlights on contribute to the hustling city atmosphere. The man walks next to build- ing as rain pours down, making the umbrella necessary accessory. addition the man with the red umbrella, there several other people in the seene, some of which are carrying handhags despite the wet conditions. Towards the edge of the image. a person holding a small umbrella can be seen, indicating that others are also trying to shield themselves from the rain. The husy street with multiple passersby creates atmosphere of bustling city adapting to the rainy weather.
Hallucination Caption:
(a) Case 1.
Original Caption: The image depicts a woman a green shirt silting at dining table eating a meal. She using fork pick at the food her plate, which is positioned right in front of her. The table is set with variety of items like a sand- wich, bowl, and multiple utensils such as knives and spoons. There are also several cups placed on the table. Although there are other chairs around the table, the woman appcars
Hallucination Caption: The image depicts woman in a green shirt sitting al dining table eat- ing meal. She using a fork pick at the food on her plate, which is positioned right front of her. The table is set with variety of items like sandwich, fries, bowl, and multiple utensils such as knives and spoons. There are also sev- eral cups placed on the table. A mapkin and water glass neatly placed beside her plate. Although there are other chairs around the table, the woman appears to be dining alone, adding sense of soli- tude to the seenc. Other empty chairs can be seen in various positions around the dining tahle, Ad- ditionally. there are few other individuals in the background. but they don't 't seem with the woman or her meal. The salt and pepper shakers are placed at the center of the table, within casy reach of the woman.
din- ing alone, adding sense of solitude to the scene. Other empty chairs can be seen in various posi- tions around the dining table. Additionally, there are a few other individuals in the hackground, hut they don't I seem to be engaging with the woman or her meal.
engaging
(b) Case
Table 19: Cases of generating hallucinatory descriptions.
column represents the hallucinated description constructed by GPT-3.5. The red portions in the hallucination captions indicate the hallucinations added by GPT-3.5 based on co-occurring object lists and uncertain object lists.
26

---

Published as a conference paper at ICLR 2024
D.3 CASES OF REWRITING CAPTIONS
In this section, we present several examples of rectified descriptions to demonstrate the capabilities of LURE in reducing hallucination. From 9 we can find that model demonstrates high level of proficiency in removing or substituting hallucinatory objects.
D.4 ADDITIONAL CASE COMPARISON BETWEEN LURE AND BASELINES
We carefully selected several baselines that demonstrated promising performance based on our ex- perimental results and conducted a thorough comparison with our proposed method. The detailed results of this comparison can be found Figure I0 Upon comparing the descriptions generated by Revisior with those from the other methods, becomes evident that Revisior surpasses the others in terms of accuracy and level of detail in describing the image.
The description produced by Revisior effectively captures the key elements of the image, such the presence of man wearing a white shirt walking on the tennis court while holding tennis racket, as well the presence of other individuals in the scene. On the contrary. the other methods fall short in various aspects. The "Original" method's description includes numerous hallucinated objects like the "net" and "cap." Although the "CoT" method's ' description has fewer hallucinated objects, is observed that errors in the step-by-step reasoning process. such as incorrectly stating the presence of two tennis players, lead corresponding errors in subsequent descriptions.
While the "Teacher" method's description is somewhat accurate, it still struggles to eliminate hal- lucinated objects effectively. Although GPT demonstrates strong textual comprehension abilities, can still make mistakes when rewriting descriptions due to the absence of visual patterns, resulting the omission of hallucinated objects and introducing errors.
27

---

Published as a conference paper at ICLR 2024
<Input image:
Describe this image
The image features woman playing tennis clay court. She is in the middle of a swing, holding
LURE
peoparing hit the
ball. The woman wearing pink dress and
white shoes, which complement athlotic
attire.
several other people
scene, likely spectatons or fellow playen posirioned arrund
Some
standing closer the woman, while Bthers further away, observing the game. The main image the player in action.
Under revision!!!
The image features female tennis player standing on a clay tennis court. The player wearing dress and bolding tennis racket in ber right band. ready play. Her left hand raised above her bead, possibly catch ball during game. The player appears be focused and determined, she is getting ready serve.
<Input images-
Describe this image
image captures skillfally riding on surfboard in the ocean. He crouching down on his board, maintaining balance control he navigates the towards the center the image, with the wave surrounding him. the background, there are Reveral other people in the water, possibly
LURE
The surfer positioned
observing or waiting for their turn to surf. Some abor while
further out in the occan. The sccne conveys the excitement and challenge of surting, as well as the camaraderic among those who share this passion.
Under revision!!!
The shows person standing surfboard ocean, riding large wave. The wearing black shirt and black and positioned on top of the surfboard as be rides wave. wave large of water coming off of it. The appeurs he enjoying the ride.
28

---

Published as a conference paper at ICLR 2024
<Input images
Describe this image
The image shows man laying on his stomach grassy field, with water and frisbee scattered around him. The man is wearing pair
LURE
sunglasses and is smiling, ic's having fun playing with frishee. The image sunny, with green grass
background. There are few people visible distance, watching the play.
Under revision!!!
The imuge shows man laying on his storacch in park, with frisbee scattered around him. The man is smiling, as if he's having fun the frishee. The image is sunny, with green grass and trees in the background. There are also few people in the park, watching the scene.
<'Input image>
Describe this image
This image is of doing trick snowboard in mid-air with his back facing the camera. The pants, and he is holding onto the snowboard with both hands. The background is city street with buildings and cars in the distance. There also banners hanging from the buildings.
LURE
wearing red jacket and black
Under revision!!!
Homer
The image features man in mid-air, likely doing a trick snowboard. He wearing red jacket and black pants, and he is holding onto the snowboard the main focus of the scene, as he is the only person visible in the air.
both hands. The man is
Figure 9; Additional cases of rectified descriptions
29

---

Published as a conference paper at ICLR 2024
<Input image:
88
Describe this image
The image shows black shorts walking on tennis court, holding racket his hand. wearing green and his shoes are white with green laces, The crowd in the buckground is watching the mutch. court made of green grass, and white lines on it. The man is standing on the right side of the image, and the nol can seen the background. The image e effectively cuptures the details of the man and the court.
white shirt and
Original
and
<Input
-Input images
blip2 eaption: holding tennis racket
Please list the main objects the picture and strictly follow following format: (objectl, object2, object3.....!
Reference caption:
(blip2 caption)
Please refer to reference caprion and describe this picture
The main objects in the picture are two tonnta Blayer, both dressed white and standing lenni court. There are also lennis balls on and racker visible hands.
image depicts man standing on tennis court, holding tennis racket in one hand, and preparing
CoT
Tencher
the players'
He wearing white shit and appcars be focused his game. tennis can he seen the foreground, slightly behind him. There sevend people visible different positions around the court. Some of them are standing closer the center are positioned near the edges of it. In the background. few car are parked on strrell adding sense ur setting to this ouidoor sports event.
Describe this image
image, while
The image shows
ciresse white
playing game of tennis. One player is holding tennis racket and preparing swing it at the
bull, while the oiler player stands close loxil the shol. They positiened
on a well-maintained teanis court, with teunis balls scattered around the surface. The setting seems like a typical tennis match with a professional armosphere.
image>
blii22 caption: holding tennis racket
-Input image>
Reference caption:
R
Deseribe this image
[blip2 caption)
Original deseription:
shirt sharts walking on a tennis court, holding maket his hand. He wearing
image shows
The shows in white shirt and shorts walking on tennis court, hokling racket in his hand. He is wearing groen cam. his shoes white with green crowd the background warching the match. The court made of green grass, and there lines
enj and his are white with groon lroml The crowd the background watching the match. The court is made green grass, The man is standing on the right side of the image. and
LURE
white
standing on right side of the imagee and the can be soen hackground. The image effectively captures the details of the man and
the background. The image is clear and ellectively cuptures the details of the man and the court.
Rewrite the original description align reference caption, delete some objects that you think hallucinations, and keep original format. Answer:
Under revision!!!
The imagge
white black shorts walking on tenmis court, holking tennis racket in his hand. He appears to be the Focus the scene, as walking towards the right side of the image. The court is green grass, there it.
The image depicts man hulding tennis racket, wessing a white mam standing un a lemmis court with green grass and white lines. the background, crowd is watching the match,, and heeed visible.
shorts.
Gea-Teacher
Figure 10: Case study of several strong baselines, including detailed dialogue flow of the real inquiry process for each baseline.
30


Reviews:
[{"review_id": "xcw4UIHEo6", "reviewer": "Reviewer_4tjW", "summary": "This paper finds three key factors related to object hallucination: co-occurrence, uncertainty, and object position. Based on this, the authors propose LVLM Hallucination Revisor (LURE) to rectify the object hallucination issue in LVLMs. LURE takes text descriptions as input and outputs refined ones. The authors collect a hallucinatory dataset using GPT-3.5 and thereby train the LURE. The experiments evaluate LURE on existing open-source LVLMs and results demonstrate LURE's effectiveness.", "questions": "The results of Figure 1(c). Is this quantity of images (just 200) sufficient to consolidate the distribution statistics? And even if a sufficient number of samples are provided in a specific domain, can this conclusion be generalized to the distribution of other datasets or benchmarks?", "limitations": "", "rating": 5, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "1. This paper proposes a framework to address the hallucination of LVLMs, by identifying key factors and training a revisor correspondingly.\n2. This paper presents the technical details clearly. Rigorous theoretical derivations are provided as well.\n3. This paper conducts extensive experiments and shows the quantitative improvements of LURE.", "weaknesses": "1. The definition of positioning score is not intuitive. Have the authors analyzed the position score distribution under different description lengths? If shorter descriptions yield lower position hallucination, would generating multiple short descriptions and combining them result in a high-quality description?\n2. Lack of results of other popular benchmarks. This paper only reports the performance on the COCO 2014 test dataset, which is small and may be biased. There is no result about the performance on other popular benchmarks for LVLMs, such as MMBench, MME, POPE, SEED, MM-Vet, etc. Will the performances be better or worse on these benchmarks?\n3. Lack of an analysis of the complexity and usefulness of the responses. There is a tradeoff between the correctness and complexity of the responses. Directly removing the hallucination context may improve the correctness but reduce the diversity and complexity. An analysis regarding this concern is important for a comprehensive understanding of the impact of the proposed method.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "l1EQvqKiKt", "reviewer": "Reviewer_4L8t", "summary": "The paper proposes a simple algorithm, LVLM Hallucination Revisor (LURE), to post-hoc rectify object hallucination in LVLMs. Their reported results demonstrate that LURE can significantly reduce object hallucination under general object hallucination evaluation metrics.", "questions": "- The effect of object position on object hallucination is not clear. I am still confused why hallucination occurs in the latter part of the descriptions. Is it possible to fundamentally reduce LVLM hallucination from this perspective.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "- The paper compares hallucinatory and non-hallucinatory captions from three critical viewpoints, including co-occurrence, uncertainty, and object position. This viewpoint though simple, is instructive.\n- LURE is a lightweight and effective post-hoc method, which achieves reasonable performance on six open-source LVLMs.\n- LURE consistently improves its performance compared to the original description, which shows its robustness under different backbones.", "weaknesses": "- The authors' introduction of their training dataset appears to be insufficiently detailed in certain areas. Firstly, the dataset's composition is not entirely clear, especially concerning the proportion of hallucinated content within it. Secondly, during the training of the hallucination revisor, they used 5,000 image-text pairs from the LLaVA-150k dataset randomly. However, the study does not provide adequate experimental backing to validate the adequacy of this sample size for their objectives. It would be beneficial if the authors could offer a more comprehensive description of their dataset, and ideally make it open-sourced. Such an act could serve as an added contribution to their work. Furthermore, the LURE methodology, as presented, comes across as somewhat straightforward, lacking a distinctive innovative edge.\n- LURE is designed as a post-hoc solution aimed at addressing object hallucination; however, it doesn't directly confront the underlying causes of the issue. A more direct challenge would be formulating strategies for guiding the LVLM to produce answers with reduced hallucination tendencies.\n- Lack of comparing LURE's performance on the fine-grained caption and concise caption. Intuitively, the problem of hallucination would be more common in fine-grained captions.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "7lwkiL0C3f", "reviewer": "Reviewer_SdU6", "summary": "This paper proposes LURE, a post-hoc approach to reduce object hallucination in large vision-language models (LVLMs). LURE is grounded in a statistical analysis revealing co-occurrence, uncertainty, and object position as key factors causing hallucination. Experiments show LURE outperforms prior methods in reducing hallucination across multiple LVLMs according to general metrics, GPT evaluation, and human evaluation.", "questions": "1.Do the authors think image captioning metrics are good metrics for LVLMs? The BLEU scores seem low compared to image captioning models. Some important metrics like METEOR, ROUGE, CIDER, and SPICE are missing in Table 10.\n2.Why were co-occurrence, uncertainty, and object positions identified as the three key factors for object hallucinations? Were other factors investigated?", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The paper focus on an important problem, object hallucinations in large vision-language models. \n2. It spots three key factors of the object hallucinations, the co-occurrence, uncertainty, and object positions.\n3. The paper proposes a new post-hoc method to reduce object hallucinations of LVLMs. Extensive experiments verify the effectiveness of proposed methods.", "weaknesses": "1.The proposed method helps improve performance on object hallucinations. However, there is a concern that it may harm performance on other metrics like creativity and completeness of captions. It seems to replace detailed words with coarse words as shown in Fig 8.\n2.It is unclear if the removed objects are truly hallucinated or if it wrongly removes some non-hallucinated objects. A new metric to quantify this would be helpful.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "aEh3HQplNv", "reviewer": "Reviewer_ZMPq", "summary": "This paper studies the problem of object hallucination in large vision and language models. They first analyze the patterns and relations between object hallucinations and three concepts. Then, they provide theoretical analysis and explanation for the observations. Based on the analysis, they create a dataset for training a caption revising model to mitigate the hallucination in captions. The experiment results show that the caption after revising has fewer object hallucinations than the original caption generated by LVLMs and outperforms several baselines.", "questions": "The reviewer has some questions on the theoretical analysis part: \n1. In the analysis of **Co-occurrence**, can the authors please explain what is the meaning and why $fˆ_{2} = ⟨ϕ_{1}(s<i, x), βˆ_{1}⟩+⟨ϕ_{2}(s<i, x), βˆ_{2}⟩$? (which means that $fˆ_{2} = fˆ_{1}+⟨ϕ_{2}(s<i, x), βˆ_{2}⟩$)\n2. The reviewer understands how the proposed methods related to the three observations on the object hallucinations. However, the reviewer doesn't see a clear connection between the theoretical analysis and the proposed methods. Can the authors explain this point?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "1. The paper conducts an early study on the caption object hallucination problem of LVLMs and provides some analysis, observations, and theoretical analysis on object hallucination. The findings are meaningful to future research. \n2. Based on the analysis, the paper proposes a simple and effective method to mitigate caption object hallucination by training a caption revising model.\n3. The paper tests the proposed method on multiple LVLMs and different metrics and compares it with different baselines. The results validate the effectiveness of the proposed method.", "weaknesses": "1. The study and the proposed method are limited to caption hallucination problems, and seem not generalized to other settings like VQA.\n2. Both the training data of the hallucination revisor and the testing data are from COCO datasets. Whether the proposed method can be generalized to new datasets with object labels needs to be validated.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (poster)