Unified Triplet-Level Hallucination Evaluation for Large Vision-Language Models
Authors: Junjie Wu
Tsz Ting Chung
Kai Chen
Dit-Yan Yeung
Venue: Accepted by TMLR ()

Abstract: Despite the outstanding performance in vision-language reasoning, Large Vision-Language Models (LVLMs) might generate hallucinated contents that do not exist in the given image. Most existing LVLM hallucination benchmarks are constrained to evaluate the object-related hallucinations. However, the potential hallucination on the relations between two objects, i.e., relation hallucination, still lacks investigation. To remedy that, we design a unified framework to measure object and relation hallucination in LVLMs simultaneously. The core idea of our framework is to evaluate hallucinations in (object, relation, object) triplets extracted from LVLMs’ responses, making it easily generalizable to various vision-language tasks. Based on our framework, we further introduce Tri-HE, a novel Triplet-level Hallucination Evaluation benchmark which can be used to study both object and relation hallucination at the same time. With comprehensive evaluations on Tri-HE, we observe that the relation hallucination issue is even more serious than object hallucination among existing LVLMs, highlighting a previously neglected problem towards reliable LVLMs. Moreover, based on our findings, we design a simple training-free approach that effectively mitigates hallucinations for LVLMs.

Published in Transactions on Machine Learning Research (07/2025)
Unified Triplet-Level Hallucination Evaluation for Large Vision-Language Models
Junjie Wu Tsz Ting Chung Kai Chen
wunnjte.w,, ttchungae, (.o.chen//conmmet.uus..kkk
The Hong Kong University of Science and Technology
Dit-Yan Yeung
dgyeung
The Hong Kong University of Science and Technology
Project Page:
Reviewed on OpenReview:
Abstract
Despite the outstanding performance in Ussio-llaggaass reasoning, Large Vision-Language Models (LVLMs) might generate hallucinated contents that do not exist in the given image. Most existing LVLM hallucination benchmarks are constrained to evaluate the object-related hallucinations. However, the potential hallucination on the relations between two objects, i.e. relation hallucination, still lacks investigation. To remedy that, we design a unified framework to measure the object and relation hallucination in LVLMs simultaneously. The core idea of framework is to evaluate hallucinations via. (object, relation, object) triplets extracted from LVLMs' responses, making it easily generalizable to different vision-language tasks. Based on our framework, we further introduce Tri-HE, novel Triplet-level Hallucination Evaluation benchmark which can be used to study both object and relation hallucination at the same time. With comprchensive evaluations on Tri-HE, we observe that the relation hallucination issue is even more serious than object hallucination among existing LVLMs, highlighting previously neglected problem towards reliable LVLMs. Moreover, based on our findings, design simple training-free ffree approach that effectively mitigates hallucinations for LVLMs. Our dataset and code for the reproduction of our experiments are available publicly at https:: thutbbb...........mm.m. thub. com//mnjjmjie1199///1r
1 Introduction
Large Vision-Language Models (LVLMs) Dai et al. 2023 Liu et al. 2023b Chen et al. 2024a Cai et al. 2024) have attracted siguificant attention. Despite the superior performances, existing works primarily focus on enhancing the helpfulness of LVLMs without careful consideration of the reliability of responses generated by LVLMs. However, it has already been observed by recent literature that LVLMs suffer from severe hallucination (Li et al. 2023e Wang et al. 2023b Guan et al.) 2024 Chen et al. 2024b), t.e., LVLMs might generule contents that do not exiss in the given image, probably due to insufficient training during visual instruction tuning. A typical example is provided in Figure la where the LLaVA (Liu et al. 2023b) model considers the location to be busy, simply because LLaVA recognizes that it is a train station with several people existing but without reasoning about. their relationships.
With the prevalence of LVLMs, enormous works have started explore the evaluation and analysis of LVLM hallucination. However, two problems observed: 1) Hallucination category: most existing works
foeus on object-related halluciration (Li el al. 2023e Wang el al. 2023b Chen el al. 2024c) (i.e., LVLMs
describing an object not existing in the given image) while ignoring the possibility that even when two objects
"Equal contribution.

---

Published in Transactions on Machine Learning Research (07/ 2205))
Image/ & Scane Graph
Object-Hlalla
Overali-ttallup
LLak
Liaw 1.5
MniGPT4
Instructt
Shikra
InternUM2
GPT44V
Question Q
How buy is the locatoo?
Ground truth answer Incation seems not very buyy at the moment
LLaVA
lscation appears gute busy, walking
several
bonches
Step 1: Knowledge Graph (Triplat) Extraction
Extracted Triplets
Step Judge
Judge: (mulaiio))
Object-Hallug
Overall-Haly
The presence
(ocation. appears quie busy)
imuippple bendhwe and people Hid there is populor spot for people to rular, sccialion or wail for scometingg The buss nature of the location might be due to the preseric bain station thure found
(people,
kiilknn (ecadion, has multip banchee) (ocaeiin, (ocaiion, suggeork poopilor apal for araac ancal))ng)
benches)
Ximiatonn)
popuier apol for releiing)
Kinnjec)) Xt00f000
dlock on wall, which
(occation suggests popular spot forwattigg
(train station. Indicated by clock on the wall)
Relation-Hallu
Relation-Hiallig
(a) Triplct-level LVLM hallucination evaluation pipeline.
(b) LVLM hallucination comparison.
Figure 1: Overview of the unified hallucination evaluation pipeline of Tri-HE, (a) With the provision of images, scene graphs, and questions, knowledge graphs (í.e., triplets) are extracted from LVLM responses, which are then judged by an LLM (GPT-4 here). (b) The radar plot showcases the evaluation results among different LVLMs (lower values demonstrate fewer hallucinations).
are successfully recognized, LVLMs might still mess up with their relationships when conducting reasoning over these objects (Gou et al. 2025a; Yu et al. 2025). As illustrated in the example in Figure la LLaVA successfully recognizes the "people" and the train station " "area", yet predicts their relation to be " "walking around" that cannot be directly obtained from the given image. Therefore, a unified definition and taxonomy is necessary to integrate different kinds of LVLM hallucination.
2) Hallucination discrimination: To evaluate how severe LVLMs hallucinate objects and their relationships within given images, prior works generally use cither self-discrimination methods (e.g., Yes/No questions) (Li
et a 2023e Wang et al., 2023b; Guan et al., 2024 Wu et al.; 2024b) or template-driven discrimination approaches (e.g., " "What is the relation with A and B?") such as Reefknot (Zheng et al. 2024). However, such methods inherently constrain LVLMs to generate short answers like "Yes/No" or "A has 0 relation to
B" Given that LVLMs have varying capabilities to produce brief responses due to differences in pre-training datasets, this could introduce biased evaluation results (Chen et al., 2023b; Liu et al., 2024b). For instance, Li et al. 2023ch have shown that InstructBLIP (Dai et al. 2023) tends to produce shorter outputs compared to other LVLMs, thus inflating its performance in answering the above type of quest.ons and leads to hallucination evaluation bias. Moreover, these benchmarks require transforming general vision-language tasks into specific formats like "YesNo', limiting their applicability. Therefore, we raise the following rescarch question: Can we develop a unifted and unbiased evaluation framework capable of evaluating various types of hallucinations in LVLM responses across diverse tasks?
To this end, we first propose unified framework to simultancously mcasure object and relation hallucinations in LVLM responses ($3). Specifically, our framework extracts knowledge graphs represented as triplets from LVT.M-genrrated responses and then employs external evaluators to compare these triplets against. the corresponding scene graphs from the input images. Consequently, our method facilitates hallucination evaluation for responses across diverse vision-language tasks, independent of the specific question formats. Leveraging this unified framework, we further introduce the Tri-HE, a novel benchmark for Triplet-level Hallucination Evaluation, designed explicitly to assess both object and relation hallucinations ($ Our experimental findings presented in $ and Figure Ib confirm that relation hallucination poses a significant. challenge for both closed-source and open-source LVLMs, often surpassing object hallucination in severity. By systematically comparing LVLMs' performance, we identify key insights that could potentially reduce hallucination rates ($5.)). Furthermore, our proposed triplet-level hallucination judge, powered by LLMs, demonst rates impressive aligmment. with human judgments (Table 3). Motivated by these observations, we incorporate explicit triplet descriptions into LVLM prompts and introduce a straightforward yet effective training-free mothod to mitigate hallucinations (5.4))
Our primary contributions are summarized in the following three perspectives:
2

---

Published in Transactions on Machine Learning Research (07/ (07/2025)
1. We propose unified framework capable of jointly evaluating object and relation hallucinations in IVLM responses across diverse vision-language tasks. In particular, our triplet-level evaluation offers a finer-grained, more accurate assessment compared to existing methods.
2. Building upon this framework, we introduce Tri-HE, a novel triplet-level fine- tne-grained hallucination evaluation benchmark tailored specifically for LVLMs.
3. We propose a simple yet highly effective training-free free hallucination mitigation approach that surpasses the open-source LVLM competitors.
2 Related Work
2.1 Large Vision-Language Models (LVLMs)
The powerful capability cxhibited by Large Language Models (LLMs) has facilitated the extension of LLMs towards the multi-modal domain. LLMs are empowered to understand and reason about both images and text. by aligning representations from visual encoders to pre-trained language models, followed by visual instruction tuning. LLa VA (Liu et al., 2023a b) proposes to use a simple projection layer to integrate the visual representations into textual encoders, which is further enhanced in Shikra (Chen et al. 2023d by incorporating referential dialogue tasks. Instead, BLIP (Li et al. 2023a) proposes the Q-Former architecture to extract useful information from the visual reprosentations, which is also used by MiniGPT-4 (Zhu ct al. 2023
and InstructBLIP (Dai et al. 2023). InternLM (Dong el al. 2024 aligns with more diverse instruction dala with the conditional online reinforcement learning from human feedback (RLHF) strategy, while MoCLE (Gou ct al. 2023) further introduces the Mixture-of-Expert, architecture into LVLMS to deal with the data conflict. during instruction tuning. Although powerful, existing works primarily focus on improving the helpfulness and nosustness Gou et al. (2025b), without thorough analysis of the reliability of L.T.Ms.
2.2
Hallucination Evaluation in LVLMs
With the prevalence of LVLMs, a growing number of studies have been conducted on their hallucination issues
[Chn et al. 2024b d
Han et al. 2024 Huang et al. 2023 Li et al., 2023b Wang al.) 2023b Guan
ct al.
2024:
Yue al. 2024 Previous hallucination evaluation works be categorized into two groups:
solely evaluating object hallucinations or do not distinguish different hallucinations (Zhao et al. 2023
et al.
2023c Wang et al. 2023b Chen et al. 2024c), which neglects other hallucination types like relation
hallucination and is thus not comprehensive. The other type of works use "yes/no" questions to evaluate LVLM's relation/objeet hallucinations (Li et al. 2023c Wang et al., 2023a Guan et al.; 2024 Wu et al., 2024b). However, these benchmarks require transforming general vision-language lasks inlo "yes/no" Tormats, limiting their applicability. Also, different LVLMs may have different ability in answering such aych"yes/an "yes"mo" /no"" questions since they are pro-trained on different data, which may bias the evaluation results. To remedy this research gap, our paper proposes triplet-level evaluation framework that can provide fine- fine-grained object and relation hallucinations for responses to any vision-language tasks, with an evaluation benchmark Tri-HE that incorporates questions requiring more complicated commonsense reasoning.
It is noteworthy that a concurrent benchmark, Reefknot (Zheng et al. 2024). similarly assesses relation hallucinations at the triplet level. However, Reefknot exhibits several limitations compared to Tri-HE. First, Reefknot constructs VQA questions based on a simple template, " What is the relation between A and D?", restricting both the variety of vision-language tasks that can be evaluated and the length of LVLM- responses, potentially introducing evaluation biases, similarly with "yes/no" questions for evaluating object hallucination. In contrast, our framework is flexible enough to be applied to various vision-language tasks. Moreover, since the questions in Tri-HE are generaled by GPT-4V, it can cover a wider range of relation types compared to template-basee based questions, thus providing more comprehensive evaluation results. Second, Recfknot relics solely on a single ontailment-based hallucination discriminator, whereas Tri-HE leverages powerful 4LM-based discriminators capable of accurately and simultancously identifying both object and relation hallucinations, leading to more comprehensive hallucination evaluation results.
3

---

Published in Transactions on Machine Learning Research (07/2025)
3 Unified Hallucination Evaluation Framework Formulation
Inspired by the relation extraction (Xiaoyan et al. 2023 task, in this section, we propose unified framework to evaluate both object and relation hallucinations via the object-relation triplets (i.., (Object, Relation, Objectz)). Here the objects and relations can either be a word or a phrase with attributes. We start by defining object and relation hallucinations via triplets $31 based on which, we define our evaluation metrics and pipeline in 3.2 and $3.3 separately.
3.1 Definitions
As illustrated in Figure la we formulate our framework with the standard VQA setting (although they can be generalized to evaluate hallucinations in any lasounlanguaae tasks given available scene graph annotations, as discussed in $3.4). Specifically, considering an input image I, a corresponding question Q associated with image I, its ground truth answer A, and the answer Ag predicted by an LVLM Ao(-IQ, I) parameterized by 0, we can first have the following definitions:
== as the scene graph of I, where V and E refer to all the objects existing I and all the possible relations among existing objects, respectively.
G' = (V' VV,'', S E) as the knowledge graph that includes all the required objects and relations answer preciscly.
Gg (Va, Es) as the knowledge graph extracted from Ag, where Vo and Es include all the objects and all the possible relations among objects mentioned in Ag.
Note that here all graphs can be converted to a set of triplets (d.e., G= [(v1, e, 02)}, where E V and e E E). common nightmare in previous LVLM hallucination literature lics in the ambiguous discrimination between prediction hallucinations and errors (Ji et al.j 2023). To obtain unbiased hallucination evaluation results, we separate them depending on whether or not the wrongly generated objects or relations exist in the given image I. Specifically, given triplet (u),e,02) E Gg, have the following definitions,
Object hallucination: if e1 & V or U2 & V, suggesting Ag includes an object not within I. For example, the triplet (location, suggests, popular spot for socializing) in Figure la encounters an object hallucination since the object popular spot for socializing" cannot be obtained from V.
Relation hallucination: if on, 12 E V yet e & E, suggesting that Ag correctly recognizes two related objects from I but pair them with a non-existing relation. For example, the triplet (people, walking around, area) in Figure La] has a relation hallucination since the relation "walking around" cannot be obtained from G, despite that the objects are all in V.
Prediction error: if B,, U2 V and E yet (vi,e, U2) & G, suggesting As correctly recognizes objects and relations from I, yet pairs in wrong way.
3.2 Evaluation Metrics
With the above definition in hand, given the knowledge graph Go extracted from a model response Ag, we calculate the hallucination rates Ag as the proportion of hallucinated triplets in Gg. Most previous works (e.g., POPE (Li al. 2023c)) directly evaluate the hallucination rate at the object-level with respect the total number predicted objects, yet make their results not comparable among LVLMs, since different. LVTMSs might refer to different numbers of objects in their responses. To address this issue, we instead opt to calculate the hallucination rate at the question- and image-level. Specifically, calculate two types of hallucination rates, including the question-level hallucination rate (Halluq) and image-level hallucination rale (Hallu), as defined in the following,
TT in Go )
Halluo(())) 1
HT in Go
x 100%.
(1)
HQH
('E(Q))
4

---

Published in Transactions on Machine Learning Research (07/2025)
Hallur((1)))
1 KTH
Σ Halluq((Qr))) 100%, rE(T)
(2)
where HT is Hallucinated Triplets, TT is Total Triplets, {Q} and {1} are the sets of questions and images that LVLMs are evaluated on, respectively, and {Qr) (Q) suggest the subsets of questions related to the image I'. For both metrics, lower values demonstrate fewer hallucinations. Since the total number of questions and images is maintained the same for all evaluated LVLMs, Halluq(:) and Hallur(-) are indeed comparable and unbiased.
3.3 Evaluation Pipeline
With the definitions and evaluation metrics provided in $3.1 and $3.2 the remaining problems contain two folds: 1) how to extract the knowledge graph Ga from LVLM responses Ae, and 2) how to judge whether a triplet in Gg is hallucinated or not. The overview of our pipeline is illustrated in Figure la
Knowledge Graph Extraction. Given an LVLM response As with the corresponding question Q and image I, we extract the knowledge graph Gg from Ag via prompting GPT-4. Check our prompl for knowledge graph extraction in Appendix { Afterwards, we propose two different strategies to judge whether a triplet (V1, e, U2) E Go includes hallucination basod on the ground truth answer A and the image scene graph G, described in the following.
NLI Judge. The first strategy is implemented with a natural language inference (NLI) (Reimers & Gurevych 2019) model ! Specifically, given extracted triplet, first calculate its cosine similarity scores with all triplets in the image scene graph G and only retain those ground truth (GT) triplets with similarity scores greater than 0.5 to refine the information that will be used for the NLI model. If no triplets in G meet this criterion, only the top three GT triplets with the highest similarity scores will be kept, which are then taken as ground truth imputs for the NLI model to make predictions. the NLI score between the extracted triplet and ground truth triplets is lower than 0.6, suggesting the extracted triplet cannot be induced based on GT triplets, and therefore, resulting in a hallucination.
To determine the threshold, randomly selected question instances from 10 images and reviewed the set of filtered triplets that were returned. The similarity score threshold was adjusted to 0.5 for the most reasonable returned triplets. These triplets later concatenate together as the ground truth required for generating NLI judgments. In determining a generated triplet was hallucinated, we further review the NLI judgment results in different thresholds, ultimately deciding on a threshold 0.6.
LLM Judge. Another evaluation strategy to leverage prompting of a powerful LLM, a widely-adopted practice recent works for assossing LLM outputs (Zheng et al.] 2023). In this work, we primarily utilize GPT-4 in LLM judge to determine whether given extracted triplet (U1, e, 02) E Go can be directly obtained inferred from the image scene graph G. Note that:
We do not employ GPT-4V in LLM judge, as Li et al. (2024) have reported that the text-only GPT-4 judge is more consistent with human preferences than GPT-4V judge.
Frontier open-source models, such as LLaMA-3.3, can similarly deliver reliable and cost-efficient hallucination evaluation results (see detailed analysis in Table 3) which might be superior alternatives when the GPT-4 judge is unavailable.
Additionally, a triplet (vi, e, 22) judged as hallucinated, we further prompt the LLM judge clarify whether the hallucination pertains specifically to the relation the objects vi U2. Refer to Appendix (A.2 for the prompt of LLM judge in our experiments 2
https:/ lattps://huggnnttaee htt.g//gggggac...cccc// http:///wee..eeeeetoeeeeeeeeeern
base-v2
For knowledge graph extraction and LLM judge, we utillize "gpt-4-1106 prev-erev model via OpenAT's API with default. inference parameters.
5

---

Published in Transactions on Machine Learning Research (07/ (07/2025)
3.4 Generalizability of our Framework
Although formulate our unified triplet-level hallucination evaluation framework in the sections mentioned above primarily based on the VQA tasks, it is capable of evaluating hallucinations in LVLM responses for any vision-language task (since knowledge graph extraction is suitable for all natural-language-aasee responses), provided that the corresponding scene graphs for the lest images are available or extracted by pre-trained expert models. This underscores the task-agnostic design of our proposed framework and highlights its strong generalization capability. We leave the detailed exploration for other vision-language tasks for future work.
4 Tri-HE Construction
Following the formulation in $3 this section, we provide a detailed discussion on how to construct our benchmark Tri-HE for a unified triplet-level evaluation of both hallucinations in LVLMs.
Image Collection. The construction of Tri-HE begins with images from the GQA dataset (Hudson & Manning 2019), as the scene graph annotations provided by GQA naturally fit our triplet-level hallucination evaluation formulation. Nevertheless, some scene graphs in GQA contain incomplete object relationships, omitting information necossary for accurate question answering. To mitigate this issuc, initially filter the GQA images, retaining only those whose scene graphs contain at least five object relations (edges between nodes). Subsequently, we select 300 images from the filtered images according to the following criteria:
1. Each image must contain more than two related objects.
2. Each image must be sufficiently clear to discern all visual details.
This procedure ensures a set of high-quality images suitable for subsequent dataset construction.
VQA Question Generation Next, since the VQA questions in the GQA dataset have already been extensively used during the pre-training of current LVLMs, we instead employ GPT-4V + to generate novel question-answer pairs for each image to avoid data contamination. To effectively examine both object and relation hallucinations in IVLM responses, we aim to generate questions that necessitate commonsense reasoning grounded on the provided images. Specifically, for every image, GPT-4V is prompted to generate 10 questions along with their answers | each requiring image-based commonsense reasoning to be answered. Furthermore, we ask GPT-4V to produce relation triplets describing the reasoning processes of answering the questions. These additional triplets can subsoquently be used to enrich the original image scene graphs.
VQA Question Verification Following the initial generation of VQA questions, three annotators manually examine all generated questions, answers, and triplets based on the following critcria:
Each question must be valid and answerable using commonsense reasoning based on the input image.
2. Each answer must appropriately address the question using commonsense reasoning.
3. Each triplet must accurately describe the corresponding answer and must only contain objects visible within the image.
Questions or answers failing to meet these conditions are discarded, while invalid triplets are excluded from the respective scene graphs. To validate the annotation consistency, the annotators jointly annotate 100 common question-answer pairs, achieving a Krippendorff's alpha (Krippendorf]] 2011) of 0.62, demonstrating substantial annotaoro agreement and guaranteeing the consistency of Tri-HE annotations.
We the "gpt-4-vision preview" "ioor---iiionnpreieeee"" model here, the same as 5.2 4 (Check A.3 for the prompt used here.

---

Published in Transactions on Machine Learning Research (07/2025)
# Images
# #Questions
# Objects
# #Relations
# Questions/Image
# #Triplets/SG
300
1226
1723
618
4.09
19.10
Table 1: Statistics of Tri- HE. "SG" refers to Scene Graph.
Scene Graph:
Scone Graph:
Scene Graph:
hanging
pida
foo
Fanding
haddig
peeking
vikp
workig
Sample 1::
Sample 1:
Q What kind of flavors can we expect from the dish
'Sample
Q: What is the likely role of the person wearing helmet?
being cooked?
How the location? location seems very buay at moment.
A: The flavors can likely be rich and sevory, possibly with mix of spices.
likely
the person wearing
Triplet(s) support the answer: (benches, mosily empty, platform) crowd, visible, area) Sample 2:
of batter.
Triplet(s) support the
Triplet(s) support the answer:
(Food pan, contains, vegetables and meat) Sample
(Person helmet, holding, bet)
................................
Sample
a Why there someone standing in the background?
What might people waiting A: The people are ikaly wailing for a train. Triplct(s) support the answer:
Q: What could be the reason for the individual with the glove being alert stance?
A: The person the background might be wailing for assisting in cooking process.
individual the glove being
Triplat(s) support the answer: (Person in background, standing, In kitchen)
(people. sitting, benches) (triin, adjacent to, platform)
an alart stance to prepare to catch the ball.
Tripict(s) support the answer:
[Person glove, assuming. ready posilion)
Figure 2: Visualization of data samples in Tri-HE. Each image is associated with a scene graph and question-answer pairs with the reasoning triplet annotations.
Statistics. The overall statisties for Tri-HE are summarized in Table I As described in Figure 2 each image in Tri-HE is linked to a scene graph and a set of question-answer pairs that require reasoning, accompanied by ground truth triplet annotations. Note that since the quality of each question in Tri-HE is manually verified, expanding its size requires significant resources and poses challenges. Nonetheless, the number of images and questions in Tri-HE is comparable existing LVLM hallucination evaluation benchmarks such as Zhao et al. 2022) and Guan et al. (2024). Furthermore, demonstrated in Table 3 Tri-HE has been sulficient to provide reliable hallucination evaluation results, largely thanks to the high-quality annotation procedure described above.
5 Evaluation Results
5.1 Evaluated LVLMs
We selected six open-source LVLMs for evaluation, including the LLaVA scries [Liu et al. 2023a MiniGPT-4 (Zuu et al. 2023), InSI (Dai et al. 2023), Shikra (Chen el al. 2023d), and MMternLM-XCompeln abbrev. InternLM-X2)) (Cai et al. 2024). For all evaluated LVLMs, we selected the 7B variants to ensure (abbrev., LLaMA-3.2) MctaAl 20244a and use its smallest version (11B). The prompt templates and inference configurations used All experiments conducted two Nvidia A100 GPUs.
fair comparison. Additionally, we test the recent popular Llama-3.0 3lama-3.2-Vis
for LVLMs are detailed in Appendix A..4 and g
7

---

Published in Transactions on Machine Learning Research (07/2025)
LLM Judge
NLI Judge
Method
Overall
Object
Relation
Overall
Hallur +
Halluq |
Hallur
Halluq
Hallur +
Halluq +
Hallur +
Halluq +
MiniGPT-4
53.60
51.79
28.32
26.77
25.25
24.98
55.61
53.36
InstructBLIP
46.68
45.57
22.19
20.88
24.50
24.69
58.25
55.56
LLaVA
42.34
41.30
19.88
18.50
22.46
22.80
54.49
51.51
Shikra
42.20
41.76
18.55
17.54
23.65
24.22
56.46
53.98
LLaVA-1.5
40.66
39.10
18.63
17.28
22.03
21.82
54.14
51.67
LLaMA-3.2
40.16
38.95
22.30
21.08
17.86
17.87
48.46
45.64
InternLM-X2
38.83
37.54
18.25
17.50
20.58
20.04
54.41
52.08
Table 2: Comparison on hallucination rates among different LVLMs on Tri-HE. The best rcsults
under each column are boldfaced. InternLM-X2 short for IntemLM-XXCoppsee> (Cai et al. 2024 Appendix for evaluation results on more LVL.Ms.
Check
Method
LL La MAANAALLLAA...MMMGGPPPP-Innnnnn
Shikra MnterLM-XX22PPM GPT-4V
NLI Judge(Sentence)
0.2182
0.0970
0.3609
0.2596
0.2684
0.2524
0.2787
NLI Judge (Tripeet))
0.2951
0.2838
0.2264
0.4259
0.2829
0.2647
0.4190
LLM Judge(Llama-a3 Sentence)
0.4705
0.4842
0.3617
0.2520
0.4941
0.4366
0.4969
LLM (Llamam333 Triplct)
0.5138
0.5262
0.4150
0.4798
0.5311
0.5323
0.5519
LLM Judge Sentence)
0.6631
0.5409
0.3669
0.5532
0.5821
0.5998
0.5548
LLM Judge(GPTT4 Triplet)
0.8115
0.6320
0.4283
0.6235
0.6939
0.7169
0.7292
Pearson correlation scores among automatic hallucination judgments and human judgments. The best results under each column are boldfaced. The LLM Judges utilized are specified the brackets.
Table 3:
5.2 Main Result
LVLM comparison. Table 2 compares hallucination rates of different LVLMs on our Tri-HE benchmark. As can be seen, all the evaluated LVLMs suffer from generating hallucinations with at least 38% hallucination rates. Among these LVL.Ms, InternI.M-22 (Cai et al. 2024 obtains the best overall performances, suggesting that its strategy to train with both text-image and textual-only instruction data simultaneously helps better align its visual encoder and LLM, and thus, reduces its hallucination rales. Moreover, compared to LLaVA (Liu) et al. 2023b), Shikra (Chen et al. 2023d) has consistently lower hallucination rates, which is built upon LLaVA's 's structure with extra grounding capability introduced, indicating that introducing extra grounding could help LVLMs reduce hallucination. Additionally, LLaMA-3.2 achieves the lowest relation hallucination rates, suggesting that a strong textual backbone can help mitigate relation hallucination. However, it exhibits a weaker ability to accurately identify objects, impacting its object and overall hallucination rates. Since LLaMA-3.2 does not outperform other LVLMs with even more paramcters, we do not adopt it in the remaining experiments for parameter consistency.
Relation hallucination is more severe. Except for MiniGPT-4 and LLaMA-3.2, all the LVLMs generate more relation hallucinations than object hallucinations. A possible explanation is that existing LVLMs lack reasoning abilities, which makes them easily confused and mess up the relations among objects. This further suggests that focusing on object hallucination [Li et al. 2023e) is not enough for a throughout analysis of the IVLM reliability, and a unified and comprehensive study like our triplet-level evaluation necessary.
Evaluation pipeline. In addition, we observe that LLM Judge can provide clearer and more reasonable discrimination between models compared to the NLI judge. We provide a more comprehensive investigation into the differences between these two judges later in $5.3 Besides, the evaluation results under both Hallu, and Halluq metrics demonstrate similar trends, proving the robustness of our proposed triplet-level hallucination evaluation setting under different evaluation granularities.

---

Published in Transactions on Machine Learning Research (07/2025)
LLaVA
LLaVA-1.5
MiniGPT-4
Instruct BLIP
Shikra
InternI.M-X2
Hallu +
22.46
22.03
25.25
24.50
23.65
20.58
Original
Halluq J
22.80
21.82
24.98
24.69
24.22
20.04
Hallu +
20,86
18.44 44
23.00
21.73
22.47
18.57
First 20%
Halluq 4
18.73
18.06
22.68
19.82
19.34
16.10
Table 4: Relation hallucination rates for the top 20% frequent object pairs of different LVLMs
under the LLM Judge. Original refers to the results in Table 2
In addition to evaluating open-sourced LVLMs, we further investigate the performance of closed-sourced LVLMs. Due to limited experimental resources, we specifically evaluate the GPT-4V (OpenAl 2023) model on subset of 25 randomly selected images from Tri-HE Tri-HE. Specifically, we prompt GPT-4V to obtain responses to all questions related to these selected images and compute the associated hallucination rates following the steps described in Table 2 For comparison, we also inchude results from open-sourced LVLMs evaluated on the same set of 25 images. As illustrated in Figure Th GPT-4V clearly demonstrates superior performance, surpassing all open-sourced LVLMs. Although GPT-4V exhibits slightly higher object hallucination rates comparod to InternLM-X2 -ikcklikkk likely because it tends to associate additional objects not present in the image it achieves notably lower relation hallucination rales due to its stronger reasoning capabilities, resulting in lower overall hallucination rates.
Evaluate Closed-sourced LVLMs.
5.3 Analysis
Investigating automatic hallucination judgments with human judgments. Here, we further illustrate the effectiveness of the triplet-level evaluation setting by studying its correlation with human judgments. To conduct fine-grained hallucination analysis, previous works (Jing et al. 2023 Min et al. 2023 split a model response into sub-sentences first, on which their hallucination measurements are conducted. We regard this method as a baseline for comparison. Specifically, we sample a subset of 20 images from Tri-HE and invite human annotators to score five-point-scale hallucination rates of the responses of all the LVLMs in $5.1 (check Appendix $C for the detailed annotation guidelines). The human annotators achieve a Krippendorff's alpha score of 0.66, indicating high inter-agreement.
Results are shown in Table 3 We find that triplet-level hallucination rates have higher correlations with human judgments with both NLI and LLM Judges, indicating that identifying hallucination on triplets can lead to a more accurate, human-preferred evaluation for model responses. Moreover, we notice that the LLM Judges achieves a higher correlation to human judgments compared to the NLI counterpart, revealing LLMs superior abilities to find hallucinations, which also consistent with our observation $5.2
Applying Different LLMs in LLM Judge. While GPT-4 allows the LLM Judge to produce reliable hallucination evaluations, the associated API expenses could become large when evaluating large number of examples. To mitigate potential cost constraints, also examine whether alternative open-source LLMss can serve effectively in LLM Judge. Specifically, we replace GPT-4 with Llama-3.3) (MetaAl 2024b) and re-evaluate all examples listed in Table 3 As shown, similar to GPT- 4, Tlama-3.3 consistently achieves higher correlation scores at the triplet-level than at the sentence-level. Furthermore, its Pearson correlation scores with human evaluations, while significantly outperforming those obtained using NLI Judge, remain comparable to GPT-4's results for certain LVLMs. These findings suggest that open-source LLMs can serve as viable alternatives to GPT-4 in LLM Judge, providing reliable evaluation results under tight budget constraints, thereby further validating the robustness of our proposed LLM Judge.
(abbrev.,
Investigating relation hallucination with object information. As concluded from 5.2 existing LVLMs tend to generate both object and relation hallucinations in their replies, while the relation hallucination rates are even higher. Since different LVLMs have pairs of objects (v1, U2) that they are familiar with (e.g., high-frequency object pairs in the instruction data they are fine-tuned on) and might generate the correct relations on these objects easily, we suppose that the relation hallucination problem might mostly be located in less-frequent object pairs. To verify this assumption, we extract all object pairs for each LVLM from their respective Go generated from responses on Tri-HE, and rank these pairs based on their frequency. Then, we caleulate each LVLM's relation hallucination rates their most frequent object pairs.
9

---

Published in Transactions on Machine Learning Research (07 (07/202) /025)
0.50
0.45
Hallu
Halluo
0.40
0.35
LLaVA
LLaVA
0.30
0.30
MiniciPT-4
MiniGi7-4
InstructaUP
InstructBUP
Shikra
0.55
Shikra
0.25
InternLM2
InternLM2
20
40
60
80
100
20
40
60
80
100
Number of tokens
Number of tokens
Figure 3: Trends of the hallucination rates of the imago-level (left) and question-level (right) evaluations for different IVLMs with respect. to the number of tokens in the model responses.
To obtain object pairs and their rankings from LVLM responses, suppose that for the targeted LVLM, we have its responses to all questions in Tri-HE, i.e., Gg, we first extract all the object pairs (V1, (2) from Gg. Then, for each object, replace it with the name of its synset using WordNet to reduce the total types of objects. Afterward, could calculate the frequency of each object pair and rank them based their frequency. This ranking will then be used to calculate the first 20% frequent object pairs in Table
As in Table 4 all the LVLMs have significantly lower relation hallucination rates on frequent object pairs they are familiar with, suggesting that they know the possible relations among objects and understand how choose relation appropriately for frequently occurring objects.
Investigating hallucination rates with response length. Previous studies on LVLM hallucination evaluation suggest that the length of model responses may influence the extent of hallucination (Li al., Zhou et al. 2023) some LVLMs tend to produce shorter, safer outputs. However, directly instructing LVLM to generate response of a specific length is challenging. To address this, we instead truncate the responses to the first K tokens and compute ballucination rates, varying K to assess its impact on the results.
2023c
As shown in Figure 3 while the exact hallucination rates vary, the ranking of different LVLMs remains consistent as the number of tokens increases from Overall, as fewer tokens provide insufficient data for triplet extraction, this finding supports the robustness of our proposed triplct-level evaluation across LVLMs with varying response lengths.
5.4 Hallucination Mitigation
After demonstrating that LVLMs exhibit significant hallucination problems, we further explore potential approaches to reduce both object and relation hallucinations. Prior works (Jing et al. 2023 Zhou et al.)
2023 Li et al.] 2023c Gou et al.] (024) have suggestod that modality mésalignment might be primary cause behind IVLM hallucinations. Motivated by this claim, we propose training-fres method to mitigate hallucinations by improving modality alignment within LVLMs.
Method. Specifically, propose two-step strategy. Given image and its corresponding question, we first prompt the evaluated LVLM to generate description of the image guided by the given question (General Description in Figure g Next, we prompt the same LVLM (in new version without the image memory) using this generated description to answer the question. Through this approach, we elfectively leverage the strong cnstruction-oollowiini capability intrinsic to the LVLM's LLM backbone, instead of requiring the LVLM simultancously comprehend the image and answer the question, thereby roducing hallucinations caused by modality misalignment. Moreover, as indicated in $5.3 triplet-level evaluation is more effective than sentence-level evaluation in assessing hallucinations. Hence, we further explicitly guide LVLMSs to concentrate
10

---

Published in Transactions on Machine Learning Research (07/2025)
AFTER
General Description:
Triplet Description:
USER:
USER:
REQUEST:
REQUEST:
What are the two people likely about to do?
What are the two people likely about to do?
Based on the REQUEST, describe the image.
Based on the REQUEST, identify the objects that are CERTAINLY PRESENTED provided image, and describe the relationships between the identified objects.
MiniGPT44:
The image shows two people standing on
BEFORE
sandy beach, each holding surfboard
MiniGPT-4:
w/o Mitigation:
USER:
The two people in the image are likely walking along the beach with their surfboards
You are given some hints regarding a question on an image.
USER:
What are the two people likely about to do?
USER:
MiniGPT-4:
Hints: "Answer: The image shows two
You are given some hints regarding a
The two people in the image are likely walking along beach, each carrying surfboard under their arms. The ocean is visible in the background, with waves crashing against the shore. The sky is overcast, with a few clouds visible. The image appears to be taken in the morning, with the sun just beginning to rise over the horizon.
question on an image.
Based the hints, answer the following
question without hallucination.
Hints: "Answer: The two people in the
What are the two people likely about to do?
Based on the hints, answer the following
question without hallucination.
MiniGPT-4 w/o seeing the image:
Based on the hints, the two people are likely
What are the two people likely about to do?
preparing to go surfing. They are dressed in black wetsuits and have their hair lied back
MiniGPT-4 w/o seeing the image:
It difficult determine exact location
ponytails, which typical attire for surfing. The ocean is visible in the background, with waves crashing against the shore, and the sky is cloudy and overcast, which is typical weather for surfing. Therefore, it likely that the two people are about to go surfing.
Based on the hints, the two people in the image are likely walking allong the beach with their surfboards under their arms. They appear to be heading towards the ocean, possibly to go surfing. The surfboards they are carrying suggest that they are preparing to go surfing. Therefore. the two people are likely about to go surfing.
of the image, but it appears to be a beach with a sandy shoreline and rocky outcroppings. The surfboards suggest that the people in the image are planning to go surfing....
Figure 4: An illustration demonstrating hallucination mitigation. The three prompting strategies (w/o Mitigation, General Description, and Triplet Description) are listed from left to right. Hallucinated content is highlighted in Red and repeating contents are marked with italic and underline.
LLM Judge
NLI Judge
Mitigation
Hallur +
Halluq 4
Hallur 4
Halluq 4
w/o Mitigation
45.86
47.44
55.93
54.94
MiniGPT-4
General Description
46.50
49.19
54.59
53.03
Triplet Description
44.14
42.96
51.19
47.12
w/o Mitigation
30.72
30.17
53.84
52.06
LLaV5-1.5 General Description
28.70
29.80
51.40
49.80
Triplet Description
28.39
32.68
48.97
48.40
Table 5: Hallucination mitigation results. The best results under each column are boldfaced.
more on identifying objects and their interrelations via triplets when describing images (Triplet Description in Figure 4p. We evaluate MiniGPT-4 and LLa VA-15 combined with our proposed mitigation approaches using the subset previously employed in Figure 1b The corresponding prompts along with an example illustration shown in Figure
Results. As demonstrated in Table 5 both LVLMs exhibit reduced hallucination rates after applying our two-stage mitigation method, indicating that improved modality alignment effectively alleviates hallucinations. In addition, explicitly prompting LVLMs to emphasize objects and their relationships consistently yields the lowest hallucination rates across most cases, further reinforcing the findings presented in Table 3
A Ablation Study. We also perform an ablation study on the best pest-performing "Triplet Description" variant of our mitigation approach to gain deeper insights into the role of cach module within our proposed method. Specifically, we compare the Triplet Description (i.e., Triplet+ Close) results obtained by MiniGPT-4 with two alternative setups:
11

---

Published in Transactions on Machine Learning Research (07/2025)
LLM Judge
NLI Judge
Mitigation
Hallur |
Halluq |
Hallu
Halluq
w/o Mitigation
45.86
47.44
55.93
54.94
Eyes- Close
46.50
49.19
54.59
53.03
Triplet
45.65
45.16
59.35
55.57
Triplet+ Eyes- Close
44.14
42.96
51.19
47.12
Table 6: Ablation study on hallucination mitigation. The best results under each column are boldfaced.
1. This setting is equivalent to General Description. Image accoss is disabled (i.c., eyes-close) while prompting LVLM to answer the question. It is designed to assess the impact of employing triplet-level descriptions.
2. Triplet: This setting is similar to Triplet Description but allows image accessibility. It incorporates both the original image and the generated triplet-level description simultaneously as inputs. It is designed to examine the effects of modality alignment.
The experimental results are presented in Table 6 As shown, the combined use of triplct-level description and restriction of visual input access leads to the lowest hallucination rates. These findings further validate the design choices made in our mitigation method.
Model
Mitigalion
Hallur + Halluq
w/o Mitigation
53.84
52.06
LLaVA-1.5
LogicCheckGPT
51.10
50.84
Ours (triplet-+ eycs-close)
48.97
48.40
w/o Mitigation 55.93
59.94
MiniGPT-4
GogicCheckGPT
52.34
53.04
Ours (triplet-+ eyca-close)
51.19
47.12
Table 7: Hallucination mitigation results on LLaVA-1.5 Liu et al.
(2023a) and MiniGPT-4 Zhu
et al. (2023) with baseline comparison. The best results under each column are boldfaced. Check more baseline comparison results in Appendix D
Baseline Comparison. Current methods for mitigating hallucinations typically involve retraining, integrat- ing external detection models, and devising decoding strategies. Compared to existing works, our approach is a plug-and-play method that neither requires costly retraining nor relies on external models. To make a more fair comparison, we conducted experiments with LogicCheckGPT Wu et al. (202aa), a training-fre free approach that addresses hallucinations through prompting interactions with the help of GPT-3.5. Under the cost consideration, an evaluation was conducted only with the NLI judge. The results indicate that our method outperforms LogicCheckGPT, highlighting its effectiveness in mitigating hallucinations shown in Table 7 Qualitative comparison with LogicChecKGPT (Wu et al. 2024a) is provided in Figure
6 Conclusion
Starting from unified definition of hallucinations, propose novel triplet-level LVLM hallucination evaluation framework for both object and relation hallucinations. Then we introduce Tri-HE, novel triplet-level LVLM hallucination evaluation benchmark, with which, we conduct a throughout analysis of the discrepancy among object and relation hallucinations. Finally, we propose a simple yet effective training- hallucination mitigation method, which integrates our findings regarding objects and inter-object relations.
This work has been made possible by Rescarch Impact Fund project (RIF R6003-21) and General Research Fund project (GRF 16203224) funded by the Research Grants Council (RGC) of the Hong Kong Government.
Acknowledgments
12

---

Published in Transactions on Machine Learning Research (07 2025)
References
Zheng Cai, Maosong Cao, Haojiong Chen, Kai Chen, Keyu Chen, Xin Chen, Xun Chen, Zehui Chen, Zhi Chen, Pei Chu, Xiaoyi Dong, Haodong Duan, Qi Fan, Zhaoye Fei, Yang Gao, Jiaye Ge, Chenya Gu, Yuzhe Gu, Tao Gui, Aijia Guo, Qipeng Guo, Conghui He, Yingfan Hu, Ting Huang, Tao Jiang, Penglong Jiao, Zhenjiang Jin, Zhikai Lei, Jiaxing Li, Jingwen Li, Linyang Li, Shuaibin Li, Wei Li, Yining Li, Hongwei Liu, Jiangning Liu, Jiawei Hong, Kaiwen Liu, Kuikun Liu, Xiaoran Liu, Chengqi Lv, Haijun Lv, Kai Lv, Li Ma, Runyuan Ma, Zerun Ma, Wenchang Ning, Linke Ouyang, Jiantao Qiu, Yuan Qu, Fukai Shang, Yunfan Shao, Demin Song, Zifan Song, Zhihao Sui, Peng Sun, Yu Sun, Huanze Tang, Bin Wang, Guoteng Wang, Jiaqi Wang, Jiayu Wang, Rui Wang, Yudong Wang, Ziyi Wang, Xingjian Wei, Qizhen Weng, Fan Wu, Yingtong Xiong, Chao Xu, Ruiliang Xu, Hang Yan, Yirong Yan, Xiaogui Yang, Haochen Ye, Huaiyuan Ying, Jia Yu, Jing Yu, Yuhang Zang, Chuyu Zhang, Li Zhang, Pan Zhang, Peng Zhang, Ruijie Zhang, Shuo Zhang, Songyang Zhang, Wenjian Zhang, Wenwei Zhang, Xingcheng Zhang, Xinyue Zhang, Hui Zhao, Qian Zhao, Xiaomeng Zhao, Fengzhe Zhou, Zaida Zhou, Jingming Zhuo, Yicheng Zou, Xipeng Qiu, Yu Qiao, and Dahua Lin. Internim2 technical report. ArXiv preprint, abs/2403.17299 2024. URL https://arxiv. htrg/bbb/24005 2403 17297
Kai Chen, Lanqing Hong, Hang Xu, Zhenguo Li, and Dit-Yan Yeung. Multisiam: Self-supervised multi-instance siamese representation learning for autonomous driving. In ICCV, 2021.
Kai Chen, Zhili Liu, Lanqing Hong, Hang Xu, Zhenguo Li, and Vin-Yan Yeung. Mixed autoencoder for sclf-supervised visual representation learning. In CVPR, 2023a.
Kai Chen, Chunwei Wang, Kno Yang, Jianhua Han, Lanqing Hong, Fei Mi, Hang Xu, Zhengying Liu, Wenyong Huang, Zhenguo Li, Yan Yeung, Lifeng Shang, Xin Jiang, and Qun Liu. Gaining wisdom from setbacks: Aligning large language models via mistake analysis. ArXie preprint, abs Abs/2310. 2023b. URL httpp:::/aaiiiv.rrr///aaaaa//e 20t1//0.////31111771
Kai Chen, Enze Xie, Zhe Chen, Yibo Wang, Lanqing Hong, Zhenguo Li, and Dit-Yan Yeung. Geodiffusion: Text-prompted geometric control for object detection data generation. ArXiv preprint, 0/5//2306.0400
http/////230060000000 2023c. URL huttps///arxiv http:////aarriv.oo/aa/////a/
Kai Chen, Yunhao Gou, Runhui Huang, Zhili Liu, Daxin Tan, Jing Xu, Chunwei Wang, Yi Zhu, Yihan Zeng, Kuo Yang, et al. Emova: Empowering language models to see, hear and speak with vivid emotions. arXiv preprinl Xiv::24409. 18042, 2024a.
Keqin Chen, Zhao Zhang, Weili Zeng, Richong Zhang, Feng Zhu, and Rui Zhao. Shikra: Unleashing multimodal I1m's referential dialogue magic. ArXiv preprint, abs/ 15195, 2023d. URL https://arxiv org/ hrg/abs/2306..c 15195
Xiang Chen, Chenxi Wang, Yida Xue, Ningyu Zhang, Xiaoyan Yang, Qiang Li, Yue Shen, Jinjie Gu, and Huajun Chen. Unified hallucination detection for multimodal large language models. ArXiv preprint, abs/2402.03190, 2024b. URL
Xuweiyi Chen, Ziqiao Ma, Xuejun Zhang, Sihan Xu, Shengyi Qian, Jianing Yang, David F Fouhey, and Joyce Chai. Multi-object hallucination in vision-language models. arXiv preprint arXiv:2407.06192 2024c.
Zhaorun Chen, Zhuokai Zhao, Hongyin Luo, Huaxiu Yao, Bo Li, and Jiawei Zhou. Hale: Object hallucination reduction via adaptive focal-contrast decoding. Ar preprind, abs/2403.00425 2024d. URL https
Zhe Chen, Jiannan Wu, Wenhai Wang, Weijic Su, Guo Chen, Sen Xing, Muyan Zhong, Qinglong Zhang, Xizhou Zhu, Lewei Lu, et al. Internvl: Scaling up vision foundation models and aligning for generic visual-linguistic tasks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 24185 24198, 2024e.
Wenliang Dai, Junnan Li, Dongxu Li, Anthony Meng Huat Tiong, Junqi Zhao, Weisheng Wang, Boyang Li, Pascale N Fung, and Steven Hoi. Instructblip: Towards general-purpos vision- languae models with instruction tuning. In NewrIPS, 2023.
13

---

Published in Transactions on Machine Learning Research (07/2025)
Xiaoyi Dong, Pan Zhang, Yuhang Zang, Yuhang Cao, Bin Wang, Linke Ouyang, Xilin Wei, Songyang Zhang, Haodong Duan, Maosong Cao, et al. Internin-xcomposer?" Mastering free-form text-image composition and comprehension in lesion-language large model. ArXic preprint, abs/ abs/2401.16420, 2024.
URL https://arxix: org/ atggabb//4401.19222
Ruiyuan Gao, Kai Chen, Enze Xie, Lanqing Hong, Zhenguo Li, Dit Young, and Qiang Xu. Magicdrive: Street. view generation with diverse 3d geomelry control. ArXiv preprinl, abs/2310.02601, 2023. URL
attps://arxiv.orge 221p.///.rr//aa/........ 2310. 02601
Ruiyuan Gao, Kai Chen, Zhihao Li, Langing Hong, Zhenguo Li, and Qiang Xu. Magicdrive3d:: Controllable
3d generation for any-view rendering in street scenes. ArXiv preprint, abs/2405.14475, 2024a. URL
14475
Ruiyuan Gao, Kai Chen, Bo Xiao, Lanqing Hong, Zhenguo Li, and Qiang Xu. Magicdrivodit: High-rcsolution
long video generation for autonomous driving with adaptive control. arXiv preprint arXiv:2411.18807, 2024b.
Yunhao Gou, Zhili Liu, Kai Chen, Lanqing Hong, Hang Xu, Aaxue Li, Dit- Yan Yeung, James T Kwok, and Yu Zhang. Mixture of cluster-conditional lora experts for vision-language instruction tuning. ArXiv preprint, 2tt//2312.123799 2023. URL htttp://aarxx..rrg////////1/ 12379
Yunhao Gou, Kai Chen, Zhili Liu, Lanqing Hong, Hang Xu, Zhenguo Li, Dit-Yan Yeung, James T Kwok, and Yu Zhang. Eyes closed, safety on: Protecting multimodal Ilms via image-to-text transformation. ArXiv preprint, abs/2403.. 2ts/2443.09972,, 2024. URL lattps://arxi// wwt.:oog//aaaaa.eoooooooooe 2943000077
Yunhao Gou, Kai Chen, Zhili Liu, Lanqing Hong, Xin Jin, Zhenguo Li, James T Kwok, and Yu Zhang. Perceptual decoupling for scalable multi-modal reasoning via reward-optimized captioning. arXiv preprint 0rXiv:2506.0455 2025a.
Yunhao Gou, Hansi Yang, Zhili Liu, Kai Chen, Yihan Zeng, Lanqing Hong, Zhenguo Li, Qun Liu, James T Kwok, and Yu Zhang. Corrupted but not broken: Rethinking the impact of corrupted data in visual instruction tuning. arXiv preprint arXi::2502, 12635, 2025b.
Tianrui Guan, Fuxiao Liu, Xiyang Wu, Ruiqi Xian, Zongxia Li, Xiaoyu Liu, Xijun Wang, Lichang Chen, Furong Huang, Yaser Yacoob, et al. Hallusionbench: advanced diagnostic suite for entangled language hallucination and visual illusion in large vision-language models. In CVPR, 2024.
Jianhua Han, Xiwen Liang, Hang Xu, Kai Chen, Lanqing Hong, Chaoqiang Ye, Wei Zhang, Zhenguo Li, Xiaodan Liang, and Chunjing Xu. Sodal0m: Towards large-scale object detection benchmark for autonomous driving. ArXiv preprinl, abs/2106.11 11118, 2021. URL
11118
Zongbo Han, Zechen Bai, Haiyang Mei, Qianli Xu, Changqing Zhang, and Mike Zheng Shou. Skip \n A simple method reduce hallucination in large vision-language models. ArXiv preprint, abs/2400.011345, 2024. URL https: wrgg abs/ abs//402....22/////////////2 22422.0134
Qidong Huang, Xiaoyi Dong, Pan zhang, Bin Wang, Conghui He, Jiaqi Wang, Dahua Lin, Weiming Zhang, and Nenghai Yu. Opera: Alleviating hallucination multi-modal large language models via over-trust penalty and catpopppetiiooallcatatie ArXie preprint, abs abs/231 Abs/2211.17911, 2023. URL https: :
//axxiv.or wtrrgabs//311...
Drew A. Hudson and Christopher D. Manning, GQA: A new dataset for real-world visual reasoning and compositional question answering. In CVPR, 2019.
Ziwei Ji, Naycon Lee, Rita Frieske, Tiezheng Yu, Dan Su, Yan Xu, Etsuko Tshii, Ye Jin Bang, Andrea Madotto, and Pascale Fung. Survey of hallucination in natural language generation. In ACM Computing Surveys, 2023.
Liqiang Jing, Ruosen Li, Yunmo Chen, Mengzhao Jia, and Xinya Du Faithscore: Evaluating hallucinations in large vasion-language models. Xve preprint or Xin:2311.01 2023.
14

---

Published in Transactions on Machine Learning Research (07/2025)
Klaus Krippendorff. Computing krippendorff's alpha-reliability, 2011.
Sicong Leng, Hang Zhang, Guanzheng Chen, Xin Li, Shijian Lu, Chunyan Miao, and Lidong Bing. Mitigating object hallucinations in large vision-language models through visual contrastive decoding. In CVPR, 2024.
Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models. In ICML, 2023a.
Junyi Li, Xiaoxue Cheng, Wayne Xin Zhao, Jian-Yun Nic, and Ji-Rong Wen. Halueval: A large-scale hallucination evaluation benchmnark for large language models. In EMNLP, 2023b.
Junyi Li, Xiaoxue Cheng, Xin Zhao, Jian-Yu Nie, and Ji-Rong Wen. Halueval: A large-scale hallucination evaluation benchmark for large language models. In EMNLP, 2023c.
Kaican Li, Kai Chen, Haoyu Wang, Lanqing Hong, Chaoqiang Ye, Jianhua Han, Yukuai Chen, Wei Zhang, Chunjing Xu, Din Yeung, et al. Coda: A real-world road corner case dataset for object detection in autonomous driving. ArXiv preprint, abs/2203. 0772447724, 2022. URL https: org/ wttpg/bb//200..7////
Pengxiang Li, Zhili Liu, Kai Chen, Lanqing Hong, Yunzhi Zhuge, Dit-Ya Yan Yeung, Huchuan Lu, and Xu Jia. Trackdiffusion: Multi-object tracking data generation via diffusion models. ArXiv preprint, 2023d. URL httpss://arxvv
abs/ 0aabs/2312.002 231.2 00651
Yanze Li, Wenhua Zhang, Kai Chen, Yanxin Liu, Pengxiang Li, Ruiyuan Gao, Lanqing Hong, Meng Tian, Xinhai Zhao, Zhenguo Li, et al. Automated evaluation of large vision-language models on self-driving corner cases. ArXiv preprint, abs/2404.10595, 2024. URL https://arxis wwwaaiiiw....aaaaaeeeeeooo org/abs/ hts//bbb//400// 2404. 10595
Yifan Li, Yifan Du, Kun Zhou, Jinpeng Wang, Xin Zhao, and Ji-Rong Wen. Evaluating object hallucination in large vision-language models. In EMNLP, 2023e.
Haotian Liu, Chunyuan Li, Yuheng Li, and Yong Jac Lee. Improved baselines with visual instruction tuning. ArXio preprind, abs/ /310000043774, 2023a. URL https: //tpavi/////iv/////a http://wwwwww.w//////////// wwtp:/or//a/aa/////// 23300000444100
Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Tee. Visual instruction tuning. In NewrIPS, 2023b.
Zhili Liu, Jianhua Han, Lanqing Hong, Hang Xu, Kai Chen, Chunjing Xu, and Zhenguo Li. Task-customized self-supervised pre-training with scalable dynamic routing. In AAAI, 2022.
Zhili Liu, Kai Chen, Yifan Zhang, Jianhua Han, Lanqing Hong, Hang Xu, Zhenguo Li, Dit-Yan Yeung, and James T Kwok. Implicit concept removal of diffusion models. In ECCV, 2024a.
Zhili Liu, Yunhao Gou, Kai Chen, Lanqing Hong, Jiahui Gao, Fei Mi, Yu Zhang, Zhenguo Li, Xin Jiang, Qun Liu, et al. Mixture of insightful experts (mote): The synergy of thought chains and expert mixtures in self-alignment. ArXiv preprint, abs/ abs/2405.00057, 2024b. URL http:///aaaxk..oorrab/////00.0.
MetaAl. Introducing llama 3.2, 2024a. URL https://wwst http//wwwllaaaa..mmm.. com/
MctaAI 1lama-3.--0b-inssruut. 2024b.
URL https://huggingface wwwp://ea.ooetttaaaaaaaaaaaaaaae
3-70B-Instruee
Sewon Min, Kalpesh Krishna, Xinxi Lyu, Mike Lewis, Wen-tau Yih, Pang Wei Koh, Mohit Tyyer, Luke Zettlemoyer, and Hannanch Hajishirzi. Factscore: Fine-grained atomic evaluation of factual precision in long form text generation. Ar preprint, abs/2305.14251, 2023. /UU.UUppp/////xx/o///////////// 14251
OpenAl. ChatGPT, 2023. URL http::///pppaaa..c////eaaaaaaaap
Nils Reimers and Iryna Gurevych. Sentence-BERT: Sentence embeddings using Siamese BERTT-nttorks.. FMNIP-IJCNLLP 2019.
Qwen Team. Qwen2.5-vl, January 2025. URL http:///wwenm..ittuubbbo/////ooa v-v
15

---

Published in Transactions on Machine Learning Research (07/2025)
Junyang Wang, Yuhang Wang, Guohai Xu, Jing Zhang, Yukai Gu, Haitao Jia, Ming Yan, Ji Zhang, and Jitao Sang. An l1m-free multi-dimensional benchmark for milms hallucination evaluation. Xiv preprinl arXiv:2311.07397, 2023a.
Junyang Wang, Yuhang Wang, Guohai Xu, Jing Zhang, Yukai Gu, Haitao Jia, Ming Yan, Ji Zhang, and Jitao Sang. An I1m-free multi-dimensional benchmark for milms hallucination evaluation. ArXie preprint, abs/2311.07397, 2023b. URL https: http://xaxxvvvo///aaa// wwgg/ba/////aaa 2211 201..119997
Junyang Wang, Yiyang Zhou, Guohai Xu, Pengcheng Shi, Chenlin Zhao, Haiyang Xu, Qinghao Ye, Ming Yan, Ji Zhang, Jihua Zhu, et al. Evaluation and analysis of hallucination in large vision-language models.
Arx preprint, abs/ 0///2308.111222 2023c. URL lattps://aaa http://aaxvvvvva wwwpxii.orr/bb///3000. 15126
Yibo Wang, Ruiyuan Gao, Kai Chen, Kaiqiang Zhou, Yingjie Cai, Lanqing Hong, Zhenguo Li, Lihui Jiang, Din Young, Qiang Xu, and Kai Zhang. Detdiffusion: Synergizing generative and perceptive models for enhanced data generation and perception. ArXiv preprint, abs/2403.13304, 2024. URL
https://arxiv, http//riii..oggaaaa//a//aaa 2403. 13304
Junfei Wu, Qiang Liu, Ding Wang, Jinghao Zhang, Shu Wu, Liang Wang, and Tieniu Tan. Logical closed loop: Uncovering object hallucinations in large vision-language models. arXic preprinl arXin:2402. 11622, 2024a.
Mingrui Wu, Jiayi Ji, Oucheng Huang, Jiale Li, Yuhang Wu, Xiaoshuai Sun, and Rongrong Ji. Evaluating and analyzing relationship hallucinations in Ivims. arXiv preprint arXie:2406.164/3 2024b.
Zhao Xiaoyan, Deng Yang, Yang Min, Wang Lingzhi, Zhang Rui, Cheng Hong, Lam Wai, Shen Ying, and Xu Ruifeng. A comprehensive survey on deep learning for relation extraction: Recent advances and new 2023. URL htttp:://artat http::///xxvvoo////aae// wttrg/ab//3306..2200
frontiers. ArXio preprint,
Zhihe Yang, Xufang Luo, Dongqi Han, Yunjian Xu, and Dongsheng Li. Mitigating hallucinations in large vision-language models via dpo: On-policy data hold the key. arXiv preprint w-Xiv:2501.09660 2025.
Mo Yu, Lemao Liu, Junjic Wu, Tsz Ting Chung, Shunchi Zhang, Jiangnan Li, Dit-Yan Young, and Jie Zhou. The stochastic parrot on I1m's shoulder: A summative assessment of physical concept understanding. arXiv preprint ar Xiv:22502. 08946, 2025.
Zihao Yue, Liang Zhang, and Qin Jin. Less is more: Mitigating multimodal hallucination from an eos decision
perspective. ArXiv preprint, abs/ abbs2240214544 14545, 2024. URL https://arxiv: org/abs/2402/ 14545
Zhiyuan Zhao, Bin Wang, Linke Ouyang, Xiaoyi Dong, Jiaqi Wang, and Conghui He. Beyond halluci- nations: Fahancing Ivlms through hallucination-aware direct preference optimization. arXiv preprint arXiv:2311.16638 2023.
Kening Zheng, Junkai Chen, Yibo Yan, Xin Zou, and Xuming Hu. Reefknot: A comprehensive benchmark for relation hallucination evaluation, analysis and mitigation in multimodal large language models. ar preprint Xrx::22408.000 09429, 2024.
Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric Xing, el al. Judging llm-as-a-judge with me-hhnch and chatbot arena. Arxio preprint, hbs/2306.05668, 2023, URL https://arxi:: wwtpsiiivorrrgaaae org htrgg/bb/23006.. 05685
LIU Zhili, Kai Chen, Jianhua Han, HONG Lanqing, Hang Xu, Zhenguo Li, and James Kwok. Task--usstmmzze masked autoencoder via mixture of chuster-conditional experts. In ICLR, 2023.
Viyang Zhou, Chenhang Cui, Jaehong Yoon, Linjun Zhang, Zhun Deng, Chelsea Finn, Mohit. Bansal, and Huaxiu Yao. Analyzing and mitigating object hallucination in large vision-language models. arXie preprint
arXiv:2310.0075. 2023.
Deyao Zhu, Jun Chen, Xiaoqian Shen, Xiang Li, and Mohamed Elhosciny. Minigpt-4: Enhancing vision- language understanding with advanced large language models. ArXie preprint, abs/ abs/2304.10535 10592, 2023. URL httpaa//aarivo.or/aaa////0000 wwwp/bs///....o..//.ee 10592
https:// :ttps://arxivv
16

---

Published in Transactions on Machine Learning Research (07 (07/2025)
A Prompts
A.1 Prompt for triplets extraction with GPT-4
The prompt for extracting triplets in the answer generated by LVLMs is illustrated Figure 5
Given description of the image, please extract KG from the text and represent the KG with triples formatted with ('subject', 'predicate', 'object") with readability, each triplet in a line. If 'and' or 'or' exists in the input sentence, split the objects into multiple triplets. Please do not extract from uninformative sentences.
Here some in-context examples:
### Input:
Optimus (or Tesla Bot) is robotic humanoid under development by Tesla, Inc. It was announced at the company's 8 Artificial Intelligence (AI) Day event on August 19, 2021. It is planned to measure 5 ft. in (173 cm) tall and weigh 125 lb (57 kg). It hard to answer whether Tesla is good to invest without more information.
# ### KG:
('Optimus", "is", 'robotic humanoid") ('Optimus", 'under development by', "Tesla, Inc.") ("Optimus", "also known as', "Tesla Bot")
('Tesla, Inc.', 'announced", 'Optimus") (" "'Anoounemment Optimus', "occured at', 'Artificial Intelligence (AI) Day event") ('Artificial Intelligence (AI) Day event", 'held on", "August 19, 2021") l'Artfficial Intelligence (AT) Day event", 'organized by", "Tesla, Inc.')
('Optimus", 'planned to measure', '5 ft 8 in (173 em) tall")
('Optimus", "planned to measure', 'weigh 125 lb (57 kg).")
<Dnn>>
# # Input:
The image doesn't provide information about the popularity of the song. The song 'Here Comes the Boom" was originally released by American rock band Nelly in 2002 for the soundtrack of the film "The Longest Yard."
KG:
('The song 'Here Comes the Boom", 'originally released by", 'American rock band Nelly")
('The song 'Here Comes the Boom", 'released in", '2002")
('The song 'Here Comes the Boom", "featured in', 'soundtrack of the film 'The Longest Yard") ('American rock band Nelly', 'released', "The song 'Here Comes the Boom'*)
('The Longest Yard', 'had soundtrack featuring', "The song 'Here Comes the Boom'")
<Done>
Now generate the KG for the provided input lext:
# Input: finput_texx)
### KG:
Figure 5: Prompt for triplets extraction with GPT-4.
17

---

Published in Transactions on Machine Learning Research (07 (025)
A.2 Prompt for LLM Judge
The prompt for our proposed LLM judge method is illustrated in Figure 6
Given a list of reference triplets ("object1', 'relation", 'object2") extracted from the scene graph of an image, along with a list of objects observed in this image, your task is:
Task 1.
Determine if claim triplet ('object1", 'relation', 'object2") is directly supported by any single triplet in the reference, can be logically inferred from multiple reference triplets and the list of objects. Follow these steps when finishing the lask:
1. Answer 'yes' if the claim appears in the reference.
2. Answer "yes" if the claim can be logically inferred from one or more triplets in the reference. Consider:
a. General Inferences: Assess common associations or implications.
b. Conditional Phrases: Note phrases like "could be", 'might', 'suggests", which allow broader inferences.
c. Equivalence of Objects: In your judgment, treat objects of the same kind as equal. For example, 'woman', "man" should be considered under the general category of 'person".
d. Support from Object List: If the claim is not directly supported inferable from the triplets, assess whether the list of objects provides additional evidence to support or infer the claim.
3. Answer 'no' if the claim neither directly matches any triplet in the reference nor can be reasonably inferred from the triplets and the object list.
Task 2: Error categorization.
If your answer to the previous task is 'no', determine whether the not supported iinooreediiferred /infereed part in the claim is 'object1' or "object2" or 'relation'.
Reference:
RRFFERENCE>
List of Objects:
<LIST OF OBJJECT>>
Claim:
<CLAI>>
Figure 6: Prompt for the LLM Judge method.
A.3 Prompt for question generation with GPT-4V
The prompt for generating questions, answers, and corresponding triplets with GPT-4V is shown in Figure 7
A.4 Prompts for Evaluating LVLMs
When evaluating LVLMs on Tri-HE, the prompt we use is the question itself. Questions are fed into LVLMs along with the corresponding images.
18

---

Published in Transactions on Machine Learning Research (07 (07/2025)
Generate ten questions about the given image that require an inferential answer, which is not directly observable from the image. The answer to each question relation, object) triplets that appear in the scene graph of the given image. Note that the triplets should consist of objects and relations that are visible in the given image. Output the results in the format of:
be explained by more (object,
Generated Questions:
Answers:
Explanations:
Figure 7: Prompt for question generation with GPT-4V.
B
Configurations for LVLM Evaluation
For LVLM evaluations, we directly use the default configuration settings provided in their publicly available code repositorics. For instance, the configurations utilized for evaluating LLaVA models are accessible at
C Human Annotation Guideline
The detailed guidelines of our human evaluation tasks are shown in Table Noting that two types of inferences in the model responses are regarded as hallucinations during human annotation:
Unreasonable inferences (inferences that violate commonsense knowledge).
2. Inferences that are correct, yet cannot be correctly inferred from the image.
Score
Description
1) The text is totally hallucinated, and is irrelevant to the given input image and question.
or
2) The text is very hard understand.
1) Most of the given responses hallucinated, yet few sentences of them (one two) are related to the given image and question.
1) Half of the sentences in the given response are hallucinated.
4
1) Most of the sentences in the generated response not halluci- nated.
1) No hallucination exists in the generaled response.
Table 8: Detailed human evaluation instructions.
D Additional Results
In this section, we present additional evaluation results to supplement those in Table 2 and Table 5 enabling a more comprehensive evaluation. Due to the deprecation of gpt-4-1106-preiie: previously employed in LLM Judge (GPT-4 Triplet)-by OpenAl's API, and our limited budget for querying proprictary models, we adopt LLM Judge (Llama-3.3 Triplet) for the following experiments. Its effectiveness has been demonstratod in Table 3
19

---

Published in Transactions on Machine Learning Research (07 (07/2025)
Question: W hat could it be based the attire of the people?
What might the person's relationship the animal?
Question:
MiniGPT-4: Based on the attire of the people the image, it appeurs to be warm season. The mujority of the people ure weuring light clothing, such as shorts and t-shirts, which ure suitable for warm weather. Additionally, some of the people wearing sandals, which also footwear choice warm weather. urchitecture the buildings in the image also supports the ideu of warm The buildings have large windows and open bulcomies, which allow for good air circulation and natural ventilation, making them suitable for warm weather. The presence of greenery, such the plants on the balconies, also suggests a warm scason, as greenery tends to thrive weather. Overall, the image appears set warm scason.
LLaVA-1.5: Based on the image, the person is woman crouching down and holding small kitten in her ams. It close relationship with the kitten,
likely that she possibly as its owner or caretaker. The woman's actions holding the kitten and providing care suggest that she familiar with animal
is attentive its needs. This could indicate that she is responsible for the kitten's well- being and nurturing its growth and development.
LogleCheckGPT [LL5VA-1.5]: Based on the image, the is holding small kitten her she has a close relationship with kitten, possibly as its owner. The woman's actions of holding the kitten suggest that she is familiar with the animal and is attentive its needs. This could indicate that she is responsible for the kitten's well-being and is nurturing its growth and development.
is likely that
LogieCheckGPT [MiniGPT-4]: The image appears to be of group of people standing cobblestone street. There are several tall buildings in the background with balconics and windows. The people are dressed in warm clothing, indicating that it likely fall winter. Some the people are wearing scarves and jackets, while others have on lighter sweaters or jackets. The scene is lit by streetlights, and the buildings have white, sandy-colored facade.
Ours (tripletteyes-close) VA.L5A- The person's relationship with animal likely caretaker. The presence of the small black kitten lying on the floor and the person holding bottle, which could be baby bottle for the kitten, suggests that person providing care and nourishment to the kitten. Additionally, the presence bowl in image supports the idca that the person is taking care of the kitten, as it could be used for feoding the kitten or providing water. This indicates close and nurturing relationship between the person and the kitten.
owner
Ours (triplet-eyes--lose)) [MinGGPT-4]: The people in the image are dressed in warm clothing, indicating that it is likely winter scene. buildings
background are
made of stone and have large windows, suggesting that they old and possibly historic. The people are standing the sidewalk, which is made cobblestones and has a drainage grate in the middle. There are no cars or other vehicles in the scene, indicating that utmosphere of the scene is quiet und still, with the people standing in line and not interacting with each other. Based scene
podestrian arca. The overall
information, it likely winter season.
the
Figure 8: More illustrations demonstrating hallucination mitigation with comparison to Logic- CheckGPT. Hallucinated content is highlighted in Red.
20

---

Published in Transactions on Machine Learning Research (07 (025)
D.1 Evaluating more recent LVLMs Tri-HE
To ensure more comprehensive evaluation, we further evaluate InternVI.2 5-8B Chen et al. (2024e) and Owen2.5-VL-7B-
Team (025) on Tri-HE. Since we use the LLM Judge (LLaMA-3.3 Triplet) in this sctting- which differs from the judge used in Table 2 we also report the results InternLM-X2 and LLaMA-3.2 under the same judge for comparison.
The results are presented in Table 9 As shown, all recent models still exhibit certain degree of both object and relation hallucinations, demonstrating the effectiveness of Tri-HE in identifying hallucination issues in LVLMs. Notably, InternVL2_544 5-8B achieves the lowest hallucination rates across all metrics, suggesting that its superior pre-training data quality and the use of Random JPEG Compression a data augmentation strategy are effective in mitigating hallucinations in model responses.
LLM Judge
Method
Overall
Object
Relation
Hallur
Halluq
Hallur
Halluq
Hallur +
Halluq +
LLaMA-3.2
38.98
37.37
23.25
21.13
13.51
13.30
Qwen2.5- Swen2..5VLL773 I------Inttt
38.19
35.92
26.16
24.09
11.83
11.38
InternLM-X2
31.75
29.93
19.29
18.09
12.46
11.84
VT2 5-8B
29.12
26.99
17.16
14.95
12.22
11.13
Table 9: Comparison hallucination rates among recent LVLMs Tri-HE with LLM Judge (Llama-3.3 Triplet).
D.2 More Hallucination Mitigation Methods
In this section, we include additional recent training-free methods for hallucination reduction to enable more comprehensive comparison. Specifically, compare mitigation framework with decoding- based approach, VCD (Lnng et al. 2024). As shown in Table I0 although VCD is designed to address object hallucination, it exhibits more severe hallucination issues when evaluated on our reasoning-grounder question set compared to our proposed mitigation method.
addition, reinforcement learning (RL) approaches have received growing attention for addressing halluci- nation. To enable more comprchensive evaluation, we assess RL-trained method, OPA-DPO (Yang et al. 2025), using our evaluation framework. The results in Table 10 demonstrate its effectiveness. It is worth noting that, since OPA-DPO requires additional training, its performance advantage over our method is expected. Nonetheless, it can potentially be integrated with proposed approach foundation for further RL fine-tuning.
Model
Mitigation
Type
Hallu;
Halluq +
w/o Mitigation
21.73
21.07
VCD
training free
28.89
30.64
LLVVA-155
Ours
training-fres free
19.53
20.98
OPA-DPO
RI-trained
14.44
16.82
Table 10: Additional baseline comparisons on hallucination mitigation with Llama3.3 Judge. The best results under each column are boldfaced.
E
Future works
Currently, the proposod triplct-level evaluation is primarily deployed on LVLMs, whose extension to diffusion
models (Chen al. 2023c Gao al., 22023 2024a by Li et al.) 2023d Liu al. 2024a Wang al. 2024 is feasible, while for the hallucination mitigation proposed in 5.4 can be further enhanced by utilizing
21

---

Published in Transactions on Machine Learning Research (07/2025)
stronger vision encoder (Chen et al.,
2023a Liu et al. 2022 Zhili et al. 2023 and visual tools (e.g., 2021:
object detectors (Han et al. 2021 Li et al. 2022 to better extract visual information for LLM reasoning. Furthermore, additional types of hallucination can be incorporated into our triplet-level evaluation framework, such as temporal changes of an object over time, represented in the form of (old, change, new) triplets. We plan to explore these extensions in future work.
22


Reviews:
[{"review_id": "T8W5CT8J9X", "reviewer": "Reviewer_xQk8", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths\n\n1. The paper addresses an important and timely issue in LVLM evaluation—hallucination beyond object misidentification, extending into relation-level reasoning.\n\n2. The paper is well-writen and easy to follow\n\nWeaknesses\n\n1. High conceptual and methodological overlap with prior work (Mementos):  The proposed evaluation framework bears striking resemblance to the Mementos benchmark [1], which also extracts structured representations (object-behavior pairs) from multimodal model outputs, uses GPT-4 as a judge, and evaluates hallucination severity across object and relational reasoning. The key idea—extracting structured tuples and comparing them to ground-truth annotations using LLM assistance—is central in both papers.\n\n2. No citation or discussion of Mementos: Despite the obvious similarities in task definition, evaluation pipeline (structured reasoning units judged by GPT-4), and even the conclusion that behavioral (relation) hallucinations are more problematic than object hallucinations, the current paper fails to cite or even acknowledge the Mementos benchmark. This omission significantly undermines the contribution claim and raises concerns of originality.\n\n3. Marginal novelty: While Tri-HE targets single images and Mementos targets image sequences, this distinction is not sufficient to justify a new benchmark in the absence of novelty in methodology. Both frameworks aim to evaluate hallucination in structured formats via LLM judgment—thus, the technical advance is incremental at best. \n\nReference:\n\n[1] \"Mementos: A Comprehensive Benchmark for Multimodal Large Language Model Reasoning over Image Sequences\" Wang et. al. https://arxiv.org/abs/2401.10529", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper introduces Tri-HE, a new benchmark for evaluating hallucination in Large Vision-Language Models (LVLMs) at the triplet level, covering both object and relation hallucinations. The proposed framework extracts (subject, relation, object) triplets from model responses and evaluates them using either natural language inference (NLI) or a large language model (LLM) judge. The authors claim this unified framework enables a fine-grained and generalizable assessment of hallucinations across tasks.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "1. Explicitly acknowledge and cite Mementos in the related work and clearly delineate how Tri-HE provides additional insights or functionality not already covered by Mementos.\n\n2. Test a broader range of recent models: The evaluation currently covers a limited set of LVLMs, primarily those released in 2023. To better assess the generalizability and robustness of the proposed framework, I recommend including more recent state-of-the-art models . These models have made significant progress in reducing hallucinations, and benchmarking them would provide a more up-to-date and competitive comparison.\n\n3. Expand the benchmark in both scale and scenario diversity: The current Tri-HE benchmark includes only 300 images and focuses narrowly on VQA-style prompts. To improve its utility and impact, I strongly suggest expanding the dataset in terms of both size and coverage—adding more images, a wider range of question types, and more diverse visual scenarios (e.g., indoor vs. outdoor, real-world vs. synthetic). This would better validate the general applicability of the framework and enable more robust model comparisons.", "broader_impact_concerns": "N/A", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "j4BdT3jSHA", "reviewer": "Reviewer_TPUu", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strength:\n1. The paper is well written and easy to understand.\n2. The construction of the triplets and use of scene graph is novel and very interesting. \n3. Based on their findings, a mitigation method is proposed based on iterative prompting using triplets relations.\n\n\nWeakness:\n1. Evaluation and comparisons are limited. The author should add more evaluation and comparisons, as a benchmark paper.\n2. The size of the dataset is too small concerning. The author should enlarge the dataset and putting in more efforts in the dataset and benchmark. One of the big concerns is that the annotation of the scene graph is very difficult to scale up. \n3. The author should add more evidence on the effectiveness of their triplets and compare with some baseline training-free mitigation methods. To further improve the paper, can the construction of the triplet be useful in the training-based approach?", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "In this paper, the author proposed a united framework to evaluate object hallucination and relation hallucinations in LVLMs. In particular, the authors proposed a triplet evaluation to offer an accurate assessment of existing methods. The author also proposed Tri-HE benchmark that contains 1226 questions based on 300 images. Finally, the author proposed a training-free hallucination mitigation approach.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "Please see the weakness in the previous section.", "broader_impact_concerns": "N/A", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "vGIkVLhMQV", "reviewer": "Reviewer_mps8", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "# Strengths\n\n1. This paper introduces a triplet-level hallucination evaluation framework for LVLMs, which is a notable shift from prior approaches. The proposed framework evaluates hallucinations in a unified manner, capturing both object-based and relation-based hallucinations within model outputs. It does so by extracting knowledge triplets from the model’s response and comparing them against the image’s ground-truth scene graph. This is conceptually novel: instead of judging a whole sentence or answer, the evaluation judges the factual correctness of each extracted (object1, relation, object2) triplet.\n\n2. The Tri-HE benchmark (Triplet-level Hallucination Evaluation) built on this framework is a fine-grained testbed specifically for hallucinations. The authors highlight that this offers a finer-grained, more accurate assessment compared to existing methods. Unlike earlier benchmarks that might require task-specific transformations (e.g., forcing every query into a yes/no format), Tri-HE can be applied to diverse vision-language tasks without such restrictions, as it operates on the universal representation of triplets. This generality is the contribution – it addresses the research question the authors pose about developing a unified, unbiased evaluation across diverse tasks.\n\n3. The paper introduces two metrics: HalluQ (question-level hallucination rate) and HalluI (image-level hallucination rate). Both metrics quantify the proportion of hallucinated triplets in model responses, but at different granularities. HalluQ is calculated per question: for each question, take the number of hallucinated triplets in the model’s answer divided by the total number of triplets in that answer, then average this fraction over all questions. HalluI is calculated per image: for each image, average the HalluQ of all questions pertaining to that image, then average over images. Essentially, HalluQ treats each QA pair equally, and HalluI treats each image equally, which helps smooth out any variance due to some images having more questions than others. Both are expressed as a percentage of hallucinations (lower is better). Another important aspect is that the evaluation distinguishes object vs relation hallucination rates (and even “prediction errors”) internally, even if HalluQ/HalluI combine them into one number. The radar chart in Figure 1b and associated discussion separate “Object-Hallu” and “Relation-Hallu” for each model, which is very informative. It appears the metric can be broken down by type: e.g., compute HalluQ but considering only triplets that are object hallucinations, versus only relation hallucinations. This flexibility is good; it allows diagnosing what kind of hallucinations a model is prone to. The paper finds that relation hallucination poses a significant challenge, often surpassing object hallucination in severity, a conclusion drawn from these metrics. That insight would have been obscured if the metric didn’t differentiate hallucination types at all.\n\n# Weakness\n\n1. The authors explicitly considered some biases of previous evaluations (like models giving shorter answers to game yes/no tests) and designed against them. Their approach is largely task-agnostic, requiring only that an image have a scene graph. However, one dependency is exactly that: many real-world datasets or scenarios don’t have ground-truth scene graphs readily available. This means the framework currently shines for benchmarking and analysis, but applying it beyond curated datasets might require an automated way to get image annotations (perhaps via an off-the-shelf detector or visual grounding system). The paper’s scope is a controlled evaluation, so this is not a flaw, but it’s a practical limitation to be aware of. Another point is the reliance on GPT-4 throughout: it generates data, extracts triplets, and judges hallucinations. This heavy reliance on one AI system could pose a risk if that system had consistent blind spots.\n\n2. The authors’ approach is essentially measuring precision of the informational content (i.e., what fraction of stated facts are incorrect?). An alternative could be to also measure something akin to recall – e.g., does the model miss mentioning relevant facts? However, recall is less relevant to hallucination (it’s more about completeness than avoiding falsities), so the choice to focus on hallucination rate (precision error) is appropriate. Another alternative metric could be a severity-weighted score – perhaps not all hallucinations are equal (saying “clock on the wall” when there is none might be less severe than hallucinating a non-existent person, depending on context). But quantifying severity would introduce subjectivity, and the current metrics keep it simple and objective (a triplet is either in the image or not).\n\n3. More experiments is needed to confirm the evaluation robustness and showing broader applicability.\n  - The framework is touted as task-agnostic (it can work for any vision-language task given a scene graph). A valuable experiment would be to demonstrate this generality. For example, the authors could apply their triplet-based hallucination evaluation to an image captioning task or a visual dialogue task. In such a setup, instead of a specific question, the model generates an open-ended description. The knowledge graph extraction prompt could be adapted accordingly. If the framework works well (perhaps by using ground-truth captions or region descriptions as the “scene graph”), it would show that Tri-HE is not limited to the VQA-style setting. Even a small-scale test on a dataset like MS-COCO with ground-truth captions (converted to pseudo-triplets) could be illustrative.\n  - The authors could perturb the input to see if the hallucination evaluation is robust. For example, if there are slight errors in the scene graph (simulating a less-than-perfect ground truth), does the evaluation metric degrade gracefully? Or if the question is phrased differently but asks the same thing, do models hallucinate differently? The latter could be interesting: ask two semantically identical questions phrased differently and see if the model’s hallucination rate changes. This might reveal if certain phrasings induce more imaginative answers. It’s a bit tangential to the main focus, but any findings could be useful for understanding how to query LVLMs to avoid eliciting hallucinations.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This work makes three key contributions: first, it defines a unified triplet‑level evaluation framework that rigorously captures both object and relation hallucinations by extracting and verifying (subject, relation, object) triplets from LVLM outputs; second, it introduces Tri‑HE, a carefully curated benchmark of 300 images and over 1,200 commonsense reasoning questions (with enriched scene graphs) that enables fair, fine‑grained comparison of models via the novel HalluQ and HalluI metrics; and third, it demonstrates that powerful LLM judges (e.g., GPT‑4 or LLaMA‑3.3) align closely with human judgments for hallucination detection and leverages this insight to propose a simple yet effective training‑free mitigation strategy—two‑step prompting with focused triplet descriptions—that consistently reduces hallucination rates across multiple open‑source LVLMs.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "The requested changes can be addressed according to the weakness points made in last section.", "broader_impact_concerns": "The rest potential issue is about ethical and privacy which not deeply discussed in this paper. For example, scene graphs (from GQA/Visual Genome) may have annotations about people or objects; if an LVLM hallucinates something about a person in an image (say, infers an occupation from clothing that’s not actually evident), is that counted? The framework would count that as a hallucination (since the occupation attribute isn’t in the scene graph). This is good – it discourages models from making stereotyped assumptions (a subtle form of hallucination). It might be worth mentioning that reducing hallucinations also helps reduce stereotypical inferences when those are not grounded in the image, which is a positive ethical outcome. On the flip side, if scene graphs have biases or omissions, a model might get penalized for a correct statement that wasn’t labeled.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "eELdOnJGvp", "reviewer": "Reviewer_Ev5K", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n1. The paper is logically clear and easy to follow.\n\n2. The paper thoroughly investigates the issue of relation hallucination in LVLMs. It designs a new evaluation framework, collects a novel benchmark, and proposes a training-free mitigation method.\n\n3. The data annotation process is rigorous. The authors provide quantitative metrics for inter-annotator agreement in the paper.\n\nWeaknesses:\n1. Insufficient evaluation of models, with outdated versions. As a main contribution of the submission, the authors should evaluate more LVLMs on the proposed benchmark Tri-HE to provide a more comprehensive view of relation hallucination issues in LVLMs. Currently, the authors evaluate 8 open-source LVLMs and 1 closed-source LVLM on a subset. The authors could expand the evaluation to include models of more sizes. Additionally, the evaluation lacks recent and more advanced open-source models, such as the InternVL series and the Qwen2-VL/Qwen2.5-VL series. The absence of evaluations for these models weakens the reliability of claims in the paper, such as \"existing LVLMs lack reasoning abilities, which makes them easily confused and mess up the relations among objects\" (Section 5.2).\n\n2. Necessity of NLI-based evaluation. Tri-HE employs two evaluation methods: NLI-based and LLM-based. However, based on Table 3, the NLI-based evaluation shows significantly lower agreement with human judgments. I doubt its reliability as an evaluation method.\n\n3. Effectiveness of the proposed mitigation method. Many prior works have proposed training-free approaches to mitigate hallucinations in LVLMs (e.g., VCD[1], OPERA[2], etc.). The authors do not compare their method with these existing approaches.\n\n[1] Leng S, Zhang H, Chen G, et al. Mitigating object hallucinations in large vision-language models through visual contrastive decoding[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2024: 13872-13882.\n\n[2] Huang Q, Dong X, Zhang P, et al. Opera: Alleviating hallucination in multi-modal large language models via over-trust penalty and retrospection-allocation[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2024: 13418-13427.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "1. The authors propose a unified framework to measure object and relation hallucination in LVLMs. The framework first generates knowledge graphs from the model's response, then compares the (object, relation, object) triplets from the knowledge graph to the scene graphs of the image for hallucination evaluation.\n2. Based on this framework, the authors further propose a new hallucination evaluation benchmark, Tri-HE. Tri-HE consists of images from GQA. The questions are generated by GPT-4V and filtered by human annotators. The authors evaluate several open-source LVLMs using Tri-HE. The authors conduct several analysis based on the evaluation results, such as the alignment between human and automatic judgement, different LLM's impact on evaluation results and influence of object information on hallucination.  \n3. The authors propose a training-free hallucination mitigation method, which first prompts the evaluated LVLM to generate a description of the image guided by the given question, then asks the LVLM to answer the given question solely based on the description.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "Critical:\n1. Evaluate more up-to-date LVLMs.\n2. Compare the proposed methods with other training-free hallucination mitigation baselines.\n\nMinor:\n1. Using InternLM2 to represent InternLM-XComposer2 is confusing since there is also a pure language model named InternLM2.", "broader_impact_concerns": "No concerns about the ethical implications.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "Jd4Rs2Ah0e", "reviewer": "Reviewer_PXaj", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "## Strengths\n- **Innovative Evaluation Framework**. The triplet-based approach uniquely evaluates both object and relation hallucinations, addressing a gap in LVLM reliability assessment.\n- **Practical Mitigation Method**. A training-free technique to reduce hallucinations is introduced, making it immediately applicable without requiring additional model retraining.\n- **Benchmark**. The Tri-HE dataset, with 300 carefully curated images and manually verified questions, offers a reliable resource for testing hallucination detection.\n---\n\n## Weaknesses\n- **Limited Impact on Relation Hallucinations**. The paper presents a fresh perspective on relation hallucinations in VLMs, but its impact is limited by: (1) a focus on common concepts for relations, driven by Scene Graph dataset constraints, which restricts the number of relations (618 relations) and relation categories; (2) modern VLMs exhibits less hallucinations in common cases, while hallucinations are more prevalent in rare cases, yet the paper lacks results for rare cases; (3) scaling the dataset is challenging and limited due to its dependence on Scene Graphs, which itself is not scalable.\n- **Limited Benchmark Size**. With only 300 images, the Tri-HE dataset may not fully represent the diversity needed for a comprehensive evaluation across various scenarios.\n- **Reliance on Scene Graphs**. The evaluation framework depends on scene graphs for comparison, which could restrict applicability, especially for tasks without pre-existing annotations.\n- **Limited Experimental Focus**. The experiments focus on VQA tasks, with the proposed method reducing hallucinations in VQA through captioning. However, this approach does not scale to other VLM tasks, such as captioning.\n- **Limited Originality in Mitigation Approach**. The mitigation solution lacks novelty, as it mirrors approaches in prior work, essentially using a COT process to generate contextual understanding, such as caption, and include it when providing an answer.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "- **Unified Evaluation Framework**. A triplet-based method assesses object and relation hallucinations in LVLMs, using scene graphs for consistent, unbiased evaluation across models.\n\n- **Tri-HE Benchmark**. Tri-HE offers 300 images and 1,226 verified questions to rigorously test object and relation hallucination detection in LVLMs.\n\n- **Hallucination Mitigation Method**. A training-free technique cuts hallucinations via better modality alignment, using a two-step process with triplet emphasis for improved accuracy.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "No", "audience_explanation": "", "requested_changes": "* Enlarge the Tri-HE benchmark to three times its current size, ensuring the inclusion of rare relation categories.\n* Gather RL prompts and rewards training data using the expanded datasets. Execute the RL algorithm and report the results. Alternatively, for a cost-efficient approach, collect pairwise ranking data based on evaluation metrics and apply the DPO algorithm.", "broader_impact_concerns": "* The use of GPT-4 as an LLM judge may introduce biases from its training data, potentially skewing hallucination assessments. Without transparency, this could unfairly favor certain models, affecting fairness in AI development and deployment.\n* The reliance on proprietary models like GPT-4 for evaluation raises accessibility issues for researchers with limited resources. This could exacerbate inequalities in AI research, limiting diverse contributions to LVLM reliability improvements.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Decision: Accept (poster)