Reducing Hallucinations in Large Vision-Language Models via Latent Space Steering
Authors: Sheng Liu
Haotian Ye
James Zou
Venue: ICLR 2025 Spotlight (2025)
Keywords: Large Vision-Language Models
Multimodal large language model
Hallucination
Primary area: alignment, fairness, safety, privacy, and societal considerations

Abstract: Hallucination poses a challenge to the deployment of large vision-language models (LVLMs) in applications. Unlike in large language models (LLMs), hallucination in LVLMs often arises from misalignments between visual inputs and textual outputs. This paper investigates the underlying mechanisms of hallucination, focusing on the unique structure of LVLMs that distinguishes them from LLMs. We identify that hallucinations often arise from the sensitivity of text decoders to vision inputs, a natural phenomenon when image encoders and text decoders are pre-trained separately. Inspired by this, we introduce Visual and Textual Intervention (VTI), a novel technique designed to reduce hallucinations by steering latent space representations during inference to enhance the stability of vision features. As a task-agnostic test-time intervention, VTI can be easily applied to any problem without additional training costs. Extensive experiments demonstrate that it can effectively reduce hallucinations and outperform baseline methods across multiple metrics, highlighting the critical role of vision feature stability in LVLMs.

Published as a conference paper at ICLR 2025
REDUCING HALLUCINATIONS IN LARGE VISION- LANGUAGE MODELS VIA LATENT SPACE STEERING
Sheng Liu
Haotian Ye James Zou
Stanford University
{shengl, haot james (ettesss/staaford.edll
ABSTRACT
Hallucination poses challenge to the deployment of large vision-language models (LVLMs) in applications. Unlike in large language models (LLMs), hallucination in LVLMs often arises from misalignments between visual inputs and textual outputs. This paper investigates the underlying mechanisms of hallucination, focusing on the unique structure of LVLMs that distinguishes them from LLMs. We identify that hallucinations often arise from the sensitivity of text decoders to vision inputs, a natural phenomenon when image encoders and text decoders are pre-trained separately. Inspired by this, we introduce Visual and Textual Intervention (VTI), a novel technique designed to reduce hallucinations by steering latent space representations during inference to enhance the stability of vision features. As a task-agnostic test-time intervention, VTI can be easily applied to any problem without additional training costs. Extensive experiments demonstrate that it can effectively reduce hallucinations and outperform baseline methods across multiple metrics, highlighting the critical role of vision feature stability in LVL.Ms]
INTRODUCTION
Large Vision Language Models (LVLMs) (Liu et al. 2023b Zhu et al. 2023 Ye et al. 2023 Li
et al., 2023a Dai et al. 2023 Gong et al. 2023 Bai al.
2023b) have demonstrated impressive performance across various tasks such as image captioning (Li et al., 2023b Wang et al. 2023b).
visual question answering (Lee al. 2024: Wang et al. 2024a). medical treatment planning (Liu
et al. 2024b), and many more. LVLMs take advantage of both powerful vision encoders, such as CLIP (Radford et al., (02T) as well as text decoders, which are often pretrained large language models. LVLMs can present sophisticated understandings of vision information by mapping them to the language domain that LLMs can process.
Despite their remarkable success, LVLMs still encounter numerous challenges that impede their applications in weal-world tasks, among which hallucination (Liu et al.) 2023a Lovenia et al. 2023 Leng et al., 2023b Liu al. 2024a Deng et al. 2024 Zhu al. 2024) is prominent concern (Gunjal et al. 2023 Li ct al. 2023c). Hallucination in LVLMS emerges when the generated
textual responses include inaccurate descriptions of the input image (Li et al. 2023c), such as mentioning non-existing objects or characters, or providing incorrect spatial relations. Such kind of cross-modality inconsistency arises from different mechanism from that in standard LLMs, where vision information is missing and hallucination comes solely from the linguistics level. In LVLMs, the sequential relation of information flow from the vision encoder to the text decoder can distinctly contribute to hallucination, potentially essential question to be answered.
While a few existing studies have explored the underlying mechanism in this regard, leading to different conclusions such as statistical pre-training bias (Agarwal et al., (2020) Agrawal et al. 2016
Goyal al. 2017), over-reliance on language prior (Leng et al.) [023b]] Lee et al. [2033
Zhibo
et al. 2023 Han et al. 2022, Wu et al.) (022). and biased feature learning (Zhu et al. 2024 Huang
ct al. 2023 Yue et al. 2024), they focus mainly on issues that do not distinguish between LVLMS
and L.LMs. That is, the mechanisms of hallucination they present do not take into consideration the
particular structure of LVLMs that differs from LLMs.
"Code available at httt:p:://githhib..c//////gggg

---

Published as a conference paper at ICLR 2025
Captioning
Describe the image in details.
The image features a large clock [..] InInIn addition to the clock and the doorway. there two people visible scene. [-] The presence of these individuals suggests that [..]]
The image features a large clock [..]] The clock serves as functional and decorative element. The combination of the clock and the yellow tiles creates an aesthetically pleasing environment.
Text Detection
How much it per hour to park?
VTI
costs $4,25 per hour park the parking meter.
costs $4.00 per hour park at the parking meter.
Relation
Which truck (left right) has its door open?
The truck on the right has its door open.
The red fire truck on the left has door open.
Adversarial
How many people eating in this kitchen?
There are two people eating in this kitchen
There are no people eating in this kitchen: is empty.
Relation
What are the colors of the parachutes the sky?
VTI
The colors are blue, yellow, and orange.
The colors are blue, green, and orange.
Figure 1: Illustration of the effects of VTIon mitigating hallucination with LLaVA-1.5 the back- bone, Hallucinated contents generated by the original model are marked in red. In contrast, VTI re- sults in less hallucination across different categories of questions. Examples are obtained from MMHAL-Benchh (Sun et al. 2023) and CHAIR (Rohrbach et al. 2018)
This paper explores the mechanism of hallucination based on the sequential relation from vision encoder to the text decoder: demonstrate that hallucinations unstability the vision encoders. Similar to conventional neural networks in computer vision, an ideal vision encoder should output stable features when the semantics of images remain unchanged, e.g., after we perturb them mildly. This is especially the case for LVLMs where the image encoder and the text decoder are pre-trained separately and minimally fine-tuned together. The sensitivity of the text decoder to image features can exacerbate the misalignment between both modalities and results in hallucinations. We comprehensively present and analyze the correlation between vision feature stability and model hallucinations in Section 3
from the sensitivity the
naive way to reduce model hallucination is to smooth vision features by averaging across multiple perturbations of the original image every time we have new query. Unfortunately, this not only requires multiple forward pass the model which is extremely inefficient but also introduce extra noises to the original features, leading to worse preservation of information. To better leverage the correlation and reduce hallucinations without introducing additional side effects, inspired by works representation engineering for LLMs (Liu et al. Cao et al. 2024 Luo et al.) 2024: Zou et al.) 2023), where LLM's behavior is altered by editing the latent features, we pre-compute the direction that feature averaging has brought in the latent space and apply them edit the latent features of
2

---

Published as a conference paper at ICLR 2025
any new query. The pre-comp directions in the latent space capture the underlying changes as we perturb the query image and average vision embeddings across perturbations. This simple yet effective test-time intervention on vision, when combined with intervetion text, allows proposed method, namely Visual and Textual Intervention (VTI), to mitigate vision-language hallucinations. Importantly, use fixed 50 examples to pre-compute all intervention directions, and applied consistently across all benchmarks, suggesting VTI is task and dataset agnostic.
In summary, we (1) investigate the mechanism of hallucination by focusing on the sequential relationship between the vision encoder and text decoder, uncovering a correlation between the stability of visual features and hallucinations of LVLMs, (2) propose a novel method, VTI, which guides LVLMs toward less hallucinated outputs by steering them in the latent space, and (3) conduct extensive experiments to benchmark VTI against baseline methods. The leading results across multiple metrics demonstrate the high effectiveness of VTI in reducing hallucinations. We hope that our work can shed light on the critical role of feature stability in LVLMs and inspire future research to develop more effective strategies for reducing hallucinations.
RELATED WORK
Large Vision-Languee Models.
The success of Large Language Models (LLMs) et al.
2023 Touvron et al., 2023 Tay et al. 2022 Raffel et al. 2020
Brown et al. 2020 Chowdhery et al.
2022
Taori et al. 2023 Chiang et al., 2023 Bai ct al. 2023a) has fueled significant advancements
in Large Vision-Language angguge Models (LVLMs) (Liu et al. 2023h Dai et al.. 2023 Zhu et al. 2023
Li et al. 2023a Ye et al. 2023 Bai et al.] 2023b). By incorporating visual encoders and feature
projectors (Li al. 2019 Sun et al. 2019; Wang et 2022 Li et al. 2022). LVLMs have achieved
notable improvements across a variety of multimodal tasks, including image captioning (Li et al.
20235), visual question answering (Zhang et al. 2023a), and image segmentation (Lai et al. 22024). However, similar to LLMs, LVLMs are susceptible to generating ballucinations (Li et al. (023c].
which hinder their robustness and reliability in real-world applications. Our work seeks address this by mitigating hallucinations in LVLMs, thereby enhancing their utility and dependability in practical settings.
Hallucination in Vision-L.anguage Models and LVLMs. In natural language processing, halluci- nation refers to the generation of false or nonsensical content (Ji et al.
2023 /Zang et al.) 2023b Shi
al., 2023). This issue has recently drawn attention in multimodal models, where hallucinations
significantly impair performance (Wang et al. 2023a Zhao et al.
2023 Huang et al. 2023 Yue
cct al. 2024). To counter this, various methods have been proposed, primarily involving additional
training to align models with ground truth (Gunjal al. 2023 Liu al. 2023at Sun et al. 2023
Yin et al.) 2023) Zhou et al.] 2023 Zhai et al. 2023 Yue et al. 2024). However, such approaches often suffer from practical limitations, such as the need for extensive training and additional data.
In response, training-free methods have gained traction. These approaches rely on techniques like self-feedback correction (Lee et al., 2023 Yin
2023), leveraging auxiliary models for knowl-
edge integration (Wan et al. 2024 Deng et al. 2024 Zhao et al.
2024: Yang et al. 2024: Kim et al.)
2024). and refining the model's 's decoding process
Huang al. [023 Leng al.] 2022a Favero
et al. 2024 Zhang et al. 2024 Wang et al.
2024b). These methods typically focus on adjusting the
text decoder but are limited in addressing hallucinations originating from visual components.
In contrast, our approach targets hallucination reduction by directly steering the latent feature space, impacting both the vision encoder and text decoder. This strategy enables us to mitigate hallucinations arising from both vision and text-centric sources, making our method more comprehensive than those focused solely on refining the text output.
3 MECHANISM OF HALLUCINATION IN LVLMS
A large vision language model typically consists of a vision encoder and large language model as a text decoder. Given an image 6, a text input x such as a question or prompt about the image, the vision encoder first extracts visual information from the image vision feature vectors V {u,, U2,.... vn These vision features are then projected into the same embedding space as the text input, and both the vision and text embeddings are concatenated and passed to the text decoder
3

---

Published as a conference paper at ICLR 2025
Noise Type
Nose Type
0.5
Cousiaan Nobo
Handom Mack
Hallucination rate (CHOURS)
0.52
0.52
Geussian Noise
Random Mask
1ha Busirattonn wite (Chviit)))
055
1002003
0
5
Density
0.50
0.54
0.49
0.53
0.48
1
0.5
0.0
0.2
0.6 Variance Embbddiings
0.4
1.0
# pertutbations averaged
10
20
30
40
50
0.5
Diggnul Model
Gesao en hote:
Tandom Must
Figure 2: (Left) Distribution of vision feature stability when different types of noise injected into the raw images. The æ-axis represents the variance of features across 50 perturbations, and the y-axis represents the frequency. (Middle) Illustration of the correlation between object hallucination and vision feature stability. Averaging vision features across multiple perturbed images reduces hallucination (measured by CHAIR, details in Section as the number of perturbations averaged increases. (Right) Noise alone tends to increase ballucination, suggesting that the reduction of hallucination is not due the noise itself but the averaging process across perturbations.
to generate text outputs ruto-regressively. While LVLMs are known to hallucinate, the underlying causes of this phenomenon remain unclear, particularly in relation to the vision encoder's role in the process. In this section, we explore how the stability of vision features impacts hallucination.
We begin by highlighting a fundamental relationship between the vision and text components of large vision- lision-language models: the output of the vision encoder serves as the input to the language decoder. This sequential connection implies that the stability of the vision features plays a crucial role in the model's outputs and can influence the occurrence of hallucinations. To examine the connection between feature stability and object hallucination, we perturb raw images with various types noise and analyze the variance in the resulting feature distributions. Ideally, noise that does not alter an image's 's semantic content should have minimal impact on the vision features or the model's 's output, provided the vision encoder is robust and trained to capture semantic information effectively.
However, as shown in the left figure of Figure 2 while most vision features remain stable, approx- imately 15% exhibit significant variance, resulting in long-tailed distribution. These unstable features are closely tied to hallucinations, as the model becomes overly sensitive to these features, leading inaccuracies in its outputs. This connection is further demonstrated in the second figure of Figure 2 averaging vision features across multiple roise-injected images reduces hallucination as the number of perturbations increases, regardless of the type of noise injected. Importantly, as shown in the right figure, this reduction is not due to the noise itself but to the averaging process across perturbations, while noise alone tends to increase hallucination.
Although feature averaging can mitigate hallucination, it introduces significant drawbacks. Since vision encoders are trained on clean data, perturbing images compromises information extraction, leading to loss of detail. Moreover, averaging vision features requires multiple forward passes through the model, greatly increasing computational costs. Thus, our goal to develop a method that enhances vision feature stability and reduces hallucination efficiently, without sacrificing information or incurring excessive computational overhead.
METHOD
The experiments in the previous section demonstrate that features averaged across mildly corrupted images are robust and effective in reducing hallucinations, despite the side effects. avoid this, inspired by works representation engineering for LLMs (Liu et al. Cao et al. 2024 Luo et al.) 2024, Zou et al.) 2023). where LLM's behavior is altered by editing the features in the latent space during inference, we develop computationally efficient algorithm called visual and textual intervention (VTI) that can improve vision feature stability as well as text to image dependancy to reduce hallucination of LVLMM.. In particular, we pre-compute the "direction" of more stable features
4

---

Published as a conference paper at ICLR 2025
Step 1: Obtain visual and textual direction
Vision Encoder
Text Decoder
Viskn (eatures
There a oae-
There a dog.
Layer
Layer Layer 2
Layer
Layer
Principla direction from YPCA
A
Principle direction from PCA
Vieaa
Tectual directie Lien
dinotian
Step 2: Test-time intervention
Per token layer addition
Text Decoder
projector
tokenizer
Decaribe
Vision Encoder
Figure 3: Overview of the proposed algorithm visual and textual test-time intervention (VTI). Given example set {(U,, 22, ..,);;), where e; is the vision input and (z,, is paired captions with and without hallucination, VTI first runs the model on each query (G, T2, 5.) and records all hidden states. It then computes the shifting vectors dyision and d!at for all layer / and token according to Section During inference, the vectors are subsequently added to every layer of the vision encoder and text decoder, respectively, when processing new query. Notice that the vectors are task and dataset-agnostic i.e., they pre-computed using few samples from one specific task and dataset, and fixed unchanged throughout the entire experiments our paper.
and then apply them consistently query examples during inference to reduce hallucination, without introducing additional training or inference cost. As sometimes hallucination rise from the text decoder, i.e. the LLM,, we further obtain textual direction and apply it to the text decoder to maximize the performance. The overview of the proposed method is illustrated in Figure 3
Given a vision input v, let the latent states of the vision encoder for be represented as hi,,, where I E (1.2...., L} denotes the layer index in the encoder, and f {1,2.... T) represents the index of the vision tokens.
To enhance the robustness of the latent representation. we leverage random masking to create multiple perturbed versions of Specifically, apply different random masks, C,, 1,..., m. to producing m corrupted versions, (v(). For each perturbed input (u), the vision encoder generates (v))))
the corresponding latent states 11,

---

Published as a conference paper at ICLR 2025
Intuitively, robust latent embedding for can be approximated by averaging the embeddings obtained from the perturbed inputs. This averaged embedding is computed as:
C((o) L,
The visual shifting vector then defined as the difference between the robust averaged embedding and the original embedding:
Ai, bi, h/...
(1)
To make this shifting vector applicable to new image queries, we aim to remove image-specific information from A retaining only the general effect introduced by feature averaging. To achieve this, we compute the shifting vectors Ai for a of example images, {vi, UN)...,u By
stacking these vectors into a matrix,
A 12
AT
we extract the first principal direction of this matrix, denoted as d' This principal direction captures the dominant pattern of change introduced by feature averaging (an ablation study of the PCA dimensions is provided in T).le
It is worth noting that for each image, the random mask applied to generate the perturbed inputs C, (v) independently sampled, ensuring diversity in the perturbations and improving the generality of the computed shifting vector.
Apart from the vision token shifting, we further introduce the textual shifting vector that steers the latent states of the text decoder when generating model outputs. Obtaining the textual shifting vector is as simple as what previous work proposed aligning the style of LLMs; Following (Zhou et al. 2023). curated N image captions without hallucination, denoted z and adopt GPT model generate the hallucinated version द. As a result, we obtain paired captions with and without hallucination. We simply the original corresponding images the visual inputs. We then compute the textual direction for each of the sample as
Je: 1,8
(2)
where h4, stands for the hidden states of the t-th text token in layer I when generating the last token of outputs. In particular, since the text decoder is causally modeled, only use latent states of the last token, i.e. t = the last token. Similarly, we perform PCA to remove extra noise from a specific example choice obtain the overall direction dext.
We separately apply the visual and textual direction to Intervene vision encoder and the text decoer, ie. the LLM. Since the vision encoder not causally modeled, we shift the latent states of all layers of the vision encoder all token positions in the forward pass:
hi == hi + a dy,iion
(3)
and for the text, shift the latent states of the text decoder with the textual direction:
A R,8 Lit = 1 + 3 d1,t=last token
dext
As we will demonstrate in the experimental section below, such a combination of vision and language latent space intervention can address various styles of hallucination.
5 EXPERIMENTS
In this section, we empirically investigate the effectiveness of VTI in reducing hallucinations. Re- markably, we use 80 examples with paired images from COCO (Lin et al. 2014)), hallucinated and non-hallucinated responses to pre-compute the visual and textual direction and apply them to all tasks, datasets, and queries. This ensures the universality and generalizability of our results. We aim to answer the following questions: (1) Can visual intervention effectively reduce hallucination in LVLMs? (2) Can textual intervention effectively reduce hallucinations in LVLMs? (3) What the benefit of combining them?

---

Published as a conference paper at ICLR 2025
Model
LLaVA-1.5
InstructBLIP
Qwen- VL
Method
Accuracy T
FI Score t Accuracy
Fl Score f Accuracy T Fl Score
Vanilla
79.8
79.4
76.3
78.0
83.5
81.2
VCD
82.3
83.4
80.1
81.0
84.5
83.3
OPERA
84.2
83.7
79.6
80.9
84.3
82.6
VTI
86.5
85.9
81.8
83.2
85.2
84.1
Table 1: POPE hallucination evaluation results three different LVLMs. We report the average F1- score and accuracy averaged across three sub-tasks. The complete table can found in Appendix
5.1 EXPERIMENTAL. SETTINGS
Datasets. We evaluate model both discriminative and generative datasets, as listed below. More details about the datasets are provided in the appendix. (a) POPE: The Polling-basee Object Probing Evaluation (Li et al. 2023c) contains 27,000 Yes/No questions about object existence in MSCOCO (Lin et al. 2014). where the task to judge whether the given object is in the given image Following existing works, we compute accuracy, precision, recall, and Fl score for each method. (b) CHAIR: Caption Hallucination Assessment with Image Relevance (Rohrbach al. 2018) quantifies object hallucinations in image captions by comparing generated objects to ground-truth objects. Following previous works (Huang et al. 2023] Yue et al.) 2024), we randomly select 500 images from the MSCOCO dataset (Lin et al. 2014) and use CHAIR, CHAIRs, and Recall evaluation metrics. (c) MMHAL-
(examples are provided in Figure
(Sun et al. 2023 y This benchmark
evaluates LVLMS beyond object hallucination and contains eight question types: object attributes, adversarial objects, comparisons, counting. spatial relations, environment, holistic description, and others. We evaluate the hallucination rate and response informativeness using GPT-4.
Implementation Details. We evaluate the effectiveness of model three mainstream large vision-language models, including InstructBLIP (Dai et al. 2023), LLaVA 1.5 (Liu et al., 2023b) and VW (Bai et al. 2023b) with beam search the default decoding strategy (denoted as Regular). We also compare our model with state-of-the-an baseline methods: OPERA (Huang et al. 2023) and VCD (Leng et al. 2023b). These methods focus mitigating object hallucinations by improving the decoding strategy. We perform a grid search for the strength of vision and text vectors where a, {0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0). For bascline methods, we followed the settings in their papers and released code to ensure a fair comparison. More details are provided in the Appendix A
5.2 EXPERIMENTAL RESULTS
Results on POPE. We begin with the most extensively used benchmark for object hallucination. In Table 1 compare various decoding-based hallucination mitigation methods the POPE bench- mark (Li et al. 2023c), where "Vanilla" stands for the original model. Obviously, VTI outperforms the regular decoding strategies across all LVLMs consistently, resulting in the best accuracy and Fl score. In addition, VTI surpasses state-of-the-art contrastive decoding methods, demonstrating its effectiveness in mitigating object hallucinations. Notice that we average three different sub-tasks in POPE, and the comprehensive results can be found in Appendix
Results on Open-ended Generation with CHAIR Evaluation. Beyond the "Yes-or-No" discrimina- tive evaluations on POPE. we validate our model on open-ended caption generation using the CHAIR benchmark (Rohrbach et al. 2018). The results in Table demonstrate consistent improvement over the compared methods. Remarkably, visual shifting and textual shifting demonstrate effectiveness on different assessment dimensions: visual shifting is more effective reducing CHAIR, that calculates on image-level, while textual shifting is more effective in reducing CHAIR, that calculates on sentence-lewe Combining them (VTI, the last row) helps incorporate their benefits and effectively reduces object ballucinations in generated captions, evidenced by lower CHAIR and CHAIR, scores. addition, VTI also maintains the comprehensiveness the generated captions, as indicated by higher recall scores.
7

---

Published as a conference paper at ICLR 2025
Model
Method
CHAIRs +
CHAIR,
Recall f
Avg. Len
Vanilla
51.0
15.2
75.2
102.2
DOLA
57.0
15.9
78.2
97.5
VCD
51.0
14,9
77.2
101.9
LLa
OPERA
47.0
14.6
78.5
95.3
Vision only
43.2
12.7
78.6
93.4
Text only
41.0
12.9
78.3
92.2
VTI
35.8
11.1
76.8
93.8
Vanilla
54.0
18.1
71.1
115.3
DOLA
60.0
20.1
71.5
110.8
VCD
57.0
17.0
72.1
112.1
InstructBLIP
OPERA
54.0
12.8
69.8
93.6
Vision only
49.1
12.1
72.5
104.2
Text only
48.7
14.2
72.1
98.7
VTI
43.4
11.8
70.1
105.8
Table 2: CHAIR hallucination evaluation results with corresponds to less hallucination, and higher recall is better. We bold the best results and underline the second-best results. VTIand its variations consistently have the lowest hallucination rate with similar recall.
tokens 512. Smaller CHAIR
Overall
Other
Attribute
1.3
1.0
Holistic
Ad Adversarial
Environment
Compaaiiso
Relation
Counting
Original model
Text only
Vision only
Text Vision
Figure 4: Detailed performance of different models on the eight categories in et al. 2023), where "Overall" indicates the averaged performance across all categories. A higher score indicates that the generated response contains fewer hallucinations and more information.
(Sun
Results on MMHAI--Bench. To extensively test our method on various vision tasks, we benchmark VTI on the MMHAL dataset with the evaluations of eight different hallucination types. Comparison with existing methods are shown in Table 3 and fine- iine-grained ablation is presented Figure 4 VTI achieves much higher score (corresponds to less hallucination) across all categories. This underscores its effectiveness addressing a broader range of multimodal hallucination challenges beyond objects. We also observe that textual and visual shifting are complementary. Visual shifting performs better on vision-centric tasks such as recognizing and comparing attributes of objects. In contrast, textual shifting is good reducing hallucination of textual-centric tasks such as answering adversarial questions and counting, where relatively more complex language reasoning is required.
5.3
ANALYSIS
Can visual shifting improve feature stability? As we discussed in Section 3 simply averaging vision embeddings across multiple perturbed images reduce hallucination. VTI serves as "soft"
8

---

Published as a conference paper at ICLR 2025
Average scoee
Attribute
Adversarial
Compariion
Counting
Relation
Environment
Vanilla
1.99
D.62
2.58
1.08
2.58
1.67
2.00
3.00
1.17
1.83
VCD
2,69
0.58
3.25
2.17
3,00
2.42
2.58
3.25
2.42
2,42
2,90
Table 3: Results MMHAL-Bench (Sun et al. 2023)
with LLaVA-1.5. Our proposed method
significantly reduces hallucinations in different categories.
alternative the naive averaging, and it remains unclear whether the algorithm indeed improve feature stability. To show this, we apply various types of pertubations to images, including random mask, Gaussian blur, random noise, random brightness adjustment, and random elastic transform. For each of the settings, we compute the feature variance (across 100 perturbations) and average across different features as the metric of feature stability. We then compare the feature stability with/without applying VTI in Figure 5 As expected, the vision features obtained from the model with vision intervention exhibit lower variance and better stability, suggesting the effectiveness of the vision direction in smoothing out the vision features.
Original
Model with vision intervention
Variance
Random Gaussian
Gaussian
Elastic
Brightness
Figure 5: Feature stability is improved across different types of image corruptions with vision intervention.
Textual intervention increases attention dependency toward images. It has been observed that LVLMs can occasionally disregard input image's information but treat the next-token prediction as purely text-based continuation task (7hu et al. 2024). Text intervention, by pushing the latent space features to the non-hallucination direction, implicitly reduces such text-biased hallucinations. Figure 6 illustrtatss that textual intervention is able to reduce average attention from generated text to text while increasing attention to vision tokens, suggesting higher reliance on the image when generating text.
Combining visual and textual intervention enhance the level of detail in genera- tions. A straightforward, though unremarkable, approach to reducing hallucinations is to generate less content. However, illustrated in Figure combining visual and textual interventions offers the advantage of achieving similar hallucination reduction without the need to shorten the generation length.
Why visual intervention more effective than simple vision feature averaging? In Section we illustrate the effectiveness of averaging features to mitigate hallucination, however, adding random noise to the input images and averaging the corresponding features may result in loss of information. As we demonstrated in Figure g performing linear probing on the simple averaged features results in only around 62% classification accuracy while applying visual intervention with different strength a resulting in improvement of visual feature stability (expressed in lower feature variance) without significantly hurting the probing accuracy, suggesting better information preservation.
Ablation of and 3. Eventually, study the influence of vector strength applied hidden states. Figure X presents the results of an ablation study on a and 3, which controls the strength of visual and textual intervention, respectively. Figure illustraas that B 0, implying

---

Published as a conference paper at ICLR 2025
Original model
0.50
Text Orly
Model text intervention
Vision only
(CHAIRs)
Average attention
0.20
0.46
0.15
0.44
0.10
0.05
0.00
80
85
90
95
100
105
text to vision
text to text
Length generations
Figure 6: (Left) Textual intervention reduces the self-attention from text to text tokens and increases the self-attention the vision tokens. (Right) Combining vision and text intervention achieve similar hallucination rates but with longer generations.
intervention, results in suboptimal performance on CHAIRs. As a and / increase, improvements in the hallucination rate observed, highlighting the effectiveness of the visual and textual intervention.
CONCLUSION
In conclusion, our exploration into mitigating object hallucinations in LVLMs through latent space steering has yielded promising results. By implementing visual and textual intervention, we have significantly reduced hallucinations without compromising the models' ability to generate detailed and contextually accurate outputs. Our findings underline the importance of robust feature representation and its pivotal role in enhancing model reliability. As LVLMs continue evolve, we believe the strategies outlined in this paper will serve foundational steps toward creating more accurate, trustworthy, and efficient systems, fostering broader applicability in real-world scenarios. We are optimistic that future advancements will build these insights, bridging the gap between human-like understanding and machine- machine-generated interpretations.
ETНIC AND REPRODUCIBILITY STATEMENT
We propose the visual and textual intervention method address hallucination issues LVLMs, thereby enhancing their safety and reliability within the community. Additionally, the datasets utilized for inferring and evaluating our method are publicly accessible, promoting transparency and reproducibility in research. Furthermore, will make our code available to the public once accepted, ensuring it is convenient for researchers and practitioners to access and implement.
REFERENCES
Vedika Agarwal. Rakshith Shetty, and Mario Fritz. Towards causal vqa: Revealing and reducing spurious correlations by invariant and covariant semantic editing. Proceedings of the IEEF/VV Conference on Computer Vision and Pattern Recognition, pp. 9690-9698, 2020.
Aishwarya Agrawal, Dhruv Batra, and Devi Parikh, Analyzing the behavior of visual question answering models. arXiv preprint arXiv: 1606.0735, 2016.
Jinze Bai, Shuai Bai, Yunfei Chu, Zeyu Cui, Kai Dang, Xiaodong Deng, Yang Fan, Wenbin Ge, Yu Han, Fei Huang, et al. Qwen technical report. arXiv preprint arXiv:2309. 16609, 2023a.
Jinze Bai, Shuai Bai, Shusheng Yang, Shijie Wang, Sinan Tan, Peng Wang, Junyang Lin, Chang Zhou, and Jingren Zhou. Qwen-vl: frontier large vision- lisiun-language model with versatile abilities. arXiv preprint 2308 /2966, 2023b.
10

---

Published as a conference paper at ICLR 2025
Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners. Advances in neural information processing systems, 33:1877-1901, 2020.
Zouying Cao, Yifei Yang, and Hai Zhao. Nothing excess: Mitigating the exaggerated safety for Ilms via safety-conscious activation steering. arXiv preprint arXiv:2408. 1491, 2024.
Wei-Lin Chiang, Zhuohan Li, Zi Lin, Ying Sheng. Zhanghao Wu, Hao Zhang, Lianmin Zheng, Siyuan Zhuang, Yonghao Zhuang. Joseph E. Gonzalez, Ion Stoica, and Eric P. Xing. Vicuna: An open-source chatbot impressing gpt-4 with 90%* * chatgpt quality, March 2023. URL https
oxsyss 300022222000000---aaaaaa viiunaa
Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung. Charles Sutton, Sebastian Gehrmann, et al. Palm: Scaling language modeling with pathways. arXiv preprint arXiv:2204.023111 2022.
Wenliang Dai, Junnan Li, Dongxu Li, Anthony Meng Huat Tiong, Junqi Zhao, Weisheng Wang. Boyang Li, Pascale Fung, and Steven Hoi. Instructblip: Towards general-purpose vision- vision-language models with instruction tuning. arXiv preprint arXiv:2306. 04387, 2023.
Ailin Deng, Zhirui Chen, and Bryan Hooi. Seeing is believing: Mitigating hallucination in large vision-language models via clip-guided decoding. arXiv preprint 24fi::2402.133000. 2024.
Alessandro Favero, Luca Zancato, Matthew Trager. Siddharth Choudhary, Pramuditha Perera, Alessandro Achille, Ashwin Swaminathan, and Stefano Soatto. Multi-modal hallucination control by visual information grounding. arXiv preprint arXiv:2403.1 14003, 2024.
Fabrizio Gilardi, Meysam Alizadeh, and Maël Kubli. Chatgpt outperforms crowd-workers for text-annotation tasks. arXiv preprint arXiv:2303. 15056, 2023.
Tao Gong, Chengqi Lyu, Shilong Zhang, Yudong Wang, Miao Zheng. Qian Zhao, Kuikun Liu, Wenwei Zhang, Ping Luo, and Kai Chen. Multimodal-gpt: A vision and language model for dialogue with humans. arXiv preprint arXiv:2305.04799. 2023.
Yash Goyal, Tejas Khot, Douglas Summers-Stay, Dhruv Batra, and Devi Parikh. Making the v in vqa matter: Elevating the role of image understanding in visual question answering. In Proceedings of the IEEE conference computer vision and pattern recognition, pp. 6904-6913, 2017.
Anisha Gunjal, Jihan Yin, and Erhan Bas. Detecting and preventing hallucinations in large vision language models. arXiv preprint 1rXiv:2308.06394 2023.
Yudong Han, Liqiang Nic, Jianhua Yin, Jianlong Wu, and Yan Yan. Visual perturbation- collab- orative learning for overcoming the language prior problem. arXiv preprint arXiv:2207.11850, 2022.
Qidong Huang, Xiaoyi Dong. Pan Zhang, Bin Wang, Conghui He, Jiaqi Wang. Dahua Lin, Weiming Zhang, and Nenghai Yu. Opera: Alleviating hallucination in multi-modal large language models via over-trust penalty and retrospection-alloe alllocacioon..lloaaionn..o arXiv preprint arXiv:231 11111..799111111 2023.
Ziwei Ji, Nayeon Lee, Rita Frieske, Tiezheng Yu, Dan Su, Yan Xu, Etsuko Ishii, Ye Jin Bang, Andrea Madotto, and Pascale Fung. Survey of hallucination in natural language generation. ACM Computing Surveys, 55(12):1-38, 2023.
Junho Kim, Yeon Ju Kim, and Yong Man Ro. What if...?: Counterfactual inception to mitigate hallucination effects in large multimodal models. arXiv preprint arXiv:2403.13513 2024.
Xin Lai, Zhuotao Tian, Yukang Chen, Yanwei Li, Yuhui Yuan, Shu Liu, and Jiaya Jia. Lisa: Reasoning segmentation via large language model. In Proceedings the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 9579-9589, 2024.
the
Jusung Lee, Sungguk Cha, Younghyun Lee, and Cheoljong Yang. Visual question answering instruction: Unlocking multimodal large language model to -omaii--peccfic visual multitasks. arXiv preprint 2402.083600 2024.
11

---

Published as a conference paper at ICLR 2025
Seongyun Lee, Sue Hyun Park, Yongrae Jo, and Minjoon Seo. Volcano: mitigating multimodal hallucination through self-feedback guided revision. arXiv preprint arXiv:2311.07362, 2023.
Sicong Leng, Hang Zhang, Guanzheng Chen, Xin Li, Shijian Lu, Chunyan Miao, and Lidong Bing. Mitigating object hallucinations in large vision-language models through visual contrastive decoding, arXiv preprini 1.Niv:23/1.16920 2023a.
Sicong Leng, Hang Zhang, Guanzheng Chen, Xin Li, Shijian Lu, Chunyan Miao, and Lidong Bing. Mitigating object hallucinations in large vision-language models through visual contrastive decoding, arXiv preprint urXiv:2311.16922 2023b.
Bo Li, Yuanhan Zhang, Liangyu Chen, Jinghao Wang. Jingkang Yang, and Ziwei Liu. Otter: A multi-moda model with in-context instruction tuning. arXiv preprint arXiv:2305.03722) 2023a.
Junnan Li, Dongxu Li, Caiming Xiong, and Steven Hoi. Blip: Bootstrapping language-iman pre- training for unified lision-language understanding and generation. In International Conference on Machine Learning, pp. 12888-12906 PMLR, 2022.
Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. Blip-2: training with frozen image encoders and large language models. arXiv preprint urXiv:2301.12597, 2023b.
Bootstrapping language-image pre-
Liunian Harold Li, Mark Yatskar, Da Yin, Cho-Jui Hsieh, and Kai-Wei Chang. Visualbert: A simple and performant baseline for vision and language. arXiv preprint arXiv:/908.00 033507 2019.
Yifan Li, Yifan Du, Kun Zhou, Jinpeng Wang, Wayne Xin Zhao, and Ji-Rong Wen. Evaluating object hallucination large vision-language models. arXiv preprint arXiv:2305. 10355, 2023c.
Tsung-Yr Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and Lawrence Zitnick. Microsoft coco: Common objects in context. In Computer Vision- ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part V 13, pp. 740-755. Springer, 2014.
Fuxiao Liu, Kevin Lin, Linjie Li, Jianfeng Wang, Yaser Yacoob, and Lijuan Wang. Mitigating halluci- nation in large multi-modal models via robust instruction tuning. arXiv preprint arXiv:2306. 14565, 2023a.
Hanchao Liu, Wenyuan Xue, Yifei Chen, Dapeng Chen, Xiutian Zhao, Ke Wang, Liping Hou, Rongjun Li, and Wei Peng. A survey on hallucination in large vision-language models. arXiv preprint 200.:22402.00553,, 2024a.
Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning. arXiv preprint arXiv:2304 300.0088848 2023b.
Sheng Liu, Haotian Ye, Lei Xing, and James Y Zou. In-context vectors: Making in context learning more effective and controllable through latent space steering. In Forty-first International Conference on Machine Learning.
Sheng Liu, Oscar Pastor-Serrano, Yizheng Chen, Matthew Gopaulchan, Weixing Liang, Mark Buyyounouski, Erqi Pollom, Quynh- Thu Le, Michael Gensheimer, Peng Dong, et al. Automated radiotherapy treatment planning guided by gpt-4vision. arXiv preprint 20Xi::440.156000 2024b.
Holy Lovenia, Wenliang Dai. Samuel Cahyawijaya, Ziwei Ji, and Pascale Fung. Negative object presence evaluation (nope) to measure object hallucination in vision-language models. arXiv preprint arXiv:2310.053300 2023.
Jinqi Luo, Tianjiao Ding, Kwan Ho Ryan Chan, Darshan Thaker, Aditya Chattopadhyay, Chris Callison-Burch, and René Vidal. Pace: Parsimonious concept engineering for large language models. arXiv preprint 2xXvv24466.43311, 2024.
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International conference on machine learning, pp. PMLR, 2021.
12

---

Published as a conference paper at ICLR 2025
Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter Liu. Exploring the limits of transfer learning with a unified text-to-text transformer. The Journal of Machine Learning Research, 21(1):5485-555 2020.
Anna Rohrbach, Lisa Anne Hendricks, Kaylee Burns, Trevor Darrell, and Kate Saenko. Object hallucination image captioning. arXiv preprint 1rXiv:1809.022 2018.
Weijia Shi, Sewon Min, Michihiro Yasunaga. Minjoon Seo, Rich James, Mike Lewis, Luke Zettle- black-box language models. arXiv preprint
moyer, and Wen-tau Yih. Replug: arXiv:2301.
2023.
Chen Sun, Austin Myers, Carl Vondrick, Kevin Murphy, and Cordelia Schmid. Videobert: A joint model for video and language representation learning. In Proceedings of the IEEF/CVF international conference on computer vision, pp. 7464-7473, 2019.
Zhiqing Sun, Sheng Shen, Shengcao Cao, Haotian Liu, Chunyuan Li, Yikang Shen, Chuang Gan, Liang- Yan Gui, Yu-Xiong Wang, Yiming Yang, et al. Aligning large multimodal models with factually augmented rlhf. arXiv preprint arXiv:2309.14525 2023.
Rohan Taori, Ishaan Gulrajani, Tianyi Zhang, Yann Dubois, Xuechen Li, Carlos Guestrin, Percy Liang, and Tatsunori B. Hashimoto. Stanford alpaca: An instruction-folloiinng Ilama model, 2023.
Yi Tay, Mostafa Dehghani, Vinh Q Tran, Xavier Garcia, Jason Wei, Xuezhi Wang, Hyung Won Chung, Dara Bahri, Tal Schuster, Steven Zheng, et al. U12: Unifying language learning paradigms. In The Eleventh International Conference on Learning Representations, 2022.
Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. Llama: Open and efficient foundation language models. arXiv preprini arXiv:2302. 13971, 2023.
David Wan, Jaemin Cho, Elias Stengel-Este -skiin, and Mohit Bansal. Contrastive region guidance: Improving grounding in vision-language models without training. arXiv preprint arX:v:2403.02325, 2024.
Haibo Wang, Chenghang Lai, Yixuan Sun, and Weifeng Ge. Weakly supervised gaussian con- trastive grounding with large multimodal models for video question answering. arXiv preprint arXiv:2401.10711, 2024a.
Jianfeng Wang, Zhengyuan Yang, Xiaowei Hu, Linjie Li, Kevin Lin, Zhe Gan, Zicheng Liu, Ce Liu, and Lijuan Wang. Git: A generative image-to-ter transformer for vision and language. arXiv preprint 2tth:22255141000 2022.
Junyang Wang, Yiyang Zhou, Guohai Xu, Pengcheng Shi, Chenlin Zhao Haiyang Xu, Qinghao Ye. Ming Yan, Ji Zhang, Jihua Zhu, et al. Evaluation and analysis of hallucination in large vision-language models. arXiv preprint arXiv:2308. 15/26, 2023a.
Teng Wang, Jinrui Zhang, Junjie Fei, Yixiao Ge, Hao Zheng, Yunlong Tang, Zhe Li, Mingqi Gao, Shanshan Zhao, Ying Shan, et al. Caption anything: Interactive image description with diverse multimodal controls. arXiv preprint arXiv:200..02677 2023b.
Xintong Wang, Jingheng Pan, Liang Ding, and Chris Biemann. Mitigating hallucinations in large vision-language models with instruction contrastive decoding. arXiv preprint 2rXiv:2403.1877 2024b.
Yike Wu, Yu Zhao, Shiwan Zhao, Ying Zhang, Xiaojie Yuan, Guoqing Zhao, and Ning Jiang. Overcoming language priors in visual question answering via distinguishing superficially similar instances. arxiv preprint 2rX.022209.08529, 2022.
Dingchen Yang, Bowen Cao, Guang Chen, and Changjun Jiang. Pensieve: Retmospec-tthen-commae mitigates visual hallucination. arXiv preprint arXiv:2403. 14401, 2024.
Qinghao Ye, Haiyang Xu, Guohai Xu, Jiabo Ye, Ming Yan, Yiyang Zhou, Junyang Wang, Anwen Hu, Pengcheng Shi, Yaya Shi, Chenliang Li, Yuanhong Xu, Hehong Chen, Junfeng Tian, Qian Qi, Zhang, and Fei Huang. mplug-owl: Modularization empowers large language models with multimodality. arXiv preprint arXiv:2304.141777 2023.
13

---

Published as a conference paper at ICLR 2025
Shukang Yin, Chaoyou Fu, Sirui Zhao, Tong Xu, Hao Wang. Dianbo Sui, Yunhang Shen, Ke Li, Xing Sun, and Enhong Chen. Woodpecker: Hallucination correction for multimodal large language models. arXiv preprint arXlv:2310.16045, 2023.
Zihao Yue, Liang Zhang, and Qin Jin. Less is more: Mitigating multimodal hallucination from an eos decision perspective. arXiv preprint arXiv:2402. 14545, 2024.
Bohan Zhai, Shijia Yang. Xiangchen Zhao, Chenfeng Xu, Sheng Shen, Dongdi Zhao, Kurt Keutzer, Manling Li, Tan Yan, and Xiangjun Fan. Halle-switch: Rethinking and controlling object existence hallucinations in large vision language models for detailed caption. arXiv preprini arXiv:2310.01779. 2023.
Xiaoman Zhang, Chaoyi Wu, Ziheng Zhao, Weixiong Lin, Ya Zhang, Yanfeng Wang, and Weidi Xie. Pin--vq: Visual instruction tuning for medical visual question answering. arXiv preprint 1005 2023a.
Yi-Fan Zhang, Weichen Yu, Qingsong Wen, Xue Wang. Zhang Zhang, Liang Wang, Rong Jin, and Tieniu Tan. Debiasing large visual language models. arXiv preprint arXiv:2403.05262, 2024.
Yue Zhang, Yafu Li, Leyang Cui, Deng Cai, Lemao Liu, Tingchen Fu, Xinting Huang, Enbo Zhao, Yu Zhang, Yulong Chen, et al. Siren's 's song in the ai ocean: A survey on hallucination in large language models. arXiv preprini arXiv:2309.01219 2023b.
Linxi Zhao, Yihe Deng, Weitong Zhang, and Quanquan Gu. Mitigating object hallucination in large vision-language models via classifier-free guidance. arXiv preprint arXiv:2402.086600 2024.
Zhiyuan Zhao, Bin Wang, Linke Ouyang, Xiaoyi Dong, Jiaqi Wang, and Conghui He. Beyond
hallucinations: Enhancing Ivlms through hallucination-awar direct preference optimization. arXiv preprint ArXiv:2311. 16839, 2023.
Ren Zhibo, Wang Huizhen, Zhu Muhua, Wang Yichao, Xiao Tong, and Zhu Jingbo. Overcoming language priors with counterfactual inference for visual question answering. In Proceedings of the 22nd Chinese National Conference on Computational Linguistics, pp. 600-610, 2023.
Yiyang Zhou, Chenhang Cui, Jaehong Yoon, Linjun Zhang, Zhun Deng, Chelsea Finn, Mohit Bansal, and Huaxiu Yao. Analyzing and mitigating object hallucination in large vasion-language models. arXiv preprint arXiv:2310.0077 2023.
Deyao Zhu, Jun Chen, Xiaoqian Shen, Xiang Li, and Mohamed Elhoseiny. Minigpt-4: En- hancing vision-language understanding with advanced large language models. arXiv preprint arXiv:2304. 105522 2023.
Lanyun Zhu, Deyi Ji, Tianrun Chen, Peng Xu, Jicping Ye, and Jun Liu. Ibd: Alleviating hallucinations in large vision- languaee models via image-biased decoding. arXiv preprint 28X4 2024.
Andy Zou, Long Phan. Sarah Chen. James Campbell, Phillip Guo, Richard Ren, Alexander Pan, Xuwang Yin, Mantas Mazeika, Ann-Ke Mat-Kat Dombrowski, et al. Representation engineering: top-down approach to ai transparency. arXiv preprint arXiv:2310.01405, 2023.
A DETAILED EXPERIMENTAL SETTINGS
In all experimental setups, the mask ratio to compute visual direction set to 0.99, and we average across 50 random masks. For experiments on CHAIR, maintain similar lengths of generations, set a = 0.4 for visual intervention only, and similarly, B = 0.4 for textual intervention only. For VTI. 3 0.4. For experiments on MMHAL-Benchh we adopted
0.9 and 0.9.
POP We utilize the official benchmark from Li et al. (023c). which includes 3,000 question-answer pairs for each of the random, popular, and adversarial settings. We use the query
14

---

Published as a conference paper at ICLR 2025
template 'Is there [object] in the image?". Here, [object] is selected randomly, from the most frequent objects in the dataset, or from objects that frequently co-occur with [object], corresponding to the random, popular, and adversarial settings respectively. We evaluate the performance based on whether the model-generated output contained the ground truth ("Yes' or "No") using accuracy, precision, recall, and average Fl-score.
We select 500 random images from the COCO Lin et al. (2014) validation set and generate the output using the query "Please Describe this image in detail.". Due the computational complexity, we restrict the max new tokens to 64. Following the M3ID Favero et al. (2024), we report
CHAIR
two assessment metrics, CHAIR, and C which calculate the hallucination ratio per sentence and instance as follows:
CHAIR, (sentences with hallucinated objects]|
[haalluinattd objects}|
.CHAIR
(5)
all sentences)
alll objects mentioned]]
MMHAL--Renc In MMHAL--Benh dataset, 96 image-question pairs, ranging in 8 question categories x 12 object topics are included. The question categories are
Object attribute: LVLMs incorrectly describe the visual attributes of invididual objects, such as color and shape.
Adversarial object: LVLMs answers questions involving something that does not exist in the image, instead pointing out that the referred object cannot be found.
Comparison: LVLMs incorrectly compare the attributes of multiple objects.
Counting: LVLMs fail to count the number of the named objects.
Spatial relation: LVLMs fail understand the spatial relations between multiple objects in the response.
Environment: LVLMs make wrong inference about the environment of the given image.
Holistic description: LVLMs make false claims about contents in the given image when giving comprehensive and detailed description of the whole image.
Others: LVLMs fail recognize the text icons, or incorrectly reason based the observed visual information
Following Sun et al. 2023 use GPT-4 to evaluate different methods MMHAL-BENCE and to analyze and rate responses. Prompts we used for MMHAL-BENCH evaluation can be found in (Sun al. 2023
The rating scales are
6, very informative with good analysis or reasoning, no hallucination
5, very informative, no hallucination
4, somewhat informative, no hallucination
3, not informative, no hallucination
2, very informative, with hallucination
1, somewhat informative, with hallucination
0, not informative, with hallucination
and if the rating < 3, the generation considered with hallucination.
http://
15

---

Published as a conference paper at ICLR 2025
B ADDITIONAL EXPERIMENT RESULTS
B.1 DETAILED RESULTS ON POPE
Random setting: is there an botte the image?
Popular setting: Is there an knitle the image?
Adversarial setting: there in the image??
Figure 7: Example questions in different settings of the POPE dataset
We present the detailed performance on POPE in Table VTI exhibits significant performance im- provement with VL-15- InstructBLIP, and Qwen- VL. These results underscore the effectiveness of VTI in mitigating object hallucination.
B.2 DETAILED RESULTS ON MMHAL-BENCH
We present the detailed performance MMHAL-Bench in Tablenpresents the detailed performance metrics on MMHAL-Benchh VTI demonstrates substantial improvements across various tasks, consistently outperforming both VCD and OPERA in all question categories except for the 'Other' category. These results highlight the effectiveness of VTI reducing hallucinations beyond object- specific errors, reinforcing its potential to enhance LVLMs' accuracy in interpreting and analyzing visual content comprehensively.
B.3 MORE RESULTS ON ANALYSIS
In this section, we provide additional figures (Figure on the analysis section in the main text. Visual shifting better preserves vision information (measured by linear probe accuracy on ImageNet) than naive feature averaging while reducing feature variance.
Vieual retanvntioon
regpt awergging
57.4
40.2
42.2
35.2
72
5.
probe
70
43.4
27AA
68
64
21.5
2:8
62
0.37
variance
Figure 8: (Lefft) Trade-off between robustness and information preservation. Visual shifting improves feature robustness while preserving information. a controls the strength of the shifting. (Right) Ablation study and B.
C EXAMPLES FOR COMPUTING THE TEXTUAL DIRECTIONS.
To generate ballucinated captions for experiments, curated training set of 50 examples using the methodology outlined (Zhou et al.] 2023). Specifically, we leveraged GPT-3.5's in-context learning capabilities automatically generate hallucinatory data for refinement. First, GPT-3.5
16

---

Published as a conference paper at ICLR 2025
Setting
Model
Decoding
Accuracy
Precision
Recall
Fi Score
Regular
83,49
88.84
76.76
82.36
OPERA
87.53
94.52
79.80
86.53
LLaVA1.5
VCD
86.84
87.15
86.68
86.91
VTI
89.50
94.38
84.00
88.89
Regular
80,42
78.93
83.21
81.01
OPERA
85.07
88.39
80.73
84.39
InstructBLIP
VCD
84.11
84.20
85.36
84.78
Random
VTI
86.37
88.83
85.68
87.22
Regular
84.33
95.98
71.67
82.06
OPERA
85.13
95.54
74.13
83.48
Qwen- VL
VCD
85.53
95.17
74.87
83.81
VTI
86.73
93.67
78.80
85.59
Regular
79.98
82.47
76.76
79.31
OPERA
84.21
88.00
79.80
83.70
LLaVAI.5
VCD
82.65
87.15
80.59
83.74
VTI
87.36
91.61
82.27
86.69
Regular
76.09
73.22
82.94
77.78
OPERA
78.33
73.85
87.73
80.19
InstructBLIP
VCD
79.94
77.84
83.33
80.49
Popular
VTI
81.86
80.17
85.68
82.83
Regular
84.17
96.13
71.20
81.81
OPERA
84.73
96.52
73.13
83.21
Qwen- VL
VCD
85.63
94.28
75.87
84.08
VTI
85.67
92.06
78.06
84.48
Regular
76.03
76.11
76.80
76.45
OPERA
80.88
82.16
79.76
80.94
LLaVA1.5
VCD
77.31
73.43
86.47
79.42
VTI
82.57
84.33
80.01
82.11
Regular
72.37
68.78
83.06
75.24
OPERA
75.50
70.49
87.73
78.17
InstructBLIP
VCD
76.32
73.24
84.08
78.29
Adversarial
VTI
77.29
74.09
85.67
79.46
Regular
81.93
91.30
70.60
79.62
OPERA
82.93
91.15
73.31
81.26
Qwen-VL LL
VCD
82.19
90.14
75.53
82.19
VTI
83.13
88.20
77.67
82.16
Table Experimental results POPE (Li et al.
2023c
Method
Hallucination Rate (%)
LLaVA-1.5 115 (base)
61.46
VTI (PCA dim 1)
51.04
VTI (PCA dim 3)
51.04
VTI (simple average)
57.96
Table VTI performance with different PCA dimensions. Experiments are conducted on MMHAL- Bench.
was prompted to produce a list of objects that frequently co-occur with the ones mentioned given description. Then, using LLa5 generated descriptions for the 50 training images, incorporating a randomly selected word from the "co-occuring objects" list and another from the "uncertain objects" list. This process allowed create rich dataset of hallucinated captions. For further details on the prompts, refer to (Zhou et al. 2023). Figure shows an example of generated caption.
17

---

Published as a conference paper at ICLR 2025
The image shows a young man sitting on pile of luggage while traveling on public transportation, likely a passenger train or bus. He has puzzled look on his face he tries manage his belongings on this crowded journey. There are multiple suitcases, a handbag, and backpack nearby, indicating that the man has considerable amount of luggage with him. Apart from the man sitting on his luggage, there are a few other people the scene as well, some sitting on benches while others stand in the space. The two benches available are located on either side of the man sitting with his luggage. Additionally, there are handbags placed on the floor in the same area, suggesting that other passengers also have their belongings with them.
Original Caption:
Generated Hallucinated Caption: The image shows young man with puzzled look on his face as he tries to manage his multiple suitcases, handbag, and backpack while sitting on pile of luggage during crowded journey public transportation. Other passengers are seen with their own handbags and belongings nearby. One of the passengers is seen holding water bottle, while it is uncertain what the man front of the young man is doing or why he is smiling.
Figure 9: Illustration of the generated hallucinated captions for textual direction computing.
18


Reviews:
[{"review_id": "fyFv3iXXqW", "reviewer": "Reviewer_EA1M", "summary": "The paper introduces a heuristic approach of reducing hallucination in Visual Language Models by linearly offsetting the latent representation of both image and language modalities. The key to this heuristics is to pre-compute a set of offset vectors for image/text token embeddings that are the results of data augmentation. In particular, for image modality, the author introduced perturbations to the image input and computed the average change (averaged across all perturbed image instances) in token embedding across each layer of the image encoder. At inference time, the first PC of these change vectors across multiple images were then added to the embedding of any query image. Similar procedure was repeated for language modality where the “perturbation” is hallucinatory captions generated by another LLM. The authors show that the method, while simple, is able to significantly reduce the degree of hallucination in several benchmarks.", "questions": "Beyond addressing some of the questions I raised under \"Weakness\" above, here are a few other changes/questions.\n\nMinor:\n1. In Figure 2, CHAIRs was first mentioned. It should be described in the caption. Furthermore, the three subplots don’t seem to correspond to one another. It’ll be helpful to highly what experiments were shown in left/middle plots (sigma value and ratio value) to connect them to the third figure.\n2. Discussion of the methodology could be improved. In particular, the description of the “visual direction” around equation 1 is confusing. \n\nQuestions:\n1. What do the embedding changes actually look like? How much explained variance is the first PC? \n2. I don’t really understand how this linear offset can improve “feature stability”. If the embedding is to be viewed as a random variable whose variance is caused by input perturbation, then doing a linear offset _cannot_ change the variance of such random variable. Perhaps I missed something.", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 4, "strengths": "1. The problem of hallucination in VLM is significant and the method proposed is highly intuitive (and appears effective). It uses ideas from data augmentation and applies it directly to inference time without the need to retrain the model. \n2. The analysis in the paper is thorough. Not only does it include good ablation of the various aspects of the method (granted there aren’t many moving parts to begin with), it also included discussions of how the method could reduce hallucination (i.e. impact on attention)", "weaknesses": "All in all, this was a good paper and a pleasure to read. To nitpick, one of the biggest concerns of heuristic methods of this kind is its general applicability in practice. For example, \n1. what type of perturbations are most important in image/text modalities? \n2. how are the vision/language directions differ between models? \n3. What about more general open-ended QAs that go beyond captioning tasks? How would you even generate these hallucinatory language examples when the task is open QA?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "N8RJ3KgDrz", "reviewer": "Reviewer_Cwnc", "summary": "The hallucination problem of LVLM refers to the phenomenon where the text generated by the model does not align with the content of the image. Previous methods mainly focus on issues that do not distinguish between LVLMs and LLMs. The authors first discovered that hallucination mainly stems from the sensitivity of the text decoder (LLM backbone) to vision input.\n- By adding perturbations (Gaussian noise or random masking) to the input images, the authors found that the image encoder's generated embeddings exhibited some unstable features.\n - After perturbing the images, the model's hallucination increased, but if the embeddings were averaged over multiple perturbed images, stabilizing the features, the hallucination was reduced.\n\nInspired by this, the authors proposed a test-time intervention method called Visual and Textual Intervention (VTI).\n  - For a given model, the method requires a sample set where each sample includes an image, a hallucination-free caption, and a hallucinated caption.\n  - The images in the sample set are perturbed and sent to the vision encoder, calculating the visual shifting vectors.\n  - Additionally, the two captions are fed into the text decoder, calculating the textual shifting vectors.\n  - During testing, a new sample is processed by the model as usual, but the shifting vectors are added to the latent hidden states of the vision encoder and text decoder, respectively. \n\nVTI effectively reduced hallucinations and outperformed previous methods across multiple benchmarks.", "questions": "Please see the Weaknesses part.", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "- The motivation and the method are explained smoothly and easy to understand.\n- The experiments cover different hallucination benchmarks and LVLMs with different vision encoders.\n- As a task-agnostic test-time intervention, the proposed method, VTI, outperforms previous baseline methods.", "weaknesses": "### Major\n- The description of method in Section 3 is kind of too general. \nHow many LVLMs are tested here? Which LVLM(s) is used? \nAre the test images from COCO? \nAre the observations generalized to different LVLMs?\n- VCD [1], which has been cited and compared in the paper, also adds Gaussian noise to distort visual input. They mention “visual uncertainty amplifies hallucination” in their Figure 2, which might be related to “vision feature instability” explored in the paper. I’m wondering if the authors are inspired by VCD. It would be better if the authors can explain the differences between their exploration and VCD.\n- I’m concerned with the generalization ability of VTI, which may be a common problem of existing hallucination mitigation methods. LVLMs are expected to do a wide range of tasks. When image or task domains are changed, will the current methods damage model performance? Only hallucination benchmarks are covered in the experiments, so I'm not sure if VTI will damage model performance on other tasks. \n\n### Minor\n- It would be better to early explain the structure of LVLM in abstract or introduction, like “A large vision language model typically consists of a vision encoder and a large language model as a text decoder” in Lines 143 – 144. Currently, the paper refers to vision encoder and text decoder directly at the beginning, which I feel may be confusing to readers.\n- The authors claim that “VTI can be easily applied to any problem *without additional cost*” in the abstract. I feel “*without additional cost*” is kind of confusing, because VTI actually needs additional data and tune its shifting vectors.\n- Lines 285-286, the authors mention that “we use 50 examples with paired images, hallucinated and non-hallucinated responses to …”\nI’m wondering where the authors get the images. Are the images from COCO?\n- If the authors use more dimensions of PCA in calculating the shifting vectors, will the method get better or worse performance?\n\n- Typos\n\nIn the abstract, the abbreviation “(LLMs)” is shown twice. In the main content, no “(LLMs)”. The instruction needs to borrow one from the abstract.\n\nLines 151-152, “… between the vision and text components of large vision-language models (LVLMs):” It is not necessary to emphasize the abbreviation here now that there is \"LVLMs\" in the previous content.\n\nLine 214 “As the desirable visual direction should be readily applied to new image queries.” The sentence has no main clause. I guess the authors wanted to use “as”, following the previous equation. The same with Line 278.\n\nMissing “,” in Equation (1)\n\n\n\n----------------------------------------------------------\n[1] Mitigating object hallucinations in large vision-language models through visual contrastive decoding. CVPR2024.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "XCxTbYP9p3", "reviewer": "Reviewer_eTNv", "summary": "The authors propose that hallucinations in LVLMs are typically caused by misalignment between visual and text representations. Consequently, the authors demonstrate this conclusion by perturbing the input image to obtain a more robust visual representation. Based on this experimental approach, the authors introduce a comprehensive method for intervening in both image and text representations to mitigate hallucinations in LVLMs. Furthermore, the authors substantiate the effectiveness of their method through extensive experiments.\n\nThe paper makes the following contributions:\n\n1. Analyzes the correlation between the stability of visual representations and hallucinations in LVLMs.\n\n2. Proposes a method for separately perturbing text and image representations to achieve a more robust representation space.\n\n3. Demonstrates the effectiveness of their method through extensive experiments.", "questions": "1. Are there other more fundamental methods to prove representation stability? For example, calculating the KL divergence of representations after adding various perturbations to the image.\n\n2. To demonstrate that VTI, as an inference intervention method, is superior to OPERA and VCD, a fairer experimental setup would be to use VTI separately with beam search and nucleus sample to compare against OPERA and VCD, respectively. This would be similar to the experimental setup in PAI[1].\n\n3. What would be the effect of using average to calculate $d_{l, t}$ compared to PCA?\n\n4. When the mask ratio is set to 0.99, almost no image information is retained. Performing PCA on the representation of such input images seems to be pointless. Therefore, I believe it would be beneficial to include a comparison of what happens when the mask ratio is set to different values and when a fully masked image is directly set as the intervention direction.\n\n[1] Paying More Attention to Image: A Training-Free Method for Alleviating Hallucination in LVLMs", "limitations": "", "rating": 6, "confidence": 4, "soundness": 2, "presentation": 4, "contribution": 3, "strengths": "1. The writing of this paper is exceptionally clear. Additionally, the motivation is thoroughly argued, and the analysis of the method is comprehensive.\n\n2. It is indeed a novel attempt to mitigate hallucinations in LVLMs by perturbing the representation space through a training-free approach.\n\n3. The experiments in this paper are very comprehensive, demonstrating the effectiveness of the proposed method.", "weaknesses": "1. Using variance changes to explain changes in representation stability is somewhat one-sided. For instance, in PCA, PC1 is typically the direction with the largest variance, and it seems inevitable that the variance would decrease after using VTI. It seems more fundamental to examine the stability of the representation distribution for images with different noises or masks.\n\n2. There are issues with the comparison methods used in the experiments. OPERA and VCD are both methods based on decoding to mitigate hallucinations, with the former optimizing beam decoding and the latter optimizing nucleus sample decoding. However, the paper does not specify the exact settings for the decode methods of vanilla and VTI.\n\n3. Some components need to undergo ablation experiments.\n  \n    3.a. The role of PCA lacks ablation experiments. In Figure 8, it appears to be comparing different methods of representation updating, but what would be the impact if average were used instead of PCA?\n\n    3.b. In line 713, the mask ratio is set to 0.99, which is an unusually high value, but the author does not provide a detailed analysis of this parameter.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Spotlight)