Interpreting and Editing Vision-Language Representations to Mitigate Hallucinations
Authors: Nicholas Jiang
Anish Kachinthaya
Suzanne Petryk
Yossi Gandelsman
Venue: ICLR 2025 Poster (2025)
Keywords: Vision language models
hallucinations
logit lens
interpretability
Primary area: interpretability and explainable AI

Abstract: We investigate the internal representations of vision-language models (VLMs) to address hallucinations, a persistent challenge despite advances in model size and training. We project VLMs’ internal image representations to their language vocabulary and observe more confident output probabilities on real objects than hallucinated objects. We additionally use these output probabilities to spatially localize real objects. Building on this approach, we introduce a knowledge erasure algorithm that removes hallucinations by linearly orthogonalizing image features with respect to hallucinated object features. We show that targeted edits to a model’s latent representations can reduce hallucinations by up to 25.7% on the COCO2014 dataset while preserving performance. Our findings demonstrate how a deeper understanding of VLMs’ latent representations can enhance reliability and enable novel capabilities, such as zero-shot segmentation.

Published as a conference paper at ICLR 2025
INTERPRETING AND EDITING VISION-LANGUAA REPRESENTATIONS TO MITIGATE HALLUCINATIONS
Nick Jiang", Anish Kachinthaya", Suzie Petyrk!, ,Yssii Gandelsman University of California, Berkeley {nickj, anish,, spetryk, yossi gandelLmman/ybeereeeeyyyyo
ABSTRACT
We investigate the internal representations of vision-language models (VLMs) to address hallucinations, a persistent challenge despite advances in model size and training. We project VLMs' internal image representations to their language vocabulary and observe more confident output probabilities on real objects than hallucinated objects. We additionally use these output probabilities spatially localize real objects. Building on this approach, we introduce knowledge erasure algorithm that removes hallucinations by linearly orthogonalizing image features with respect to hallucinated object features. We show that targeted edits to a model's latent representations can reduce hallucinations by up to 25.7% on the COCO2014 14 dataset while preserving performance. Our findings demonstrate how a deeper understanding of VLMs' latent representations can enhance reliability and enable novel capabilities, such zero-shot segmentation.
INTRODUCTION
Vision-Language Models (VLMs) have recently emerged as powerful tools for understanding images via text (Dai al., 2023; Liu et al., 2024a). They have demonstrated remarkable capabilities across multimodal tasks such as image captioning (Li et al., 2023a), visual question answering (Ye et al., 2023), and complex multimodal reasoning (Bai et al., 2023). Despite their capabilities, VLMs tend to hallucinate content that does not appear in the images (Ji et al., 2023), which poses serious concerns for the reliability of these models real-world applications (Hu et al., 2023; Luo et al., 2024).
Widespread belief has been that scaling to larger models and more training data will naturally mitigate hallucinations. However, recent studies have shown that hallucinations persist even larger and more advanced models (Rohrbach et al., 2019; Li et al., 2023b), suggesting that this issue cannot be solved by scale alone. Current methods reduce hallucinations by applying external interventions (e.g. object detectors; Yin et al. (2023)) additional model fine-tuning (e.g, on hallucination examples; Zhou et al. (2024); Zhang et al. (2024a)). Nevertheless, these methods often struggle to distinguish between subtle hallucinations and existing details, requiring models updated model parameters.
In this paper. we aim to introduce fine-grained edits directly to the image latent representations of VLMs to reduce hallucinations without hindering their performance, approach that has had some success in large language models (Zhang et al., 2024b; von Rutte et al., 2024). To edit the latent representations of VLMs, first explain their role via text. We employ the logit lens technique (nostalgebraist, 2020) to directly interpret the spatial VLM image representations with VLM text vocabulary. Surprisingly, the characteristics of these image representations different for real objects that appear in the image and objects that are hallucinated. Moreover, the logit lens enables spatially localizing objects within the input image.
Relying on the ability to detect hallucinated objects, we edit them out by intervening in their internal representations. We introduce a knowledge erasure algorithm, PRoJECTAWAY, objects by linearly orthogonalizing image features with respect to the text features of target objects. We find that PROJECTAWAY can erase both real and hallucinated objects with high rates of removal.
target and remove
"Equal contribution first authors.
I EEqul contribution as last authors.
Code: https: htt//gttuuuu http:::///..tttt.cc///eaaaaaaaa http.//c//iicciann//////////

---

Published as a conference paper at ICLR 2025
(a) Vision-Language Model
(b) Project Latents to Vocabulary
(c) Object Removal
Cat
and dog<end>
Cat <end>
44.96% couch
Language Model
41.01% table
Language Model
Layor
60.32% couch 36.11% table
Layer
94.14% cat 2.01% bear
Cat dog
Cat
Input image
Edit and replace image representations:
edit
edit
Wu Unembedding Matrix
63.63% cat 32.86%
edi
"dog" text
Latent Representations
representation
edit
Figure I: Interpreting VLM internal image representations. (a) Given a VLM, (b) unembed the latent representations from image embeddings to the vocabulary and classify hallucinations. We remove hallucinations by (c) linearly editing them out of the latent representations.
We use our interpretation and editing approach for three tasks, First, we utilize the logit lens on image features to detect hallucinations in the image. We find that it improves mAP by 22.45% and 47.17% 17% in two VLMs. Then, we combine our editing and detection method to erase hallucinations from the VLM's internal representations, reducing hallucinations up to 25.7% on standard benchmarks, while preserving accuracy in image captioning. Finally, we use the logit lens to localize objects in image features. We find that our spatial mapping provides comparable performance to state-of-the-an zero-shot segmentation methods. Our results indicate that understanding the internal representations of VLMs can be achieved and used to repair model hallucinations and introduce new capabilities.
2 RELATED WORK
2.1 INTERPRETING LATENT REPRESENTATIONS IN LANGUAGE MODELS
Interpreting the inner workings of large language models enables fine-grained improvement of the language model behavior. Recent work involves utilizing the model's attention maps (Kobayashi et al., 2020; Chefer et al., 2021), activation patterns (Conmy et al., 2023; Meng et al., 2023; Bronzini al., 2024), and latent representations (Ghandeharioun et al., 2024; Cunningham et al., 2023; Bricken et al., 2023) to understand their behavior with applications such as early exiting (Halawi et al., 2024) and editing or erasing the model's knowledge (Dai et al., 2022; Ravfogel al., 2024). One class of methods prohe the VL.Ms knowledge with linear classifiers (Hewitt & Manning, 2019; Tucker et al., 2021: Li al., 2024; Belrose al., 2023). The logit lens method (nostalgebraist, 2020), which we will use in our analysis, finds the output distribution over the vocabulary of the language model at intermediate layers with the model's own unembedding matrix. We apply this approach VLMs to interpret the model's 's understanding of visual information in the model's 's textual vocabulary.
2.2 INTERPRETING LATENT REPRESENTATION IN VISION MODELS
Understanding the internal dynamics of vision models is critical for ensuring safety and reliability in multimodal systems. Early works in this area focused on producing saliency maps (Petsiuk et al., 2018), analyzing individual neurons (Bau et al., 2020; 2019; Dravid et al., 2023), and training networks to map latent representations to concepts (Esser et al., 2020). With the emergence transformer- based vision models like CLIP (Radford et al., 3021), recent methods explain latent tokens (Chen et al., 2023) and the roles attention heads and neurons with natural language (Gandelsman et al., 2024b;a). Few works currently interpret the internal computation of VLMs: Palit et al. (2023) develop a neuron causal tracing tool; Schwettmann et al. (2023) identifies modlt neurons; and Huo al. (2024) ablates domain-specific neurons to improve vision question-answeringg Whereas past.
2

---

Published as a conference paper at ICLR 2025
papers have primarily studied the mechanisms (e.g. neuron analysis) that drive VLMs, we focus on interpreting and editing their latent representations for real-world applicability.
2.3 DETECTING AND REDUCING VLM HALLUCINATIONS
While VLM performances on image caption and visual question answering are continually improving. they continue hallucinate facts that not supported by the visual input. Existing methods for detecting hallucinations in language models during inference utilize latent representations (He et al., 2024; Su et al., 2024), activations (Chen al., 2024), and output logit values (Varshney al., 2023). SAPLMA (A (Azaria & Mitchell, 2023) trains a hallucination classifier on the internal latent representations. LUNA (Song et al., 2024) learns transition function on latent representations and identifies abnormal transitions. Varshney et al. (2023) uses the final layer logits to score the model's 's confidence an entity or keyword and intervenes by instructing the model to either repair or remove the hallucinated information. Among VLMs, LURE (Zhou et al., 2024) is fine-tuned revisor model to detect and reduce hallucinations. OPERA (Huang et al., 2024) uses the model's s internal attention weights detect and suppress patterns that align with the beginning of hallucinated phrases. In contrast to these methods, we leverage the internal image representations in the VLMMs for hallucination reduction and for zero-shot segmentation.
EXTRACTING KNOWLEDGE FROM VLMS
We start by introducing VLMs and the general framework of their architectures in most recent work. We then describe our approach for decoding the features in intermediate image representations in VLMs into text, and apply it to two types of VLMs. Surprisingly, this approach effectively probes the knowledge about objects present images and can localize objects within the image.
3.1 PRELIMINARIES
Vision-Language Models. The architecture of recent state-of-the-arr VLMs for text generation typically involves three main components: a vision encoder to process image inputs, a mapping network map image features image embeddings, and an autoregressive language model to process the image embeddings and prompt embeddings to generate text. We focus on two recent state-of-the-art VLMs: LLa VA 1.5 (Liu et al., 2024a) and InstruciBLIP (Dai et al., 2023). We use 7B versions of both these models. LLaVA utilizes frozen CLIP vision encoder and MLP as a mapping network to project the vision encoder outputs into image embeddings for the language model. The MLP is pre-trained on large lision-language dataset and both the MIP and the language model are fine-tuned on an instruction-focuse dataset. In contrast, InstructBI.I IP freezes both the vision encoder and the language model and only trains the mapping network.
For the purposes of our work, we define the VLM architecture as follows. The vision encoder processes an input image to produce image features. These image features projected to embedding space via the mapping network, resulting in n d-dimensional image embeddings (k; k E Rd, = 1....n)) n}. For the language model, the entire set of text tokens constitutes the vocabulary V with vocabulary size VV. The image embeddings, followed by m text embeddings (4:44 14 E Rd i 1..... m) of the prompt tokens, are input to the language model through decoder layers. For an input embedding E Rdd, we define hr(x) E Rd to be the latent representation for embedding r at layer I {l,..., L). the output of the decoder layer, which conditioned on previous tokens of the input sequence. An unembedding matrix Wo E R V|xd maps the last latent representation (1m) to probability distribution over the vocabulary for the next token 4m+1.
Notations.
Logit Lens. Logit Lens is an interpretability method for intermediate language model representa- tions introduced in Section 2.1. The logit lens technique applies the unembedding matrix Wo to latent representations (x) in the intermediate layers in the language model to retrieve the logit distributions over the vocabulary.
((mm) Wo hr((m)) Ingit,,/ogit,, logit,, .egitt....,logiiyyyl
(1)
This is the logit distribution representing the predictions of the model after / layers, where logit, corresponds to the token in the vocabulary.
3

---

Published as a conference paper at ICLR 2025
X10
InstructBLIP
x10
LLaVA
Not in Image
Not in Image
Frequency
In Image
Frequency
In Image
10-3
10
10a
10"
Internal Confidence
Internal Confidence
Figure 2: Comparison of internal confidence in objects present and not present in the image. We examine the internal confidence of COCO objects that exist and not exist the image within intermediate VLM image representations. We observe that objects that do not exist in the image have lower internal confidence.
3.2 APPLYING LOGIT LENS ON VLMS
We apply the logit lens to probe the language model as it processes the image representations. This enables to interpret the image features' output distributions as they are transformed by the layers of the language model and localize objects spatially within the image.
Extracting probability distributions from intermediate image representations. We apply logit lens on the image representations in the VLM. For a given image embedding k,, we find the latent representation of the image embedding at layer 1, hy (k;), taking the logit lens get the probability distribution over the vocabulary, softmax( (k:)). ) We define an object o, an object word composed of tokens from the vocabulary. We inspect the probability of a specific object o, softmax (f(())) For multi-token objects, we take the maximum probability value over the object tokens. This provides a generalizable framework for analyzing specific latent image representations via text, with respect to specific objects. Next, we find the maximum probability over all image representations over all layers. For object o, we compute:
max
We define 6, as the VLMMs internal confidence of an object existing in the image: the highest probability of object presence across image representations through L layers of the language model.
Comparing the internal confidence of present and not present objects. determine if internal confidence provides meaningful information about objects in the image, examine for objects present and not present in an image. We use InstructBLII and LLaVA to caption 5000 random COC02014 images in the Karpathy validation split (Lin et al., 2015) and determine c, for all 80 COCO objects, only a few of which are present in each image. Since there are many more objects present than present, present. Figure 2 exhibits the internal confidences for objects present and not present in the image. We empirically find that the VLMs' internal confidences are higher for present objects than not present ones. We use this claim later to classify objects as hallucinations in Section 5.1.
randomly sample subset of the internal confidences for objects
Object localization. Given that the language model can distinguish between objects present and present in an image, we examine whether it can attribute high object internal confidence to specific patches in an image. For each image embedding k; in n image embeddings, we find the maximum softmax probability of an object within the layers of the model, max sISI (softmax(fft (k::))))) Using these internal confidence values, we localize the objects the image patches, each of which maps to an image embedding. We focus LLaVA for this task, since its image encoder preserves the spatial mapping of image patches to image features.
We observe that image representations that exhibit higher internal confidence for specific objects correspond to the image patches in which those objects are visually present (examples in Figure 3). Building our previous observation, see that the intermediate image representations semantically align with latent token representations of objects present in them while maintaining their spatial locality. We use this unique finding for zero-shot segmentation in Section 5.3.
4

---

Published as a conference paper at ICLR 2025
Input image
"cat" probabilities
"cat" localization "bicycle" probabilities "bicycle" localization
Image Emboediing Index
Layer
Input image
"bottle" probabilities
"bottle" localization
"bowl" probabilities
"bowl" localization
Emnbwoiing Index:
Image Emmbecding Index R
Layer
Figure 3: Localizing objects using internal confidence values. We find the probabilities of objects through layers of the language model for every image embedding in LLaVA. We use the highest layer probability per image embedding localize an object within the image.
While the model is not directly trained to map the image representations closer to the text represen- tations of objects within them, can unembed the image representations in the text vocabulary for localization and find differences in internal confidence for present and hallucinated objects. In Section 5.1, will this observation for various applications including hallucination detection and zero-short segmentation.
4 ERASING KNOWLEDGE FROM VLMS
Recognizing that image embeddings are directly interpretable (Section 3.2), we edit these embeddings to erase the presence of objects from image captions. We propose linear editing algorithm that subtracts the text embedding of target object from all image embeddings. When applied on singular and multiple object removals, we find that erases hallucinated objects more effectively than correctly detected (CD) objects (i.e. real objects that the model correctly detects).
4.1 ERASING OBJECTS FROM IMAGE REPRESENTATIONS
We present an algorithm, PROJECTAWAY (Fig- 4), that orthogonalizes image representations with respect to text representations order to erase objects in image captions, applying it to remove objects one at a time and all at once.
Algorithm I: PROJECTAWAY
Input: A set of image embeddings K, text em- bedding f, and weight factor a A set of modified image embeddings K' projected away from the text embedding Initialization: K' +- for & E K do
Output:
Given an image and object remove, we edit the latent representations hy (k;) at a hidden layer I' across all image embeddings k,. We do not modify any latent representations outside of those belonging to image features. We compute the dot product, text embedding i, subtracting weighted / from by (k;) only if the dot product is positive. 1, PROJECTAWAY is equivalent to orthogo- nalizing the image representations with respect to the text representation. To compute text rep- resentation E, we pass the object (e.g. "hot dog") into the VLM's ' text model and extract her (t.)) at hidden layer (", where t. is the last token of the object. We use the last token the object to capture the whole of the object's meaning.
p + K
if p 0 then
K' K' K'U
p ||
.1)
of by (ks) and the object's
else
K' ('
end if end for
Figure 4: Our editing algorithm erases the pres- ence of object from image embeddings by or- thogonalizing them with respect to the object's text embedding.
5

---

Published as a conference paper at ICLR 2025
Edit Scope
Model
Individual RR ( (%
Mass RR (%)
CD change (%)
C, 4
C, 4
InstructBLIP LLaVA
15.0
54.1
No edits
14.6
51.1
InstructBLIP 83.3
74.3
+0.07
8,94
33.2
Hallucinations
LLaVA
86.0
72.8
+0.01
11.2
35.5
InstructBLIP
16.2
15.0
-2.2
17.3
58.3
LLaVA
6.9
8.3
-1.6
15.2
52.4
Table 1: Removing mentioned objects individually & in-mass. Using PROJECTAWAY, hallucinated objects and observe high hallucination reduction with CHAIR, mass-removal rate (Mass RR), and individual removal rate (Individual RR). We also remove correctly detected (CD) objects but find that they are more resistant to linear editing. Denote CHAIRs as Cs and CHAIR, as C₁.
4.1.1 REMOVING OBJECTS ONE ONE
We evaluate the PROJECTAWAY algorithm's effectiveness at erasing individual objects from captions multiple images and objects.
Experimental setting. We apply PROJECTAWAY on 5000 random images from the COCO2014 training set on all mentioned COCO objects (i.e. hallucination and CD) individually and measure the removal rate at which objects no longer appear in the caption. For InstructBLIP, we set (1, (1, 2, 1.5). For LLaVA, we (1', a)) (19, 21, 3.5). These parameters are fixed irrespective of image and are chosen for their maximal effect (see ablations in Section 4.2). To differentiate hallucinations from CD, compute CHAIR (Rohrbach et al., 2019), an evaluation criteria that compares model-generated captions to ground-trum -ruth human annotations. CHAIR provides two main scores, CHAIR, and CHAIRs, that quantify hallucinations for instances and sentences, respectively:
CHAIR |captions with hallucinated objects}|
(hallueinated objects) }
CHAIR,
(3)
aall captions]
fall objects mentioned]
Results. Table shows that PROJECTAWAY is significantly more effective in erasing individual hallucinated objects at an individual level than CD objects for both InstructBLIP and LLaVA. Along with the insight that hallucinated objects have lower softmax scores (Figure 2), these results suggest that hallucinated objects manifest more weakly in image embeddings and are hence easier to remove than CD objects.
4.1.2 MASS-REMOVING OBJECTS
We iteratively apply PROJECTAWAY set of objects, following the experimental setup and observing similarly different removal rates for hallucinated objects and CD objects.
Mass-removing hallucinations. We mass-remove hallucinations identified with ground truth an- notations using PROJECTAWAY. Table shows that editing out all the hallucinations of an image yields similar removal rate individually editing out and, importantly, that erasing hallucinated objects together does not interfere with each other. We achieve hallucination reduction rate of 41.3% for InstructBLII and 23.3% for LLaVA (see Table 4). Recall count slightly increases for both models, indicating that caption accuracy is preserved. This may be because removed hallucinations are replaced with objects the model is more confident in. Qualitative results are Figure 5.
Mass removing CD. We similarly find that applying PROJECTAWAY can successfully remove CD objects when edited all together in Table Furthermore, CHAIR scores minimally change, which indicates that this mass-removal merely erases object presence without eroding caption accuracy. While the removal rate lower than for hallucinated objects, this insight proves useful when we apply PROJECTAWAY for hallucination reduction in Section 5.2.
4.2 ABLATION STUDY: MASS-REMOVING HALLUCINATIONS
We perform ablations on parameters of PROJECTAWAY to improve object removal rate for crasing hallucinations in-mass.

---

Published as a conference paper at ICLR 2025
Before: the Image, group of four young men are gathered around a pianic table, posing far photo. They are holding frisbess in their hands and seem be having good time logether. located
Before: The image features delicious siice pizza sitting a white paper piato. The pizza is sopped with various ingredients, including pesperoni, olives, and mushrooms, making mouth watenin adding the prasentation pizza.
Before: The image features black and white cat sitting in the middl of a cluttened living roam flaor. The cat is surrounded by various items such as ciiiees, books, and backpack, There side and the other two people visible adsition cot, there side of on right seem engrossed their own activites, possibly unoware of cat's presence.
chairs placed
The providing central
didlee of chairs group's the table, indicating
The
the
gathering
to several items
person standing person
group may been enjoying meal socializing play no the frisbes. car seen
pizza
the table, including two cups,
room. Both individuals
items add overall stmesphere the scone, creating cozy and nutting setting for anjoying the pizza.
knife, spoon.
parked the background, possitly bclonging one of the group members.
The image depicts black and white cat sitting in the middle a clttered room. The cat is surrounded by a variety of items, inclucing sultcases, backpacks, clothas, and shoes. There are at least thren sultcases scattered around the room, with one located closer to the cat and the other two further away. A backpack can be seen on the left side of the room, and a pair of shoes can be spotted on the right side. In addition to these items, there are sevaral diotnes spread out on the floor, including shirt jacker, and pair of pants. The cluttered enwironment
After:
After: group four young men are gathered around a picnic table, posing for photo while holding frisbees. They soom to ba onjoying thair time together, possibly playing a friandly game of frisbee. The picnio table located in the midcle of the scene, providing a convenient spot for the group to gather and play. In addition to the trisbes, there are several backpacks scattered around the area, suggesting that the group may have come prepared acventure.
After: The image features a deliciou ous slice of pizza on paper plate. piaced on dining table, The pizza is topped with vartous ingredients, inclucing pepperoni, olives, and mushrooms, making it mouth-watering meal. The pizza diice dominates the scene, taking up most of the space on the paper plate. The plate placed top of dining seen in background.
outdoor
table, which
that the recently used packing presaring
may have trip.
Figure 5: Qualitative results for mass object removal. We present example images and their captions after mass-removing hallucinations (red) with PROJECTAWAY. which can effectively remove hallucinations while preserving, even increasing, correctly detected objects (green).
Experimental setting. We ablate the three parameters of PROJECTAWAY: layer I' to edit at, layer I" to retrieve the text representation, and weight factor a. At 1" = -1, we average together the object's constituent token embeddings. At 1' = 11, we edit the image embeddings directly inputted to the text model. We evaluate across 500 training samples from COCO 2014 that have at least one hallucination.
Hidden layers. Figure 6 shows hallucination reduction rate on LLaVA from mass-removing halluci- nations on every combination of It and 1f (each from -1 to 31). As a core concern that editing erodes caption accuracy, we gray out any combination that reduces CD objects. For InstructBLIP (see Figure 10), the best parameters (1 = 1, 1f = 2) reduces hallucinations by 38.5%. For LLaVA, our best parameters (!' = 19, 1" = 21) reduce hallucinations by 25.7% and the middle layers the best to edit and extract latent text embeddings from. Our results also provide a wide range of reasonable parameter alternatives use if this reduction rate does not generalize beyond our samples.
Weight factor. Using the best-reduced hidden layers, we ablate the weight factor a for PROJECT- AWAY across the same 500 randomly selected COCO images. Figure 7 shows that as a increases, hallucinations are removed at a higher rate, and the overall hallucination count drops significantly. At high a, we observe through anecdotal examples that captions become nonsensical, as quantitatively shown by the complete loss of both correctly detected and hallucinated objects from the caption. Therefore, as a pre-caution, we only select weight factors that do not reduce CD objects when we apply PROJECTAWAY to erase hallucinated objects.
5 APPLICATIONS
5.1 HALLUCINATION DETECTION
When extracting knowledge from VLMs in Section 3.2, we found that applying logit lens on in- context image representations exhibit useful information about visual objects present in the image. Using these observations, we present an approach for object presence classification that only relies on the VLMs own parameters. We utilize the internal confidence c, value to classify object presence,
7

---

Published as a conference paper at ICLR 2025
23 25 27
Percentage change
Edited hidden layer
11 13 17
40-
Hallucinations
Correctly detected
Mass removal rate
13 17 Text embedding hidden layer
Weight factor
Figure 6: Hidden layer ablations for LLaVA. We track hallucination reduction (%) across dif- ferent layers to edit at and extract latent embed- dings for the text embedding, crossing out (red) parameters from consideration where there is a decrease in correctly detected objects.
Figure 7: Weight ablations for LLaVA. We vary the weight factor a and measure changes in correctly detected objects, removal rate, and hallucination reduction. We observe decline in hallucinations as weight grows and mark weight where there is no loss in correctly detected objects.
InstructBLIP
LLavA
ROC Curve
PR Curve
ROC Curve
PR Curve
True Positive Rate
True Positive Rate
Precision
Precision
AUC)
(AP= 92)
(MUC=0.68 68)
(AP=0.07)
Baselne (AUC=0.555)
Baseline (AP=0.76)
Baseline (AUC-0.44)
Baseline (AP=O. 74)
False Positive Rate
Recall
False Positive Rate
Recall
Figure 8: Object Presence Classification Curves for InstructBLIP and LLaVA. We show the
Precision-Reed Recaal and ROC curves of our confidence measure for present object- soject-hallucciation clas- sification on the COCO training subset. Classifying object presence with the internal confidence outperforms the baseline, indicating that the language model's image representations know which objects are hallucinations and which are truly present.
since the internal confidence for objects that are not present in the image, or hallucinated, lower within the image representations.
Experimental setting. We evaluate the strength of the internal confidence c, as an indicator of object presence. We sample 5000 images from the MSCOCO training set, using the image captioning objective to caption methods with both InstructBLIP and LLaVA. We use the & for present objects and hallucinations within the captions generated by each VLM. We assess how well the internal confidence aligns with the ground truth labels object presence, where negative sample hallucination and positive sample is present object.
Baseline. As a baseline, we use the maximum output probability of the object's tokens. This the confidence of the model prediction. Previous works such Zhou et al. (2024) have found that hallucinations occur more frequently on objects characterized by high uncertainty during generation.
Results. We present quantitative results in Figure 8 and Table 5. We show qualitative results for LLaVA (Figure 14) and InstructBLIIP (Figure 15) in the Appendix. We find that utilizing internal confidence to classify object hallucinations provides 47.17% improvement in mAP in InstructBLIP and 22.45% in LLaVA. Furthermore, the ROC AUC improves over the baseline by 50.10% in InstructBII. IIP and 44.68% in LLLVA. indicating stronger object presence classification.
8

---

Published as a conference paper at ICLR 2025
Model
Method
CHAIR,
CHAIR,
Hallucinated Objects
Greedy
57.0
23.3
512
Nucleus
58.0
24.0
508
InstructBLIP
Beam Search
53.4
14.6
564
OPERA
45,6
13.9
472
Ours
43.8
12.5
419
Greedy
49.2
14.2
532
Nucleus
55.8
17.1
618
LLaVA
Beam Search
52.4
15.0
583
OPERA
44.8
12.8
Ours
42.0
12.2
444
Table 2: Hallucination intervention performance. We mass-remove hallucinations detected by the method in Section 5.1 and outperform other baselines. We observe considerable drop in the raw count of hallucinated objects.
5.2 HALLUCINATION REMOVAL
We use the mass editing technique to remove hallucinations detected by the prior method. Sec- tion 4.1.2 successfully removes significant portion of hallucinations but presupposes a knowledge of what the hallucinations lucinations and mass-remove them using PROJECTAWAY. Our chosen threshold prioritizes precision over recall (i.e. allow classification of some CD objects as hallucinations) because CD objects less affected by the removal method, as shown in Section 4.1.2.
We threshold the internal confidence of each object to identify hal-
Experimental setting. We threshold hallucinations as Co 0.2 for InstructBLIP and co 0.1 for LLaVA. Based on prior ablations (Section 4.2), we select (l' = 1, = 2, a = 1.5) for InstructBLIP and (1" 19, 1T 21, a 3.5) for LLaVA. Our prompt is "Please describe this image in detail."
Baselines. Since our method intervenes during the decoder step, we compare our method with standard decoding algorithms. Greedy decoding predicts the next token based on the highest logit probability. Beam search maintains tree of beams and selects the best beam at generation end. Nucleus sampling selects the next token from a set high probability tokens whose cumulative probability reaches threshold We also evaluate against OPERA (Huang et al., 2024), which mitigates hallucinations by adding an overtrust penalty during decoder generation. We set p = 0.9 for nucleus sampling. We beam search in method and unify Noeam = 5 for the baseline.
Results. We apply these parameters to 500 COCO images from the Karpathy validation set. We provide qualitative results in Figure 17 and Figure 16. Quantitative results in Table 2 show that we outperform our baselines and reduce hallucinations by 25.7% on InstructBLIP and 23.8% on LLaVA compared beam search. Our approach achieves a similar hallucination reduction rate as Section 4.1.2, despite not precisely differentiating hallucinations and some CD objects being incorrectly edited out. Notably, our method relies on no training or external models, effectively offering a "free lunch." We find similar performance on additional models (Appendix A.5) and attribute hallucinations (Appendix A.7).
5.3 ZERO-SHOT SEGMENTATION
Building upon our findings in Section 3.2, we utilize the internal confidence per image feature for zero-shot image segmentation. This application leverages the spatial information encoded in the image representations and demonstrates how VLMs internally represent and localize objects within images.
Method. Our approach leverages the spatial correspondence between image patches and their associated image embeddings. We use LLaVA generate the name of the class in the image and we focus on the internal confidence of that class per image patch. We take the mean internal confidence for tokens comprising class word. We resize the of 24 24 internal confidence values per image patch back into a fixed image size of 336 x 366 pixels. We then apply threshold to these confidence values to binarize them into foreground/background segmentation for the object image.

---

Published as a conference paper at ICLR 2025
Model
Method
Pixel Acc.
mloU
mAP t
atlention (CLIP)
Image Encoder
69.81
45.19
77.30
TextSpan (Gandelsman et al., 2024b) raw attention (VLM)
Image Encoder
75.57
53.60
80.22
VLM
67.28
39.27
73.96
Ours
VLM
76.16
54.26
79.90
Table 3: Segmentation Performance on ImaggeNetsegmennation Localizing objects using their probabilities within the image representations results in more accurate zero-shot segmentation than previous methods relying on vision encoders and VLMs.
Baseline. As baseline, extract the attention values of generated tokens with the image em- beddings from LLaVA. We also compare to the segmentation method introduced by Gandels- man et al. (2024b), which utilizes the attention heads of the image encoder without the addi- tional VLM processing, using the same image encoder (CLIP-ViT-LIA) ViT-L/14 at 336px).
Input Image
raw attention (VLM)
Results. We evaluate our method on the Im- agenet validation set. Qualitative results are shown in Figure and quantitative comparisons with the baselines in Section 5.3. We improve mAP by 8.03% over using the VLMs raw atten- tion values and provide better and/or comparable performance
Qurs
other state-of-the-are methods that utilize just the image encoder. While the VLM is not directly trained for segmentation, technique reveals that it still encodes signifi- cant spatial information about objects within its intermediate image representations.
Figure 9: Zero-shot segmentation. Warmer areas indicate higher internal confidence for the class at that image patch. We binarize these values with threshold to generate segmentations.
DISCUSSION AND LIMITATIONS
We interpreted VLMs' image representations through the language model layers and discovered that linear editing of these representations can selectively remove object information via a simple orthogonalization. Our findings enabled hallucination reduction and improved zero-shot segmentation. We present two limitations of our work and conclude with future directions.
Multi-token objects. Our method simplifies the use of object words that may be composed of multiple tokens, such as by taking the max internal confidence over object tokens or utilizing the average token embedding for editing. This can introduce noise to the internal confidence if certain tokens are common in multiple different words and lead to an approximation of the object's latent representations when editing.
Fine-grad iine-grained edits. The editing approach may struggle with highly abstract or longer sentences that involve attributes interactions of objects. Removing full sentence, for example, not something we assessed in this paper, since focus is on the removal of individual objects.
Future work. While our focus was on interpreting objects and object hallucinations VLMs, we believe that our approach can be extended to other key elements of visual scenes, such as people, attributes, and actions. We also focused on object removal, but we believe that editing can also be extended to inject objects into caption (by adding instead of subtracting the text embedding). We hope to explore the applications of our approach in other multimodal architectures. Our insights may help design better VLMs that more robust hallucinations and have improved spatial understanding. We plan to explore these directions in our future work.
10

---

Published as a conference paper at ICLR 2025
6.1 ACKNOWLEDGMENTS
We thank Kayo Yin for her comments and feedback on our paper. YG is supported by the Google Fellowship. Authors, as part of their affiliation with UC Berkeley, were supported in part by the the Berkeley Artificial Intelligence Research (BAIR) commons program.
REFERENCES
Amos Azaria and Tom Mitchell. The internal state of an LLM knows when it's lying, In Houda Bouamor, Juan Pino, and Kalika Bali (eds.), Findings of the Association for Compu- national Linguistics: EMNLP 2023, pp. 967-976 Singapore, December 2023. Association for Computational Linguistics. doi: 10. (0.186533v1/2022 htwg://////////nneeegeemmgmp findings-emnlp.68. URL https: // //aclanchology 2arg/222/ findings emmlp. 68.
Jinze Bai, Shuai Bai, Shusheng Yang, Shijie Wang, Sinan Tan, Peng Wang, Junyang Lin, Chang Zhou, and Jingren Zhou. Qwen-vl: A versatile vision-language model for understanding, localization, text reading, and beyond, 2023. URL https: /tt////axivvv... www.xiiii..ggaaaaaaaoeeeaaaaaaae 2308 12966.
David Bau, Jun- Yan Zhu, Hendrik Strobelt, Bolei Zhou, Joshua Tenenbaum, William T. Freeman, and Antonio Torralba. Gan dissection: Visualizing and understanding generative adversarial networks. In Proceedings of the International Conference on Learning Representations (ICLR), 2019.
David Bau, Jun Zhu, Hendrik Strobelt, Agata Lapedriza, Bolei Zhou, and Antonio Torralba. Understanding the role of individual units in deep neural network. Proceedings of the National Academy of Sciences, 2020. ISSN 0027-8424. doi: 10. 19773//11 URL https: // //www. pnas. org/ content peantten/eeall//22222 0010//00000111777511111
Nora Belrose, Zach Furman, Logan Smith, Danny Halawi, Igor Ostrovsky, Lev McKinney, Stella Biderman, and Jacob Steinhardt. Eliciting latent predictions from transformers with the tuned lens, 2023. URL https: / wwwaaaivvv.... www.oogggooo.oooo. /abs////////// / 0////333.011..
org
Trenton Bricken, Adly Templeton, Joshua Batson, Brian Chen, Adam Jermyn, Tom Conerly, Nick Turner, Cem Anil, Carson Denison, Amanda Askell, Robert Lasenby, Yifan Wu, Shauna Kravec, Nicholas Schiefer, Tim Maxwell, Nicholas Joseph, Zac Hatfield-Dodds, Alex Tamkin, Karina Nguyen, Brayden McLean, Josiah E Burke, Tristan Hume. Shan Carter, Tom Henighan, and Christopher Olah. Towards monosemanticity; Decomposing language models with dictionary learning. Transformer Circuits Thread, 2023. URL https:/ /ttps://traaaaa index.nix. html.
pub/2023 //t////oooanii--eeeee
Marco Bronzini, Carlo Nicolini, Bruno Lepri, Jacopo Staiano, and Andrea Passerini. Unveiling
Ilms: The evolution of latent representations in dynamic knowledge graph, 2024. http::
//arxiv. org/ abs / 240 03623.
Hila Chefer, Shir Gur, and Lior Wolf. Generic attention-mooee explannniiiyyy for interpreting bi-modal and encoder-decoder transformers, 2021. URL https: org/ abs /2103. 15679.
Chao Chen, Kai Liu, Ze Chen, Yi Gu, Yue Wu, Mingyuan Tao, Zhihang Fu, and Jieping Ye. Inside: Lims' internal states retain the power of hallucination detection, 2024. URL https: hww.aa////22.... 03744.
//arxiv.org/abat.......
Haozhe Chen, Junfeng Yang, Carl Vondrick, and Chengzhi Mao. Interpreting and controlling vision foundation models via text explanations, 2023. URL https://arxiv.er /f /2310. 10591.
Arthur Commy, Augustine N. Mavor-Paree Parker, Aengus Lynch, Stefan Heimersheim, and Adrià Garriga- Alonso. Towards automated circuit discovery for mechanistic interpretability, 2023. URL https: //a farxii. orgo abs 2.304. 14997.
Hoagy Cunningham, Aidan Ewart, Logan Riggs, Robert Huben, and Lee Sharkey. Sparse autoen- coders find highly interpretable features in language models, 2023. URL https: //arxiv. org/abs 2309. 08600.
11

---

Published as a conference paper at ICLR 2025
Damai Dai, Li Dong, Yaru Hao, Zhifang Sui, Baobao Chang, and Furu Wei. Knowledge neurons in pretrained transformers, 2022. URL htttttt/a/axxii//
abs/ / 2104. 08696.
Wenliang Dai, Junnan Li, Dongxu Li, Anthony Meng Huat Tiong, Junqi Zhao, Weisheng Wang, Boyang Li, Pascale Fung, and Steven Hoi. Instructblip: Towards general-purpose vision-language models with instruction tuning, 2023. URL https:/ http://axivvv... baargg/bbs/23000 2000:/55000000000
Amil Dravid, Yossi Gandelsman, Alexei A. Efros, and Assaf Shocher. Rosetta neurons: Mining the common units in model zoo. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pp. 1934-1943, October 2023.
Patrick Esser, Robin Rombach, and Bjorn Ommer. A disentangling invertible interpretation network for explaining latent representations, 2020. URL http:////wwiii..o.////////// 131 66.
Yossi Gandelsman, Alexei A. Efros, and Jacob Steinhardt. Interpreting the second-order effects of neurons in clip, 2024a. URL https: / arxiv. org/ abs/ 04341.
Yossi Gandelsman, Alexei A. Efros, and Jacob Steinhardt, Interpreting clip's image representation https///aaxi.o.o//////////22 httppof//31pp/g///
via text-based decomposition, 2024b. URL httpt:///////
/eaetts://arxtee /tt:://aaiivv.///i
05916.
Asma Ghandeharioun, Avi Caciularu, Adam Pearce, Lucas Dixon, and Mor Geva. Patchscopes: A unifying framework for inspecting hidden representations of language models, 2024. URL 11.4000110000
pittss:/aazet http/aaxwvvggaaaaaaa////
Danny Halawi, Stan-Sttanssls Denain, and Jacob Steinhardt. Overthinking the truth: Understanding how language models process false demonstrations, 2024. URL https: http::://axxxv..aa.eeoo 2307. 794
Jinwen He, Yujia Gong, Kai Chen, Zijin Lin, Chengan Wei, and Yue Zhao. Llm factoscope: Uncovering ilms' factual discernment through inner states analysis, 2024. URL https:// axxiv.orge abs 22122232..2 6374.
John Hewitt and Christopher D. Manning. A structural probe for finding syntax in word repre- sentations. In Jill Burstein, Christy Doran, and Thamar Solorio (eds.), Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume (Long and Shori Papers), pp. 4129-4138, Minneapolis, Minnesota, June 2019. Association for Computational Linguistics. doi: 10.18653/v1/N19. 1419.
URL https:tt///.. / /aac/aaantttt aalaaaannolloooooo org/N19 1419.
Yuan Hu, Jianlong Yuan, Congcong Wen, Xiaonan Lu, and Xiang Li. Rsgpt: A remote sensing vision language model and benchmark, 2023. URL https: / / arxiv. org/ abs/ 2307. 15266.
Qidong Huang, Xiaoyi Dong. Pan Zhang, Bin Wang, Conghui He, Jiaqi Wang, Dahua Lin, Weiming Zhang, and Nenghai Yu. Opera: Alleviating hallucination in mudti- large language models via over-trust penalty and catpeecccioooallcattiio 2024. URL https: : //axxiv....////////// org/ abs/ 17911.
Jiahao Huo, Yibo Yan, Boren Hu, Yutao Yue, and Xuming Hu. Mmneuron: Discovering neuron- level domain-speciffe interpretation in multimodal large language model, 2024. URL https: //aaxxiva htt. /tff///f//000 1193v1.
Ziwei Ji, Naycon Lee, Rita Frieske, Tiezheng Yu, Dan Su, Yan Xu, Etsuko Ishii, Ye Jin Bang, Andrea Madotto, and Pascale Fung. Survey of hallucination in natural language generation. ACM Computing Surveys, 1--1 March 2023. ISSN 1557-7341, doi: 10. 1453345/3577 URL http://dx: doi org 11........11111111
Goro Kobayashi, Tatsuki Kuribayashi, Sho Yokoi, and Kentaro Inui. Attention not only a weight: Analyzing transformers with vector norms. In Bonnie Webber, Trevor Cohn, Yulan He, and Yang Liu ), Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 7057-7075, Online, November 2020. Association for Computational Linguistics. doi: 10.18653/v 16653//1/202000. /2tp22000000mmmaii.... 2020 0. emn 1 1p-ma 574.
URL ht https: / /aaallanti aclanthology /rg/
12

---

Published as a conference paper at ICLR 2025
Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models, 2023a. URL https: arxiv. abs 2301. 12597..
Kenneth Li, Aspen K. Hopkins, David Bau, Fernanda Viégas, Hanspeter Pfister, and Martin Watten- berg. Emergent world representations: Exploring a sequence model trained on synthetic task, 2024. URL mnttps://arre stttps://ax....org//bbbb. www.aos/////.e////.///......../t /21 0 13382.
Yifan Li, Yifan Du, Kun Zhou, Jinpeng Wang, Wayne Xin Zhao, and Ji-Rong Wen. Evaluating object hallucination in large vision-language models, 2023b. URL ptttpps:/aaxii.xrtt abs/2305. 10355.
Tsung-Y Yi Lin, Michael Maire, Serge Belongie, Lubomir Bourdev, Ross Girshick, James Hays, Pietro Perona, Deva Ramanan, C. Lawrence Ziticck and Piotr Dollár. Microsoft coco: Common objects in context, 2015. URL
/atr//////////// org/ a / 1405. 0312.
Haotian Liu, Chunyuan Li, Yuheng Li, and Yong Jae Lee. Improved baselines with visual instruction tuning, 2024a. URL https: : arxiv. ott.iiiorrr//oooooo org/ has//ab////// 2210. 03744.
Haotian Liu, Chunyuan Li, Yuheng Li, Bo Li, Yuanhan Zhang, Sheng Shen, and Yong Jae Lee. Leava-next: Improved reasoning, ocr, and world knowledge, January 2024b. URL https: 2024 01-30-
//1lava-vl. waa.vvvvlliggub......... wwwittub.iobbbo
Fuwen Luo, Chi Chen, Zihao Wan, Zhaolu Kang. Qidong Yan, Yingjie Li, Xiaolong Wang, Siyu Wang, Ziyue Wang, Xiaoyue Mi, Peng Li, Ning Ma, Maosong Sun, and Yang Liu. Codis: Benchmarking context-dependent visual comprehension for multimodal large language models, 2024. URL https: http://axxvvv.//// abs abs/2402.13667.
Kevin Meng, David Bau, Alex Andonian, and Yonatan Belinkov. Locating and editing factual associations in gpt, 2023. URL https: aaxiv.vvvv. org/ abs/2200 05262.
nostalgebraist.
LessWrong, Aug 2020. interpreting teeeeppreingggg--tteeeee tpt-the- lesswrong. httpp///....ooopsss//t/tttttt/a/ /otpsss//ccBBBBBoppaaavvv
Interpreting GPT: The logit lens.
URL
httpp::/wwyy http://wwwwessss.rrrr.seo
lens.
Vedant Palit, Rohan Pandey, Aryaman Arora, and Paul Pu Liang. Towards vision-language mecha- nistic interpretability; A causal tracing tool for blip, 2023. 2308. 14179.
Vitali Petsiuk, Abir Das, and Kate Saenko, Rise: Randomized input sampling for explanation of black-box models, 2018. URL htttt:t//aaxiiiii wwtp:/wwwwwgp///////...///// 07421.
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. Learning transferable visual models from natural language supervision, 2021. URL https:
Shauli Ravfogel, Michael Twiton, Yoav Goldberg, and Ryan Cotterell. Linear adversarial concept erasure, 2024. URL ptatttss///riivvv http://oorr////oo
org abs 2201 12091.
Anna Rohrbach, Lisa Anne Hendricks, Kaylee Burns, Trevor Darrell, and Kate Saenko. Object hallucination in image captioning, 2019. URL
02156.
Sarah Schwettmann, Neil Chowdhury, Samuel Klein, and Antonio Torralba Multimodal neurons in pretrained text-only transformers, 2023. URL https::pp///
httpps///axx.vooo///////t/ /
01544.
Da Song, Xuan Xie, Jiayang Song, Derui Zhu, Yuheng Huang, Felix Juefei-Xu, and Lei Ma. Luna: masel-based universal analysis framework for large language models, 2024. URL https: //arxiv. org/ abs/ 22110.11112 10110.1221. 14211.
Weihang Su, Changyue Wang. Qingyao Ai, Yiran HU, Zhijing Wu, Yujia Zhou, and Yiqun Liu. Unsupervised real-time hallucination detection based on the internal states large language models, 2024. URL ht /tattpss///riiv.... org/ /244///3///33333 /2003/43.0000 06448.
13

---

Published as a conference paper at ICLR 2025
Shengbang Tong, Ellis Brown, Penghao Wu, Sanghyun Woo, Manoj Middepogu, Sai Charitha Akula, Jihan Yang, Shusheng Yang, Adithya Iyer, Xichen Pan, Austin Wang. Rob Fergus, Yann LeCun, and Saining Xie. Cambrian-1: I: A fully open, vision-centrie exploration of multimodal Ilms, 2024. URL https:://ariiii org/ abs 2406. 6860.
Mycal Tucker, Peng Qian, and Roger Levy. What if this modified that? syntactic interventions via counterfactual embeddings, 2021. URL https: /twpvvaxivvv/vv////////a../a.iiv org/ abs 2105 14002.
Neeraj Varshney, Wenlin Yao, Hongming Zhang. Jianshu Chen, and Dong Yu. A stitch in time saves nine: Detecting and mitigating hallucinations of Ilms by validating low-confidence generation, 2023. URL https:ttp....m....... /tte:///araaae //aaxivvv.../ www.or///aa...... abse///b//// 22w./////2...222 03987.
Dimitri von Rutte, Sotiris Anagnostidis, Gregor Bachmann, and Thomas Hofmann. A language model's guide through latent space, 2024. URL https:/ arwiv. http:/pff/2240000 14433.
Qinghao Ye, Haiyang Xu, Jiabo Ye. Ming Yan, Anwen Hu, Haowei Liu, Qi Qian, Ji Zhang, Fei Huang, and Jingren Zhou. mplug-ow12: Revolutionizing multi-modal large language model with modality collaboration, 2023. URL httph:ttt///att //axiv.. http://awwxvvv...aaaa/// org/orrg//a... 2211. 04257.
Shukang Yin, Chaoyou Fu, Sirui Zhao, Tong Xu, Hao Wang, Dianbo Sui, Yunhang Shen, Ke Li, Xing Sun, and Enhong Chen. Woodpecker: Hallucination correction for multimodal large language models, 2023. URL www.hhttt//// /tt//aaxiiiv. www.iiiw..ggas/////ooeea..
Jinrui Zhang, Teng Wang, Haigang Zhang, Ping Lu, and Feng Zheng. Reflective instruction tuning: Mitigating hallucinations in large vision-language models, 2024a. htt.ttts::// http:://////xx/v.a////// abs htss/240.1110
Shaolei Zhang, Tian Yu, and Yang Feng. Truthx: Alleviating hallucinations by editing large language models in truthful space, 2024b. URL
/2402/22f24..2.. 1.111.
Yiyang Zhou, Chenhang Cui, Jaehong Yoon, Linjun Zhang, Zhun Deng, Chelsea Finn, Mohit Bansal, and Huaxiu Yao. Analyzing and mitigating object hallucination in large vision-language models, 2024. URL https: : //trx/axivvvv//////////////.aaav arg/abs//311 00754.
14

---

Published as a conference paper at ICLR 2025
Edit Scope
Model
Hallucinations
CD
InstructBLIP
4545
14178
No edits
LLaVA
4372
15053
InstructBLIP
2672
14189
Hallucinations
LLaVA
3348
15061
InstructBLIP
5078
13864
CD
LLaVA
4583
14826
Table 4: Supplemental metrics for Table 1. measure unique hallucinated and correctly detected (CD) objects.
27
100
Percentage change
Edited hidden layer
Hallucinations Correctly detected removal rate
11 13 15
15 17
Weight factor a
Text embedding hidden layer
Hidden layer ablations for In- structBLIP. We track hallucination reduction (%) across different layers to edit at and extract la- tent embeddings for the text embedding, crossing out (red) parameters from consideration where there is decrease in correctly detected objects.
Figure 10:
Figure 11: Weight ablations for InstructBI. We vary the weight factor and measure changes in correctly detected objects, object removal rate, and hallucination reduction. We observe a decline in hallucinations as weight increases and mark a weight where there is no loss in correctly detected objects.
A APPENDIX
A.1 MASS-REMOVING OBJECTS
We mass-remove mentioned objects (hallucinations and correctly detected) with PROJECTAWAY and tally up the total number of unique hallucinated and CD objects in Table 4.
A.2 ABLATIONS FOR INSTRUCTBLIP
We show hidden layer and weight ablations for mass-removing hallucinations in InstructBIIIP referenced in Section 4.2. The hidden layer ablations indicate that most of the parameter space is too sensitive to edit and leads to losses correctly detected objects. We find that smaller (" and / parameters are the most effective for reducing hallucinations. Our best parameters (L' 1, 2) reduce hallucinations by 38.5%. It not fully understood why the majority of the parameter search space is invalid in comparison with LLaVA in Figure 6. It is possible that the fine-tuning step in LLaVA semantically aligns hidden image representations with text embeddings more than InstructBII .I, allowing linear edits have the precise, intended effect.
15

---

Published as a conference paper at ICLR 2025
A.3 HALLUCINATION DETECTION
We show quantitative comparisons from our hallucination detection approach using internal confi- dence (Section 5.1) the baseline in Table 5. We also show qualitative examples for LLaVA in Figure 14 and for InstructBLIP Figure 15. These samples exhibit model-generated captions, parsed objects, and whether they are classified as hallucinated correctly detected based on their internal confidence score.
A.4 HALLUCINAAION REDUCTION
We exhibit sample results from our hallucination reduction approach (Section 5.2), which linearly removes text representations of hallucinations from image representations, in Figure 17 for Instruct- BLIP and Figure 16 for LL.VA. We show the image caption before and after our linear editing method, removing objects detected hallucinations.
A.S QUANTITATIVE EVALUATIONS ON MORE ADVANCED MODELS
We evaluate our approach on two additional models, LLa 7B (Liu et al., 2024b) and C 8B (Tong et al., 2024) with Llama 3. We threshold hallucinations as co < 0.4 for LeaVA-NeXT and < 0.3 for Cambrian-1. Based on qualitative examples and referencing optimal parameters from other models Section 4.2, we select (l' = 24, 1 = 22, a = 2) for both models. We show quantitative results for hallucination detection in Table 6 and for hallucination intervention in Table 7. With our method, we observe a 27.73% improvement in CHAIRs with LLaVA-NeXT and 28.88 28.86% improvement with Cambrian-1 demonstrating consistency with our findings the LLaVA and InstructBI... models.
A.6 OBJECT LOCALIZATION
We show qualitative examples for localization with internal confidence for specific image representa- tions, specifically for the LLaVA model, in Figure 18.
A.7 ATTRIBUTE HALLUCINATIONS
Our analysis in this paper centered on object hallucinations because automated tooling and bench- marks for attribute (ex. shape, color, number) hallucinations are relatively sparse. However, demonstrate the applicability of our editing technique on attribute hallucinations with qualitative examples filtered from the VQA 2.0 challenge in Figure 12. We reuse the editing hyperparameters for InstructBLIP (1₃ 1, = 2, a = 1.5) and only edit attributes with < 0.05.
A.8 ZERO-SHOT CLASSIFICATION
We evaluate the strength of internal confidence derived from the logit lens on image representations for classification of the COCO class within patches of the image. We the COCO ground truth segmentations to find ground truth classes for image patches. We determine the accuracy of the rankings found from logit lens internal confidence scores to predict the class per patch and present our results in Table 8. We find that these values highly vary across classes, which we hypothesize is because certain classes such "person" are represented with more specific tokens such as "doctor". "skier", "girl", etc. resulting in lower internal confidence for the tokens in "person" while other objects like "toothbrush", "banana", and "broccoli" described in the same word the COCO class.
A.9 QUALITATIVE EXAMPLES BEYOND COCO 2014
We focus COCO 2014 in our analyses because CHAIR. our main evaluation criteria, is tied with the dataset and can automatically categorize objects of interest in image captions. While COCO 2014 a diverse set of images, we provide qualitative examples of hallucination reduction (see Section 5.2) on images from LLa LLaVA-Bendl (Liu et al., 2024b), a collection of 24 images of varying subjects. The examples in Figure 13 using InstructBI.I align with the strong hallucination reduction observed with COCO 2014.
16

---

Published as a conference paper at ICLR 2025
Question: gym shoes?
Question: What the wheels rims?
Question: color tollpoop?
Before: red and black.
Before: black.
Before: whito.
After: black and white.
After: sllver
After: rec.
Questior: How many bunchos of bananas are there?
Question: How many trucies are an the road?
Question: How many yallow vosas?
Before: Twi.
Before: D.
Before:
Uheee
Onc.
After:
Figure 12: Qualitative results for attribute hallucinations using InstructBLIP. We filter the VQA dataset for color and object number inaccuracies and correct answers with low confidence scores (c, < 0.05) using PROJECTAWAY. We reuse the same hyperparameters previously chosen for InstructBLIP (L' 1 2, a 1.5).
InstructBLIP
LLaVA
Method
mAP1
ROC AUC f
MAP
ROC AUC
Baseline
0.53
0.55
0.49
0.47
Ours
0.78
0.83
0.60
0.68
Table 5: Object presence classification performance. We use internal confidence c, confidence score to classify whether the object is present in the image. We evaluate the mAp and ROC AUC of classification method against the baseline for both the InstructBLIP and LLaVA models over subset of 5000 COCO images.
LLaVA-NeXT
Cambrian-1
Method
mAP
ROC AUC
mAP
ROC AUC +
Baseline
0.93
0.66
0.73
Ours
0.95
0.75
0.97
0.83
Table Object presence classification on more models. We classify whether the object is present in the image using internal confidence for LLa and Cambrian-1 over subset of 500 COCO images.
17

---

Published as a conference paper at ICLR 2025
The mage captures breathtaking serial scenic coastine mountainous beckground. visible fonsground, stratching side of the image to the right. There are several boats scattered throughout the water, some closer to the shareine and others further out. In addition the
Before:
Before: The image features painting beautful
Before:
The Image deplcts long wooden pier extending
flowing
surrounded
depicted weering face. painting capturas essence of Leonardo da Vinal's Mona Lisa, one of the most famous painings in the world. the painting, the woman's face takes significant making her the focal point portion the Additionally,
background.
expression
the water, inviting explore the tranquil surroundings- boat can be seen docked the end of the pier, adding to the paaceful atmesphere of tha scene. addition the bost,
stretches
several people
smaller boats
visible
creating
image, closer and the other slighlly further away. On the right side of the image, there are two more people, both claser to the water's eage.
left
trees
background painting. adding depth and diimension to the spene.
activity orjoyment those spending
on the water. Overall, the image captures relaxing and picturesque setting, perfect for unwinding and taking the natural beauty of the and its surroundings.
After: The depicts painting of beautiful long, flowing She serene expression face. The painting ceptures the essence the
image depicts aerial arge mountain background, The mountain stusted on left side of the image, while the ocean can be sean the right side. There are several cars visible in the scene, likely parked or driving along the road.
scenic coastal
The image depicts long wooden pier extending into sorone lake, surrounded by trees and mountains the background. The plar situated the middle the lake, creating peaceful
After:
famous painting 'Mona Lisa" by Leonardo da Vinci, which is considered one of the most famous paintings the world. The painting for anigmatic smile and intricate brushstrokes, making
the
atmasphare.
mountain, in addition the cars, scattered throughout the landscapa, adding to the natural besuty of the area.
away several trees
masterp
surrounding several natural beauty of the scene. The adding
pier seoms to be popular spot for fishing or simply enjoying the serene surroundings.
Figure 13: Qualitative results on images from VLLaV--endd We randomly select images from the benchmark and use InstructBLIP detect and edit out hallucinations, Our hyperparameter selection is the same as in Section 4.1.1 1 (L' = 1,1" = 2, = 1.5).
Model
Method
CHAIR; 4
CHAIR, +
Recall (%)
Beam Search
23.8
63.12
LLAVA-NeXT
Ours
5.52
17.2
63.12
Beam Search
3.27
53.28
Cambrian-1
Ours
2.7
6.6
53.28
Table 7: Hallucination intervention performance on more models. mass-remove hallucinations detected by the method in Section 5.1 on two more models, LLaVA-NeXTT and Cambrian-1, on the same subset of 500 COCO images as used in Table We observe consistent improvement over the baseline while maintaining recall of objects present in the image.
18

---

Published as a conference paper at ICLR 2025
Internal Confidincces clock 06.00
tuE
Internal Confidences stop sign 99.27% 2 2.54% 2.34%
light truck 80,
car 7.20%
car 47.20%
person 2.10%.
fire hydrant 02% parson 0.44%
0.00%
Baseline Caption
Baseline Caption
Baseline Caption
The image depicts street corner with stop sign prominently displayed the right side of the scane. Next to stop sign, there a streat that reeds "Hell Canyon Rd. The posttionad
The image dopicts quist city street sunny day, with tew cars and trucks parked along the side the road. There are multipae traffic various points ensuring passage vehicles
The mmge featuras a large brick church with tall clock tower. The clock situsted d on the side of the tower, making prominent feature of building The church has steeple and top
stop sign, pedestrians, seen
adding
background, scene,
two people visbile standing closer to right Addalil,lyy there car par of the church, likely bei onging one of the visitors.
addition
back the right side.
few podestrians walking along sidewalk, One person can seen near the center of the scene, while two others are closer to the right side of the image. fire hydrant also visible on the right side of the street. Dvarall, the scene conveys a callm and pesceful atmosphere the city.
Internal Confidences
Internal Confidences horse 96.2 3..3.%
Internal Confidences
cake 99.99%
man 86 86.33% chair 57.04%
man 60.83 bench 9.58%
oven B8 22%
bottle 97.34%
table 88.02%
girl 29.65%
woman 83.76%
stove 71.44%
6.85%
0.888%
dining table 2.43%
64.54%
refrigerator 6.61%
microwave 4. 4.93%
bowl 3.02%
Basellie Caption
image Baseline Caption
Caption
The image copicts group men in militryy uniforms gathered around diring table, calebrating thair 70th arrivarsary. Thay are shaking hands and posing for picture in front of large sheet cake. The cake placed the table, taking sigrificant portion of the
young horse, ansuring the gir's safery while she enjoya the ride. The girl wearing pink dress, adding touch of color to scene.
The image festures a woman standing in kitchen, cooking meal She is wearing green shirt and focused on stirring a large pot on the stove. The por appears be filied with food, and the woran is using spoon to mix the ingredients.
stting hoiding
in the background, there is bench and dining table, suggasting that the localion might be park or outdoor gathering area.
The kitchen is well equipped with various appllancas and utansil.. There refrigerator the lell side the
There multiple chairs surrounding the dining and several as backgr aund, mounted adding stmosphere celebration.
be
the oven, bowi countertop
Additionally, two bottles, side other right kitchen possibly containing cooking ingredients or beverages.
Figure 14: LLaVA Object Presence Classification. Sample image captions from LLaVA and the internal confidence scores for objects the caption used for classification as correctly detected objects hallucinations.
19

---

Published as a conference paper at ICLR 2025
Internal Confidances
Confidences bowl 0.32%
Confidences person 0.47%
99, 99.76% table 94.79%
fork 97.48%
umbrella
knife 1.78%
0.44%
cup 0.13%
0.38%
0.4%
Image Baseline Caption plate
Baseline Caption
Baseline Caption
sitting at dining holding fork appears be about to take bite from the cake. There are several other utensils on the table, including knives, forks, and spoons, suggasting that the man might ba sharing the caka with someone else. In the tackground, there a bowl placed on the table, pessibly containing adcitional food drinks. Overal, the scens depicts man enjoying delicicus sice chocolate cake.
sitting red shoes, The interacting the shoes, possibly trying to play with Ihem or investigate them. There are several other objects in the room, such as bowl and cup placed on table near the cat's pasition. Additionally, a window can be sean in the background, adding to the overall atmosphere of the scene.
The image features
The image festures small table and two chairs placed outside on sidewalk. The table has vintage fairr dryer sitting on tog of adding unique touch to the scene. The chairs are poshtoond close the table, creating seating for people. There umbrolla additional protection from background, passing by occasionally.
provicing
Internal Confidences sirplane 56. 30.55%% % plane 19.48%
Confidences
Confidences
table 858%
vase 50%55%% bench 3.69%
sink 97.07% spoon 5.10%
0.53%
fire hydrant
chair 63 63.84%
knife 0.3%
fork 0.07%
0.23%
person 0.22%
Baseline Caption airpiane parked passibly
Baseline Caption
Baseline Caption
The image depicts American fines
features outdoor
The depicts kiichen sink with two sinks,
of the scene, are positioned furthar away from in total, there are al least 10 people present in the scenee Additionally there is fire hydrant located close to the arrpann, likeyy for amargency purposes..
plas--topped cining table surrounded pink flowers, adding elegance to space. vase placed center
left and other right side. sink laft adge of countartop, away dishwiäsher located the
chairs,
filed with beauty
anpears propartng takaoff people
right posiiooeed
some standing near airplane,
table, making it focal point of area. Thare several chairs
edge. addition
surrounding the table, providing ample seating for people to enjoy their meals ratax in the sunshine. A bench can also be seen on the left side of the patio, offering aduitional seating options. Overall, the scana radiates a warm and Inviting atmosphere, perfect for outcbor gatherings and relaxation.
kitchen area. There are several utensils scallered throughout the scenee, inclusing
spoon, a fork, and knifee placed neer the siries and the dishwashee.
Figure 15: InstructBLIP Object Presence Classification.
20

---

Published as a conference paper at ICLR 2025
Before: image deplcts
Before: image parson sitting holding dorut hend. appears half-eaten, with bite taken of it. The person seems be onjoying their snack sitting on the bench.
Before: image features wearing dress and lie, slanding room, He weering striped tie, and the shirt appears looking down his tie, possibly adjusting it checidng appaarand
shirt and gray pants, standing a skateboard. skateboard is placed on ground, with person's in procass of picking
The appears
resting on The
person appears up the skatebooar preparing to ridelt
In the room, there is couch located lowards the right dide of the image, and chair can be seen in the background on the left side. The man is positioned the center of the scona, drawing attontion to his attine.
in the background, there a chair and dining table, suggesting that the scene might be taking placa in an outdoor or somi outdoor setting.
addition the person and the donut, there is handbag placed nex: to the person on the banch, and a backpack can be seen in the beckground.
After: in the image, a person is standing
After The image depicts man wearing a tress shirt and tio, stand ng in room. The shirt is white, and the tie is blue. The man is wearing belt, which visible around his waist. The room appears be dimily it, creating a somewhat mysterious atmosphere Vnlnnn addition the there two other people the room, left side the other the right side the However, they not the focus dress shirt tie central figure.
sidewalk, holding colorful skateboard in their hand. The skateboard features vibrant design, making an eye catching sight. The person appears to either preparing skate taking break from their watebbarding session. The skateboarding
After: the image, person is sitting on
bench and holding pastry in their hand.
The pastry appeans to be coughnut, and
the person in the process of ealing it.
Another doughnut be seen the bench, close the person's hand, chair placed the bench the person sitting.
culture and enjoyment brings to partcipate
visible
The image captures a man mid-air, leaping over a bed in bedroom, He wearing tis and appears to be in the miccle of an onergatic jump. Thare are seiveral other people in the room, with one person standing near the left side of the bed and two others cloger to the right de. The bedroom is furnishd with couch, a chair, and dlining tablie. potted plant can be sean the right side of the room, aoding touch of greenery to the space, clock mounted the wall, and the dining table.
Before:
Before: The image captures a hear twarmingg scone of two elephants, an edult and baby, playing togethar in a body of water. The adult elophant is partally submarged the water, whie the bany elephant is swimming nearby. They appear to be enjoying their time the water, splashing and Interacting with each other.
Before: The image depicts cleen and well-maintained kitchen with a white stove top oven. The oven has two burnars, ana an the left side and the other an the right side. Above the oven, there is light that iluminanas the cooking area. The kitchen is equipped with verious utenslis, including multipre knives and spoon. The knives are placed in knife block, while the spoon can be found on the left side of the oven. Additionally, there is bowl lnsated the left side of the kitchen, and bottle can be seen on the right side.
eddition the two elephents, there are bench few other objects side of image, potted plant located the top
After: The depicts kitchen
ald-fashioned with
The image depicts and blue jumping the soove bedroom. appears
Aftar:
white
right side. cooking The kitchen well-equisped verious
stove, there
image depicts two elephants a body of water, possioly river lake. One elophants appears to an adult, baby elephent is swimming in the water, while the sdult elephant standing nearny. They seem to be enjoying their time together in the water.
people room, with several other bed others acattored around the area, The bedroom is fumissee with dining table, chair, and potted plant. There also a cup placed on the dining table, and handbag can be seen near one of the people in the room. Overall, the scana captunes lively and energelic stmosphere in the bedrocm.
diving
bed.
appliances. There two
standing the
stove and the other on right side. Additionally. there two spoons, one near left
while other is baby elaphant.
the other neet the right knife. A bottle can be seen on the left dede of the stove, and a clock mounted on the wall above the oven.
Figure 16: Qualitative results for LLaVA hallucination intervention. Our algorithm removes hallucinations and, at times, adds correctly detected objects.
21

---

Published as a conference paper at ICLR 2025
image deplcts young woman shtting on banch, leaning againsi a wal, taking her call phone. She wesring white shirt and appeors to be engrossed conver conversetion. selion. the beckgrount there are several other people walking around, some of them carrying bags or purses. The scone takes place at right, with streetlights Illmirating the area and casting shadows on the surroundings.
Bafore: The image foatures a young woman sitting bench outside, eatinng a dog wrapped napión. She appears be enjoying food she takes big bite out the dog. There are several bottles scatlered around the scene, including one close to the woman's left hand and another near her right foot. in addition to the botiles, there are two cars visible in the bockground, round, one on the left side and another on the right side of the mage.
Before: The Image depicts group of people riding in an old- fashioned, open-air truck. truck carry carrying large number people, with some sitting on bed of truck and others standing around it There several umbrellas the scene, one of which is held by person in the back the trucs, providing shade for the passengers. adottoon the umbreta, rete there are several handbags scettered throughout the scene, liely be paiorging to the passengers. The sruck appears to be making its way through a wooded arca, surrounded by trees and follag.
After: A young man sitting on ralling, lazning against it while talking on his call phone. He is
After: The image depicts a young woman sitting on a bench outside, eaciing hot dog wrapped a napkin. She appears to be enjoying her meal
wearing a backpack and appears to be engr assed in his satversation. There are sever other people in the scene, some of whomm are standing or walking around. One person closer to the man the raiiig,, while others are scattered throughout the area. A handbag can be seen nearby, possibly bei bolonging to one the people in the scene.
The image features a group people riding in an old- fashioned, open-air vehicle, likely an old tractor or truck, on dirt road surrounded by lrees. They are enjoying lesuurlly ride through the countryside, with one person holding purple umbrella to protect themselies from the sun. There are several people the vericle, with some seated in the front and others the back. The scane captures the essence of relaxed and onjoyoble day ou: the countryside.
After:
while suroourded ded by other people who are waking around the arew. There are several bottles scattered throughout the scene, likely containing drinks other beverages. in the background, there is buildig with multiple windows visible.
The image depicts group of four young men playing game of friebee on grassy fieid. They are al wearing orange shirts, and one of them jumping to catch the frisbe as it flies through the sir. There several benches scattered around the field, providing seating options for the players and spectators, accition to the nanches, there ane two backpacks bisible in the scene, lisaly belonging to the players spectitooss Overal, the scene captures lowy and enir energetic game of frisbee emong friends.
Before:
Before: The image depicts a city street with a red and while public transil bus parked on the side of the road. The bus surounded by other vehicies, including cars, truckss, and a motoreycle. There are several people visible in the scene, some standing near the bus and others walking along the sicewalk, bieyole is also presant in the scane, liaely belonging one of the padestrians. Over all the scone showwaases urban environment with various modes of transportation use.
filled with fumituue decorations, There black several potted center surrounded by coffee television
Before: The depicts
right
addition scattered throughout room, a greenery space. room are visible, right
col
on left coffee taple. Overall, living room has and invtting sphere, perfect for relaxing spending time with friends or family.
side,
After: A red and white city bus is parker side of street, wailing passengers board. including
After:
There group of field. Thyy
frisbee
the vicinity the front
matching
with couch, coffee tadls, appointed living room dining table. There are several potted plants soattered throughout the
image dapicts
jumping catch the
flies through
of bus, another walking
the The other players participating the game, trisbae block their opponints'
further away Ikaly
catch
backgroure bicycle vistale
room, adding a touch greenery to space, bookshelf is visible on left side of room. showcasing verious books and decoretive itemss In the center the room, there as coffee table surrounded by chairs, providing ng a comfortable seating area for quests. The room has a cazy and inviting atmosphere, perfect for relexation and socal gatherings.
nng
benches scattiered Thare providing seating those waiching game taking from
parkad
Figure 17: Qualitative results for InstructBL.I hallucination intervention.
22

---

Published as a conference paper at ICLR 2025
Input Image
raw attention (VLM)
Ours
Input Image
raw attention (VLM)
Ours
Figure 18: Object Localization Samples.
23

---

Published as a conference paper at ICLR 2025
Class
T3%
T5%
T10%
Patches
Class
T3%
T5%
T10%
Patches
airplane
51.0
61.8
72.5
102
kite
66.7
77.8
88.9
apple
70.2
76.6
80.9
47
knifc
24.0
26.0
34.0
50
backpack
44.8
48.5
58.5
614
laptop
34.1
38.3
46.1
334
banana
77.2
79.2
83.2
microwave
23.8
37.2
55.2
223
baseball
33.3
33.3
motorcycle
64.7
78.3
984
baseball glove
65.4
69.2
73.1
26
mouse
50.0
50.0
66.7
bear
37.6
41.4
47.6
739
orange
44.2
50.0
57.7
52
bed
44.0
47.8
52.0
2373
38.9
54.4
78.9
507
bench
61.3
63.9
66.6
parking meter
20,9
29.6
230
bicycle
27.2
32.3
58.3
235
person
0.5
1.1
12.6
11528
bird
75.5
79.4
80.6
155
pizza
41.8
52.1
69.4
3146
boat
26.2
29.7
38.8
516
potted plant
43.8
52.
63.0
192
book
28.1
39.0
44.8
210
refrigerator
23.7
31.4
50.0
156
bottle
46.2
53.4
62.0
remote
17.3
bowl
22.1
25.0
31.9
1364
sandwich
40.2
44.4
54.3
468
broccoli
75.9
77.2
79.7
79
scissors
71.9
71.9
71.9
32
bus
49.8
54.1
61.3
1786
sheep
49.5
52.6
56.6
489
cake
43.2
51.4
83.5
1182
sink
57.0
60.3
65.1
272
car
29.4
37.7
52.7
skateboard
48.1
50.6
57.7
239
carrot
50.0
50.0
75.0
4
skis
27.6
34.2
44.7
76
cat
57.7
61.5
66.5
2239
snowboard
37.5
41.7
47.9
48
cell phone
73.4
76.6
82.8
64
spoon
35.5
43.5
53.2
62
chair
31.6
33.1
37.6
sports ball
4.4
8.9
clock
67.6
70.6
75.9
299
stop sign
85.7
88.6
89.9
237
couch
33.4
38.9
63.2
2435
suitcase
38.1
40.7
44.7
472
cow
51.9
58.6
67.0
324
surfhoard
48.6
57.5
69.9
146
cup
9.4
14.9
30.4
181
teddy bear
38.5
43.1
47.7
239
dining table
25.4
47.6
74.9
7403
tennis racket
88.9
88.9
88.9
dog
45.9
51.3
59.7
1057
tie
46.2
49.7
52.8
197
donut
34.8
40.0
45.2
115
toilet
92.0
97.3
99.6
1131
elephant
47.0
57.4
64.9
902
toothbrush
78.9
78.9
100.0
19
fire hydrant
43.4
47.0
50.1
419
traffic light
45.5
45.5
fork
41.4
45.7
54.3
70
train
40.7
45.0
52.1
3008
frisbee
50.0
60.6
71.2
66
truck
34.5
40.1
54.6
930
giraffe
32.0
39.8
62.8
810
27.2
32.6
37.6
298
handhag
42.2
53.0
57.8
83
umbrella
39.5
40.5
42.8
526
horse
63.7
65.8
vase
29.9
32.6
45.1
264
hot dog
56,9
62.7
69.3
394
wine glass
61.9
67.0
74.3
218
keyboard
43.1
47.7
49.2
65
zebra
33.0
35.9
44.5
373
Overall
32,7
39.3
52.2
55988
Table Per-class patch classification accuracy. For each COCO class, we show the percentage of patches containing that object that top-k logit lens predictions can correctly identify.
24


Reviews:
[{"review_id": "i615ZBkfh0", "reviewer": "Reviewer_9Mwt", "summary": "The paper addresses the issue of hallucinations in Vision-Language Models (VLMs) by interpreting and editing their internal representations. The authors apply the logit lens technique to project image representations onto the language vocabulary, discovering that objects present in the image have higher internal confidence scores compared to hallucinated objects. Utilizing this insight, they propose a method to detect hallucinations within VLMs. Furthermore, they introduce a knowledge erasure algorithm called PROJECTAWAY, which linearly orthogonalizes image features with respect to hallucinated object features to remove hallucinations from the model's output. The method is evaluated on two state-of-the-art VLMs, LLaVA 1.5 and InstructBLIP, showing a reduction in hallucinations by up to 25.7% on the COCO2014 dataset while preserving overall performance. Additionally, the authors demonstrate that their approach enables zero-shot segmentation by spatially localizing objects using internal confidence scores.", "questions": "- How sensitive is the proposed method to the selection of weight factors and layers across different models and datasets? Is there a way to generalize these hyperparameters or make the method more robust to their selection?\n- How does the method perform on other tasks, such as visual question answering (VQA) or on other datasets beyond COCO2014? Have you considered testing the method on benchmarks like LLaVA Bench or MM-Vet?\n- Is there a way to automate or simplify the selection of hyperparameters (e.g., layers, weight factors) to make the method more practical for real-world applications?", "limitations": "", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The paper introduces a novel application of the logit lens technique to interpret the internal image representations of VLMs, providing new insights into how these models process visual information.\n- The proposed knowledge erasure algorithm, PROJECTAWAY, is a simple yet effective method that relies solely on manipulating the internal features of the VLMs without requiring additional training or external modules.\n- The approach enables zero-shot segmentation by leveraging internal confidence scores to spatially localize objects\n- The paper seems clear and well-written.", "weaknesses": "- The proposed method requires specifying weight factors and selecting specific layers to retrieve text representations and apply edits. These hyperparameters are determined through ablation studies and do vary between models, and likely between datasets as well, requiring cumbersome ablation process to find good numbers.\n- The experiments focus primarily on object hallucinations in image captioning tasks. It is unclear how the method performs on other types of hallucinations (e.g., action or attribute hallucinations) or on other tasks such as visual question answering (VQA).\n- The impact of the method on overall caption quality is not thoroughly evaluated quantitatively. While the authors mention that the method preserves performance and provide some qualitative examples, additional quantitative evaluations would be interesting to see.\n- The authors only seem to test their model on COCO2014.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "PGWc5RRL1V", "reviewer": "Reviewer_bJVo", "summary": "The authors use Logit Lens to interpret the intermediate image representations in LVLMs. For a given image embedding, they extract the latent representation of the image embedding at a specific layer, taking the logit lens to get the probability distribution over the vocabulary. \n- The highest probability of an object across image representations and layers, can act as the internal confidence of VLMs. The confidences for objects present are significantly higher than those of objects not present in the image.\n- The authors propose an algorithm, ProjectAway, erasing objects from image representations.\n- Moreover, they find that, using the internal confidence values, they can localize the objects in the image patches.\n\nThe authors show three applications of their findings and the algorithm: hallucination detection, hallucination mitigation, and zero-shot segmentation.", "questions": "Please see the Weaknesses section.", "limitations": "", "rating": 8, "confidence": 5, "soundness": 4, "presentation": 4, "contribution": 3, "strengths": "- The findings are well-written and easy to understand.\n- The experiments are comprehensive, exploring different aspects of internal visual information and covering different tasks.\n- The proposed approach achieves significant improvements or comparable performance to SoTA on three applications.", "weaknesses": "### Major\n- Is the unembedding matrix for image representations directly from the LVLM last layer, or trained by the authors?\n- Previous papers report the modality gap between language and vision in VLMs. In my experiments, I also notice that the distribution of vision tokens are significantly different from that of textual tokens. So I’m surprised that the logit lens can be directly used in image representations. \nI’m curious about the classification accuracy of logit lens. For example, if we feed a patch of cat, how accurate is the logit lens method to identify it is cat.\n- Lines 200-202, the authors “randomly sample a subset of” objects not present. I’m wondering if this random sampling will choose some objects “obviously” not present in the image, making the comparison of the internal confidence too easy. It might be better if the authors can show: the confidence distribution of objects that commonly appear with objects in the image but not present this time.\n- Section 5.3, I think LLaVA tends to generate some very general class when classifying an image, like predicting \"dog\" instead of “husky”. Are the authors using the generated class name from LLaVA no matter what it is or using the ground truth label?\n\n### Minor\n- InstructBLIP and LLaVA are representative LVLMs, but recent LVLMs are using more complicated vision embedding techniques [1, 2]. I’m wondering if the proposed method can still work with these new architectures.\n- If we want to detect or remove the hallucinated objects, the propose method needs to know the object name. I'm wondering if the proposed method can work on a popular hallucination benchmark POPE [3]? In POPE, every sample is a \"yes or no\" question, like \"Is there a person in the image?\"\n- Other limitations like handling multi-token classes have been mentioned in the paper.\n\n[1] LLaVA-NeXT. https://llava-vl.github.io/blog/2024-01-30-llava-next/\n\n[2] Cambrian-1: A Fully Open, Vision-Centric Exploration of Multimodal LLMs https://arxiv.org/abs/2406.16860\n\n[3] Evaluating Object Hallucination in Large Vision-Language Models. https://arxiv.org/abs/2305.10355", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "W9igjHcpQ5", "reviewer": "Reviewer_4Zea", "summary": "This paper presents a novel approach to understanding and editing vision-language models' (VLMs) internal representations through vocabulary projection and linear orthogonalization. By introducing a knowledge erasure algorithm PROJECTAWAY, the authors demonstrate significant improvements in hallucination reduction (up to 25.7%) and achieve competitive performance in zero-shot segmentation, while providing new insights into how VLMs process visual information.", "questions": "1. The paper uses the model's unembedding matrix to interpret intermediate layer representations, but this matrix is trained for the final output layer. Have you conducted any layerwise probing or training of separate unembedding matrices for intermediate layers? This could affect the reliability of interpreting earlier layer representations.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "1. The paper presents a novel approach to interpreting and editing VLM representations through vocabulary projection and linear orthogonalization, requiring no model retraining or external components.\n2. The work provides insights into VLM behavior by revealing the relationship between internal confidence scores and object presence.", "weaknesses": "1. The paper's main analysis and evaluations (Sections 3 and 4) are predominantly conducted under the assumption that hallucinated objects are known beforehand using ground truth annotations. While Section 5 addresses this limitation with a more realistic approach using internal confidence thresholds, this should have been the primary evaluation framework. The current structure potentially overestimates the method's effectiveness by evaluating under idealized conditions.\n2. The paper's structure is suboptimal, with the main analysis focusing on scenarios using ground truth annotations while relegating the more realistic approach to the applications section. \n3. The choice to use the last token for multi-token object representations (e.g., \"hot dog\", \"dining table\") lacks sufficient justification and empirical validation. The paper does not analyze potential issues with this approach, such as cases where the last token might not be the most semantically meaningful (e.g., \"traffic light\" where \"light\" alone might be ambiguous) or how this choice affects the method's performance compared to alternatives like averaging all tokens or using the first token.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "a3XAHlEeX4", "reviewer": "Reviewer_LL1m", "summary": "The paper explores the internal representations of Vision-Language Models (VLMs) to address the persistent issue of hallucinations. The authors project VLMs' internal image representations onto their language vocabulary to identify differences in token output probabilities between real and hallucinated objects. They introduce a knowledge erasure algorithm, PROJECTAWAY, which removes hallucinations by linearly orthogonalizing image features with respect to hallucinated object features. The study demonstrates that targeted edits to a model's latent representations can reduce hallucinations while preserving performance. Additionally, the paper presents a method for zero-shot segmentation using the logit lens technique, showing comparable performance to state-of-the-art methods.", "questions": "-  Would the authors consider including comparisons with the latest MLLMs, such as those incorporating more advanced architectures or larger datasets, to validate the robustness of their approach?", "limitations": "", "rating": 5, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "- The paper presents a newmethod for reducing object hallucinations in VLMs by editing their latent representations and the introduction of PROJECTAWAY offers a new technique for selectively removing hallucinated objects from VLMs' outputs.\n\n- The authors provide a thorough analysis of the internal confidence values for object presence and absence, offering empirical evidence that supports their claims.", "weaknesses": "- While the paper focuses on object hallucinations, it does not explore the applicability of the methods to other elements of visual scenes, such as people, attributes, or actions. The editing approach may struggle with abstract or complex sentences involving object attributes or interactions, which are not explicitly addressed in the paper.\n\n- Could the authors elaborate on the potential impact of their editing techniques on other aspects of model performance, such as accuracy in non-hallucination tasks?\n\n- The paper's reliance on LLaVA and InstructBLIP as baseline MLLMs does not provide a comprehensive comparison with the latest state-of-the-art models.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)