Unleashing Region Understanding in Intermediate Layers for MLLM-based Referring Expression Generation
Authors: Yaoyuan Liang
Zhuojun Cai
Jian Xu
Guanbo Huang
Yiran Wang
Xiao Liang
Jiahao Liu
Ziran Li
Jingang Wang
Shao-Lun Huang
Venue: NeurIPS 2024 poster (2024)
Keywords: Vision-Language
Region-Level Understanding
Multimodal Large Language Model
Primary area: machine_vision

Abstract: The Multi-modal Large Language Model (MLLM) based Referring Expression Generation (REG) task has gained increasing popularity, which aims to generate an unambiguous text description that applies to exactly one object or region in the image by leveraging foundation models. We empirically found that there exists a potential trade-off between the detailedness and the correctness of the descriptions for the referring objects. On the one hand, generating sentences with more details is usually required in order to provide more precise object descriptions. On the other hand, complicated sentences could easily increase the probability of hallucinations. To address this issue, we propose a training-free framework, named ``unleash-then-eliminate'', which first elicits the latent information in the intermediate layers, and then adopts a cycle-consistency-based decoding method to alleviate the production of hallucinations. Furthermore, to reduce the computational load of cycle-consistency-based decoding, we devise a Probing-based Importance Estimation method to statistically estimate the importance weights of intermediate layers within a subset. These importance weights are then incorporated into the decoding process over the entire dataset, intervening in the next token prediction from intermediate layers.
Extensive experiments conducted on the RefCOCOg and PHD benchmarks show that our proposed framework could outperform existing methods on both semantic and hallucination-related metrics. Code will be made available in https://github.com/Glupayy/unleash-eliminate.

Unleashing Region Understanding in Intermediate Layers for MLLM-based Referring Expression Generation
Yaoyuan Zhuojun
Zhuojun Cai Jian Xu', Guanbo Huang', Yiran Wang', Xiao Liang', Jiahao Liu?, Ziran Li?, Jingang Wang', Shao-Lun Huang" TTiinguaa Shenzhen International Graduate School, Tsinghua University Meituan Inc.
Abstract
The Multi-modal Large Language Model (MLLM) based Referring Expression Generation (REG) task has gained increasing popularity, which aims to generate unambiguous text description that applies to exactly one object region in the image by leveraging foundation models. We empirically found that there exists potential trade-off between the detailedness and the correctness of the descriptions for the referring objects. On the one hand, generating sentences with more details is usually required in order to provide more precise object descriptions. On the other hand, complicated sentences could easily increase the probability of hallucinations. To address this issue, we propose a training-free free framework, named as "unleash- teeesshhhee--eee seien-elminate,, which first elicits the latent information in the intermediate layers, and then adopts cycle-consistency based decoding method to alleviate the production of hallucinations. Furthermore, to reduce the computational load of cyylleconsisteenyy-oe based decoding, we devise Probing-based Importance Estimation method to statistically estimate the importance weights of intermediate layers within subset. These importance weights are then incorporated into the decoding process over the entire dataset, intervening in the next token prediction from intermediate layers. Extensive experiments conducted the RefCOCOg and PHD benchmarks show that our proposed framework could outperform existing methods on both semantic and hallucinatioon-elatee metrics. Code will be made available in https: ssttps://githie //github.. com/ Semm//llaayy//ll halaayyyuuneaashh..o eliminate.
Introduction
53 54 is task generate unambiguous text Referring expression generation (REG) 29 31
description that applies to exactly one appointed object or region in the image. A good expression should be distinguishable enough to ensure that the listener can identify the unique target among various objects within the same image. With the great success achieved by large language models (LLMs), multi-moda large language models (MLLMs) ( 12 perform this task and become increasingly popular in the research community, Some representative works 6 37l 52 55 57] conduct visual instruction tuning specialized region-involved multi-modal corpus and successfully empower MLLMs with the ability of region-level understanding.
27
[56 61] have been introduced to
Though certain progress has been made, MLLMs themselves suffer from object hallucination 14
23
46 50 60). Research suggests that the mechanism behind MILMs* hallucinations could be related to the over-reliance on prior knowledge of LLMSs rather than the multi-modal context provided by
"Equal contribution.
Cooresponniigg author.
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

(a) Cycle-consistency-Be -bsed quality ranking
Unleash
Eliminate
output from intermediate layers
REG MLLM
1. mullin with chocolate chips
RES MLLM
2. the top of chocolate muftin in front of other muffins
3. donut with chips the foreground
Finel output:
Image Mask
top chocolate muftin front other muttins
(b) Probing-based importance estimation
REG MLLM
RES MLLM
REG MLLM
MZEM
output
Cycling probing
Layer prior importance
Generate without RES
Illustration of our method. (a) Given an image and a mask appointed to the target object, we first unleash the descriptive regional information of the middle layers and gain various candidate captions. These outputs are sent to RES model that serves a "listener" and the "listener" eliminates the inaccurate candidates. (b) Our proposed strategy to diminish the computational load of RES. First estimate the layer prior importance on a probe set with RES, then leverage it for the RES-free next token prediction.
Figure 1:
the input 14 19 50 Therefore, MLLM-based REG models naturally inherit the aforementioned hallucination issues. Additionally, a further complication arises in the context of region-level understanding, where models are required to generate precise and identifiable descriptions for specific regions. To achieve that, the MLLMs sometimes have to make use of the surroundings for reference while effectively avoiding out-of-region information distorting the description of the target object, e.g. incorrectly attributing characteristics of other objects to the targeted region (Figure 7 appendix). In this paper, we aim to explore this potential trade-off between the detailed description and accurate targeting of referring objects in MLLM-based REG task. To be specific. providing precise object descriptions necessitates generating sentences with more details, which results in longer sentences. Meanwhile, the text with increased granularity (length) more likely to contain inaccurate or spurious information, commonly defined as "hallucinations"" Table I shows an example of quantitative analysis.
This requirement exacerbates the issue of attribute-level hallucinations B 50
To address the trade-off between information richness and reliability, we propose a novel approach called sunleash-then-elminee level multi-modal information does not maintain monotonic progression during inter-layer tran- sitions. Moreover, when multi- hidden states are projected into the language space using the language head, the intermediate layers sometimes hold more descriptive region information than the final layer. These observations (Section 3.2) imply that the most suitable layer for each referred region should not be solely confined the final layer. Accordingly, we adopt contrastive decoding X 22 to unleash the object information contained in the intermediate layers. To eliminate unsuitable candidate outputs, we leverage the dual task of REG: Referring expression segmentation (RES) 37 48, which aims to segment target object mask from the entire image given sentence describing the object. Ideally, for the same object, the region input (formed as a mask) for the REG task and the mask output from the RES task should exhibit cycle consistency. Based this insight, we propose syple-coniistenc-baseee decoding method, which enables us to choose among multiple outputs based on their descriptive quality thus reducing hallucinations while maintaining the richness of the output sentences. Furthermore, considering the need to diminish the computational load of RES, we develop hybrid layer importance measurement strategy to select the best layer for each token during the next word prediction. This strategy leverages both the layer-wise prior importance estimated over probing subset, and the CShsennShannon divergence [ between the logits of each candidate layer and the last layer. With the layer-wise prior importance, the MLL.M-bssed REG model maintains promising performance in reducing hallucinations and enhancing the granularity of the generated sentences, even without the assistance of RES model. Extensive experiments conducted on the RefCOCOg (33] and PHD 28 benchmark demonstrate that our proposed framework surpasses existing methods on both semantic and hallucination--rlatee metrics.
as depicted in Figure We observed that the aligmment of region-
2 Related works
Region-level understanding in Multimodal Large Language Models. Significant progress has been made in unlcashing the region-level understanding ability in MLLMs [6
18 37
52
55 57).
To

---

incorporate region-level information into sequence generation of MLLMs, some approaches [6 52 integrate bounding box coordinates into the language input in the form of natural language prompts. Ferret [52) proposes spatial-aware visual sampler that enables the arbitrary shapes of visual prompts. The current proposed Osprey (55) unlocked the capability of pixel understanding. alleviating the influence of irrelevant information in the visual prompt inputs. These methods have propelled the REG task into the era of MLLMs. Considering that MLLMs incorporate extensive knowledge from unimodal and multimodal pre-trained corpora, there is potential for the REG task to leverage this inherent knowledge within the models to generate more specific and detailed expressions without additional training, thus more effectively addressing real-world applications 21 39
Decoding strategies to mitigate hallucination in Large Language Models. Large language models are pre-trained on unlabeled corpora to aequire extensive world knowledge and subsequently undergo post-training to learn to follow instructions 34 and align with human preferences [2]. This systematic pre- and post-training pipeline makes them powerful at solving a wide range of
NLP tasks [ 36 40 45 However, some studies indicate that they may fail to accurately assess and often exhibit overconfidence in their responses 447, which results in hallucinations [59]. To mitigate these issues, recent research S 8 20 proposes inference-time decoding strategies for trained LLMs to find latent knowledge inside the internal activations without additional training. C. Burns et al. [5] introduce a Contrast Corraastt-onitteett Search (CCS) algorithm to identify a direction of truth in the activation space of LLMs that remains consistent across negations, thereby reducing generated errors. Based on the discovery of CCS, ITI [20] dives deep into attention heads and suggests shifting model activations alongside factuality-related heads during inference to help reduce hallucinations. Besides "finding the direction of truth," Dol.a [8] proposes contrastive decoding by comparing the differences in logits between the projections of later and earlier layers to better surface factual knowledge and reduce the generation of incorrect facts. In line with the motivations of (5] 20). our investigation uncovers that well-trained MLLMs' intermediate layers differ in multi-modal alignment and region-level understanding capabilities. These observations inspire us devise inference-time decoding strategy that combines latent knowledge from multiple layers (with prior importance) to alleviate hallucinations.
their own knowledge [1
Hallucination in Multi-modal Large Language Models. In the realm of MLLMs, "hallucination" typically refers to "object hallucination," where the models generate plausible outputs containing ob-
jects that either absent from or mismatched with the images [15] 19 24
38
50 and is commonly
categorized into three types: category, attribute and relation hallucinations 3 49). Some efforts based instruction tuning have been made to mitigate this issue in MILMs. IRV-Instrcction (26 introduces dataset with positive instructions and unique negative prompts with different semantic levels to better align responses with image content. HACL [15] explores the vision- lision-language embed- ding space, using contrastive learning to separate non-hallucinated from hallucinated texts. Without instruction tuning, Woodpecker [50) offers a training-free frree pipeline for hallucination correction, using expert models to enrich image context and ensuring each phase is interpretable by step-by-step correction process. In another line of work. some efforts have sought new decoding strategies to avoid relying on extensive additional data and training: Opera [14] tackles the partial over-trust issue in decoding by applying a penalty to the model logits during beam-search decoding. VCD [99] links object hallucinations to biases and language priors, contrasting outputs from distorted and original visuals to ensure consistent generation. Compared existing studies, our method initially enriches region-level context from intermediate layers rather than external expert models or knowledge base and provides multi-layer ensembling solution to mitigate hallucinations.
3 Method
3.1 Preliminary
Leveraging the nuanced language representation capabilities inherent in multi-modal large language models, recent investigations have unlocked the REG 6 37 52 555 and RES [8, 37 capabilities of MLLMs. To maintain clarity and conciseness, have omitted the discussion on instruction tuning and assume that the described tasks are well-instructed by specific prompts. In this work, we utilize a "mask" as the region prompt.
Referring expression generation. The M-.sed REG model is typically structured around three principal components: visual encoder (e.g., CLIP (35) with linear adapter), region-encoder,
3

---

and an LLM base (e.g.. LLaMA 42 or Vicuna 77) Given an image I, the visual encoder first extracts the visual information, then the linear adapter projects it into I E R #W"x, where HW denotes the flattened visual token length and C' denotes the hidden state dimension of LLM base. referred region M (presented as a mask) is encoded as M E TR by region encoder (e.g.. a CNN-based extractor [55] or an ROI pooling layer [37 (7), where Q denotes the length of encoded region prompt tokens. Taking the visual feature f regional feature My, and an embedded instruction text X E RR as input, where L represents the length of the instruction tokens, the model concatenates them and forms a multi-modal input sequence S E IR(H++++L)x;
Regarding the core architecture of the LLM base, the multi-modal sequence S is successively processed by the N stacked transformer layers. Eventually, an affine layer Φ(-) serves as language model head to predict the probability of the next token over the vocabulary set V. The logits for the token Vt, given the sequence S and all preceding tokens ye are computed follows:
logit N (We (S, yeee) = ((h, (N), y/ EV,
(I)
where ( denotes the hidden state of the last transformer layer. The probability of the next token ye is then given by:
p(y|)S, y = softmax (logit ( S. y<t)), Yt E V.
(2)
Through this process, the model autoregressively generates the output text Y as the region descrip- tion. For simplicity, we use ((3el to represent p(y [S, yet), and logit (VolDes) ) to represent logit (3v (S, V<ce) in the following.
Referring expression segmentation. Recent advances in MLLM-based RES models [8] 37 largely employ a similar MILM architecture (e.g., LLaVA 2 as used in the REG models, with the distinction that its input contains description targeted at specific region, while its output is mask that can cover the described object.
widely adopted strategy 17
I 37 incorporates [SEG] token, enabling the model to identify
the [SEG] token in the output sequence as a cue for the presence of a segmentation target. A specialized MLP head y processes the output embedding of the [SEG] token hegg E RR mapping it into the prompt space of the segmentation fundamental model (c.g., SAM (17]), represented as heeg- (heeg)) The segmentation model then decodes the target mask M, from the query token haeg and provides its confidence score CF.
3.2 The intermediate layer contains descriptive information
This section focuses on our observations of intermediate layers, attempting uncover the latent descriptive information. We adopted region understanding model Osprey-7b (55] with N = 32
layers as REG model, and GLaMM
37 as RES model by default.
Each layer has different generation tendency. To reveal the latent information, we adopt the early-exit strategy on the REG model generate a series of output sequences by applying the language model head (an affine layer) to the hidden states of each layer. As illustrated in Figure 2 the sequences from the early layers (layers 1 to 10) manifest as nonsensical strings of characters, indicating the suboptimal alignment between the hidden representation of shallow layers and the ultimate vocabulary space of MILM. Interestingly, among the middle layers (about layers 20 25), the model begins to output sentences with semantic meaning and gradually some region-specific expressions appear. Compared with the output of the last layer, these sentences with richer semantic information can sometimes provide more discriminative descriptions for the referred object, e.g. indicating the related position between the target object and others, and/or containing high granularity of attributes. The uniqueness and descriptiveness of expressions given by the middle layers are strongly related to the objective of REG, thus the latent features of intermediate layers have notable potential to enhance the unambiguity in MLL.-baased REG.
Potential of intermediate layers. The above observations demonstrate that different layers' under- standing of the region context varies. We attempted to visualize the region-aware comprehension capabilities among the layers through two approaches. (a) With the assistance of the RES model: we randomly extracted K = 2000 samples from the RefCOCOg training set to form the triplets (I, M. Y). which corresponds to the entire images I, the target regions represented by masks M. and the descriptions of regions Y. (I, Y) are sent into an MLLM-based RES model, to harvest the projected features of [SEG] token heeg- Subsequently, (I, M) are input into the REG model, and the

---

Vinual
Turken
Taken
Layer Output:
layar
layer
20th Layer
22nd layer
layar plant in planter fore- ground
layer
HP
datum
Apozs in
Himwesborr
wttch beroeeer
green plant
bwiliind
green plant
the tallest tree the
occupled woman
viet
Figure 2: Different layers' understanding of the region context varies, where early layers generate rubbish, middle layers tend to generate descriptive text with higher granularity, and final layers lend to predict shorter and more precise text. The right part shows the frequencies with which the hidden state of each layer had the smallest Wasserstein-2 distance to the [SEG] token (in orange), and the inter-layer transitions of region-level multi-modal alignment (in blue).
hidden states of the last token of each layer { i=1 are obtained. Inspired by FID metric 13 and recent investigations the latent space communication (30 32], first performed PCA dimension- ality reduction to project both features into the same dimension, then treat the dimension-reduced
between [SEG] and each intermediate hidden states. This allowed us to estimate the relative region understanding ability of each layer for the given multi-modal modal context of a certain object. We then calculate the frequency with which each layer had the smallest Wasserstein-2 2 distance to the [SEG] token. As depicted in the right of Figure [ (orange), except for the final layer, the intermediate layers also have the potential to contain better latent information for a more precise region-related description. (b) Solely within the REG model: investigated the multi-modal alignment process in region-level context across intermediate layers by calculating the Wasserstein-2 distance between each layer's region-encoded token [mask] and the last language token. Results are reported in Figure (blue). These analyses within a well-trained MLLM show that the tokens of different modalities do not approach monotonically during inter-layer transitions. Hence we should give more chances the intermediate layers for better region-level understanding. Detailed information can be found in the Appendix
[SEG] features the anchor calculate the Wasserstein-2 distances [13 44 45)
3.3 Unleash then eliminate decoding strategy
To enhance the granularity of region-level descriptions without introducing excessive hallucinations, propose method that integrates contrastive decoding with cycle consistency -bsed ranking to screen out appropriate descriptions for the interested regions (Section 3 and 3.3.2). This approach enables us specifically leverage the commonly overlooked latent information contained in intermediate layers and ensures the identifiable description through caption quality estimation. Under the concerns of computation efficiency, we further develop decoding strategy to reduce the operations of cycle ranking through hybrid layer importance measurement (Section 3.3.3). It involves two kinds of layer importance calculation influence the selection probability of candidate layers during each word prediction step. The harvest prior importance weights can be directly applied to the decoding process of the original MILMM, mitigating the hallucinations. The following subsections provide further details.
3.3.1 Unleash intermediate information by contrastive decoding
As depicted in Figure 2, the manifestation of region- aware information differs across layers. To underscore descriptions related to specific regions evident the final layer, we adopt a contrastive decoding approach g 22 by subtracting the log probabilities of the next token produced by the intermediate layer from those of the final layer. The resulting distribution defined as the contrastive decoding logits of specific subtractor layers. These logits instead of originally the logits from the final layer are used for generating the subsequent token. Concretely, given a set of candidate layer indices = (I,...,n). the probability token ye for layer E J is:
evident in the intermediate layers but faded in
Peony (ye (ucc)) N ) logit, (vi (e()) if yt Vhead,
otherwise,
5

---

Candidate Ranking
REG
Candidate layers
MASK
1st layer
32nd layer
Comparison
1oU
M
Contrastive Decoding
Mask Decoder
LLM
Caption
Caption
Caption
[SEG]
couch facing
empty black
the cary reading chair
RES
the fire place
leuther chair with
wood frame
Figure 3: First, we conduct contrastive decoding by subtracting the log probabilities produced by intermedi- ate layer from those of the final layer. Then, each of the intermediate layers in the candidate subset suggests a probability for predicting the next token. Finally, RES model is used to estimate the performance of the captions by calculating the Intersection over Union (IoU) between MASK and Mj. ranking these candidates effectively.
max p(w) W<))), [0, is cutoff hyperparameter that truncates the next token distribution p(y:lv<)) of the final layer. Following the previous works 22), we set a = 0.1 in the implementation. After contrastive decoding, each of the intermediate layers in the candidate set suggests probability for predicting the next token yr- Finally, we harvest a sentence set A {ar Q2 a, whose size equals to the size of candidate layers.
where logit, (3elv=.) )(h (head(y)) {ys V p(y:leee)
3.3.2 Cycle-consistency-beee intermediate sentence quality ranking
Consider the formed triplet (I. M, Y) in Section where I serves the global contextual background of a region, M and Y respectively represent two modalities of the same object. For pair of ideal REG and RES models, is anticipated that M and Y can be interconverted losslessly during a cyclic operation of these two models. This implies that, due the cycle consistency between the two modalities of the same object, feeding the output of the REG model into the RES model should yield mask consistent with the input to the REG. If the output generated from the candidate layer overly ambiguous polluted by hallucinations, the RES model may struggle with accurately locating the target object against the background. Based on these assumptions and observations, we utilize RES model [37] to estimate the region understanding performance of the captions generated by the candidate layers, allowing us to rank these candidates effectively.
Figure iillustrates the pipeline of sycle-consistency-baeee based quality ranking. The input image 7 and the referred region (represented as mask) are processed by the REG model, which continuously extracts and aligns features across successive layers for the multi-modal context. After the information elicitation intermediate layers, we harvest a set of sequence A {ai, a2 a. Each sequence in A, paired with the image 1, forms input pair (1, a,) to feed into the RES model. The RES model then segments out the corresponding mask MA for each input pair.
We evaluate the quality of each layer's sentences by calculating the Intersection over Union (IoU) between M and My:
M Mj M MUMM MÇ
Score, = CF}
where CF2 refers to the output ToU score head within the segmentation foundation model (e.g. confidence score of the generated mask. Subsequently, the candidate sentences from each layer are ranked by the Score,. For each sample, the a; with the highest Score, is selected the final sentence, and its layer is deemed the best candidate layer.
SAM 17 which can be interpreted

---

Algorithm Layer Prior Importance Calculation
1: Input: Score Rm
D m: Number of subset samples; n: Number candidate layers
2: Output: q E " Zj=1 Qj
=1=1 1=
P Importance weight of each candidate layer > counts of times each layer has the maximum score
3: Initialize count f- 0 E
4: for 6- 1 to n do
5: count, = 1=1
1 [ arg maxjeJ Scores.)
6: qj
couRE TR
7: return
3.3.3 Hybrid layer importance measurement
Although our ceree-coonssteec-baaseeee candidate ranking process improves the generation quality, it introduces additional computational load from the RES model compared the original decoding method, significantly affecting the per-sample decoding speed. To alleviate this issue, we propose simple yet effective strategy called Probing-based Importance Estimation speed up the decoding process. This strategy involves frequency counting of each candidate layer being the optimal layer based on their performance within a probing subset. With subset containing m samples, the specifics of this calculation are outlined at Algorithm [ The estimated weight q is then served as prior knowledge that reflects the candidate layers' importance and is then utilized over the entire dataset in the decoding process, intervening in the next-word prediction.
Furthermore, inspired by the success of iettance-guiders layer selection in LLMS 8 also apply this metric as a second guidance. Concretely, we first calculate the distance between the next-token probability of the final layer and each candidate layer the current decoding step, then the calculated values are normalized across candidate layers as follows,
d, = D2
(5)
Li=1 D;
where D, = JSD(pl Peon,) denotes the J-S divergence between the next-token probability of the final layer p and the layer Peon, Finally, the hybrid layer importance obtained by adding up the probing-bas basedd prior and sample-wise distance followed by softmax normalization:
softmax + q),
(6)
where is a vector of normalized divergence values across the candidate layers. We then sample from the probability distribution q to select one layer among candidates to predict the next token at each decoding step, till the generation finishes.
Overall, this decoding approach first estimates a prior layer- layer-wise importance weight from small subset and then applies this distribution to contrastive decoding, effectively improving decoding efficiency while preserving the ability to mitigate hallucinations. In addition, our experiments (Table 3 also show that the prior importance q calculated from one (probing) dataset could be directly transferred to another dataset with similar image domain for reducing the hallucinations. This strong transferability further illuminates a new promising application scenario where the historic estimate could serve cold-start for inference in new environments when the probing dataset is not available.
4 Experiments
4.1 Datasets and metrics
RefCOCOg, The RefCOCOg 33 dataset is a classical benchmark for referring expression gen- eration, which contains 85,474 expressions for 54,822 objects in 26,711 images. Expressions in RefCOCOg are annotated on Amazon Mechanical Turk in non-competitive way and tend to be longer (8.43 words per sentence on average) and more expressive than RefCOCO (53] and Ref- COCO+ [53]. In the MLLMMempowered REG task, the relatively short ground truth cannot fully cover the expression space of the sentences generated by the MILMM at evaluation. Therefore, our experiments focus on the METEOR metric as is more comprehensive and flexible, allowing for more nuanced recognition of linguistic variations.
7

---

Table 1: Comparison of generation and hallucination performance on RefCOCOg. f denotes the temperature parameter. "1/8" denotes using 1/8 samples of the total dataset to estimate the layer prior importance. "full-R" denotes quality ranking on the whole dataset.
Model
METEOR1
CHAIRg +
CHAIR;
Recall
Len
nCHAIRs s +
nCHAIR; +
Osprey-7h (l=0.2)
162.0
23,41
20.81
0.7631
7.15
3.2741
2,9105
Osprey-7b (I=0.9)
140.0
27.90
24.12
0.7514
8.11
3,4402
2.9741
DoLa
168.0
43.44
31.78
0.8196
23.07
18830
1.3775
Ours (1/8)
172.0
42.25
30.95
0.8211
22.96
1.8406
1.3484
Ours (full-R)
173.0
42.40
31.20
0.8237
23.16
1.8307
1.3472
CHAIR Evaluation on Hallucinations. The Caption Hallucination Assessment with Image Rele- vance (CHAIR) metric (38) is commonly used evaluate object hallucinations that occur in image description tasks. It comprises two distinct assessment dimensions, including CHAIRs that calculates on sentence-level and CHAIR, that calculates on more granular object-level. We observe that CHAIR only counts hallucinated objects for each "central object", which means that the model wants to enrich the semantic information and generate new tokens, it will face the risk of increasing hallucinations compared to the shorter-sentence models. This undermines the comparability of this metric across sentences of varying lengths. Hence we propose normalized CHAIR (nCHAIR) based on CHAIR to conduct fair comparison between sentences of different lengths. The calculation of nCHAIRs is specified as the following formula, with nCHAIR, calculated similarly:
Khallucinated sentences] }|
nCHAIRs
(7)
[avveraee number of tokens per sentence x |all sentences))
|
Prompted Visual Hallucination Evaluation Benchmark (PHD). This benchmark 28 focuses
on the four major types of hallucination faced by MLLMs, namely Object hallucination, Attribute hallucination, Multi-modal conflicting hallucination, and Counter-common-saee evaluates and explores the hallucinations through comprehensive prompt-based tasks, which also helps identify the causes of these hallucinations. While this benchmark does not explicitly evaluate region- level hallucinations, its detailed evaluating strategies for object attribute and position are closely related to region-level understanding, allowing it to effectively indicate regional- hallucinations.
hallucination.
4.2 Main result on referring expression generation
Baseline region-level MLLM model and decoding method. Table presents the performance comparison of semantic quality and hallucination evaluation on RefCOCOg dataset, with METFOR and CHAIR metrics. Recent research (58) has revealed that the temperature parameter has notable effect on the hallucination of generated sentences. In light of this, we included an analysis of the baseline region-level MLLM model, Osprey-7b [55], performing both lower 0.2) and higher (t = 0.9) temperature settings. Meanwhile, we also compared a baseline decoding method, DoLa [8 which demonstrated that leveraging contrastive decoding in vanilla LLMs can enhance the authenticity of the generated results. In comparison to the Osprey-7b [. the sentences generated by DoLa demonstrated better performance on METEOR and nCHAIR metries.
Performance of sycle-consistency-baseee sentence quality ranking, As introduced in Section 3.3.2 to balance the trade-off between information granularity and accuracy, first unleashed the region description of intermediate layers by contrastive decoding, then filtered out the inaccurate sentences by wele-cconisteeccy-aseeee quality ranking. For each sample in RefCOCOg, we evaluated the best sentence generated by the best candidate layer. The result is reported in the fifth row of Table I We observed that our method not only gains more descriptive sentences but also demonstrates a reduction in the hallucination metric (measured in terms of nCHAIR) compared to Osprey-7b 7b and DoLa. The examples of generation results are listed in Figure
Text generation via hybrid layer importance measurement. In Section 3.3.3 we proposed a method that effectively reduces the computational overhead of the RES scoring model. In our experi- ment, we divided all 32 intermediate layers (including the embedding layer) into four consecutive groups (detailed in Section 4.3) and calculated the relative importance of the layers within each group
'The version without RefCOCOg fine-tun fine-tuning was used. https:://nugginggace co/ Sunshine-wwtt Osprey- Paarry--bbbtree/maii.
8

---

Table 2: The impact of different candidate layer groups (buckets) on the performance of RefCOCOg dataset.
Bucket
METEORT
CHAIRs
CHAIR; |
Recall
Len
nCHAIRs
nCHAIR, 4
1st (0-7)
173.0
42.40
31.20
0.8237
23.16
1.8307
1.3472
2nd (8-15)
167.0
40.89
30.49
0.8236
21.58
1.8948
1.4129
3rd (16-23)
146.0
37.36
30.07
0.7797
18.96
1.9705
1.5860
4th (24-31)
127.0
31.50
27.52
0.7357
17.35
1.8156
1.5862
Osprey-7h: A wooden table that holds buby and
Osprey-7h: sandwich roll.
Osprey-7h: pizza with white cheese
Osprey-7h: The Indy checked blouse.
Dola: The right side of the mear,
Dola: A brown wooden table in the foreground the photo, wanh baby sitting
sandwich cheese. letinee tomalo, and
Dola: A pizza with spinach and being
the tell wearing a plaid shirt and fringe skirt. She has large helr buckle mocklace.
Dola:
The
tray. One man holding slice of the loppings
while the other adds
Ours: A large brown wooden dining room table dominates the The table is littered with various objects including bowl with chips, bottle. spoon and cup.
A piece of visible the right side plate. This bread appears to bun and is positioned in front of some tomnsoous.
the left has long dark hair and wearing plaid shirt She's standing to the molorcycle and appears to be a posed model.
Ours: The
pizza left, heing picked up. The pizz appears spinach topped with tomatoes
Figure 4: The visualization comparison of generations between Osprey-7b (55). DoLa (8] and ours. Osprey-7b's outputs are quite brief and omit visual details referred objects. The generations from DoLa are more extensive but are accompanied by hallucinations. In contrast, our method increases descriptive information and curtails the generation of hallucinations.
separately. Figure 5) depicts the resulting layer importance weights four different ranges. The result reveals that layer importance is not uniform across candidate layers and also varies among different groups. We applied the importance weights estimated from 1/8 subset to the decoding process and listed the first group's result in the fourth row of Table [ Notably, in comparison full-set sentence-by-senteen
quality ranking, this probing-based bbaseg decoding method offers compara- ble performance with improved efficiency, demonstrating the effectiveness of our statistical layer importance estimation.
4.3 Performance of different candidate layer groups
was observed that different layers have distinct generation preferences for the same sample, inspiring further exploration in quantitative experiments. The first 32 layers (where layer 0 is the embedding layer) of the Osprey-7b model were organized into four groups: [O, 7], [8. 15], [16, 23], and [24, 31]. In each experiment, one group selected the candidate layer set and contrasted with the last layer to generate the candidate sentences, followed by sycle-consittenyy-beeee
As aforementioned in Section 3.2
ranking choose the best. Table indicates that the first group exhibits the best performance in METEOR and nCHAIR, metrics. It is also noticeable that there are considerable performance discrepancies between the different groups, with general trend of semantic quality declining as the depth of the layer increases (after contrastive decoding). However, we found that although the fourth group had the lowest METEOR score, its nCHAIRs was slightly better than that of the first group. This observation suggests that well-trained MLLM exhibits variations in latent knowledge across its zones, leading differences in generation performance.
0.20
Layer importance
0.10
whole dataset
on 1/8 subset
0.05
5
10
15
20
25
30
Layer index
Figure 5: The layer prior importance measured by our method of different groups. observable that weight distributions of the 1/8 subset and the full dataset have similar trends.

---

Table 3: The comparison Prompted Visual Hallucination Evaluation Benchmark(PHD)
Object Recognition
Attribbuee Recognition
Sentiment Understanding
Positional Reasoning
Counting
Average
mest-mode:Nuurall
Osprey(t=0.9)
67.57
67.71
69.30
63.91
71.30
67.96
(t=
69.57
69.79
69.71
67.46
73.00
69.91
DoLa
69.19
68.90
67.96
73.37
71.93
70.27
Ours
70.27
70.34
70.91
68.64
74.26
70.88
Test-mode: Misleuding
Osprey((=0.9)
65.30
64.67
69.54
63.45
65.59
65.71
(tprey(t=0.22)
67.47
66.67
70.95
66.37
68.15
67.92
DoLa
67.38
67.31
70.62
63.16
67.88
67.27
Ours
68.33
67.64
72.16
66.67
68.64
68.69
4.4 Transferability of layer prior importance weights
From the preceding experiments, we discovered that using layer importance in the next token prediction enhances the generated output's 's quality of the same dataset. Additionally, also noticed that the determined layer prior importance can be smoothly transferred to a different dataset (different prompts) sharing similar image domain reduce the hallucinatory outputs. Specifically, we applied the layer importance ([O, 7] group) calculated on the RefCOCOg dataset directly to the PHD dataset, where the image inputs also originate from MSCOCO (25. and decoded the output using the method described in Section 3.3.3] Given the absence of any region prompt in the PHD dataset, we applied a zero mask to the region encoder. The comparative results are shown in Table We observe that our approach enhances the model's understanding across most tasks, which indicates that without additional training, the estimated layer prior importance could serve as cold-start for inference in new environments when the probing dataset is not available, helping reduce the occurrence of hallucinations during decoding.
Conclusion
Our research the MLLM- based Referring Expression Generation (REG) task explored potential trade-off between information richness and reliability of the intermediate generation results. We introduced
approach that utilizes latent information from intermediate layers and employs a sycle-consistency-hoee decoding method to reduce hallucinations. Our method outperforms existing techniques, confirming its efficacy for enhancing REG task performance.
Acknowledgements
The research of Shao-Lun Huang is supported in part by National Key R&D Program of China under Grant 2021 YFA0715202. Shenzhen Ubiquitous Data Enabling Key Lab under Grant 408152002003777100000 the Shenzhen Science and Technology Program under Grant KQTID201708101 146 and Meituan.
10

---

References
[1] Jinze Bai, Shuai Bai, Shusheng Yang. Shijie Wang, Sinan Tan, Peng Wang, Junyang Lin, Chang Zhou, and Jingren Zhou. Qwen-vl: A frontier large vision-language model with versatile abilities. arXiv preprint arXiv:2308. /2966, 2023.
[2] Yuntao Bai, Andy Jones, Kamal Ndousse, Amanda Askell, Anna Chen, Nova DasSarma, Dawn Drain, Stanislav Fort, Deep Ganguli, Tom Henighan, al. Training a helpful and harmless assistant with reinforcement learning from human feedback. arXiv preprint arXiv:2204.05862, 2022.
[3] Zechen Bai, Pichao Wang, Tianjun Xiao, Tong He, Zongbo Han, Zheng Zhang, and Mike Zheng Shou. Hallucination of multimodal large language models: A survey. arXiv preprint 24KK::2404.189300 2024.
[4] Yelysei Bondarenko, Markus Nagel, and Tijmen Blankevoort. Quantizable transformers: Removing outliers by helping attention heads do nothing. Advances Neural Information Processing Systems, 36, 2024.
[5] Collin Burns, Haotian Ye, Dan Klein, and Jacob Steinhardt, Discovering latent knowledge in language models without supervision. arXiv preprint arXiv:22/2.03822 2022.
[6] Keqin Chen, Zhao Zhang, Weili Zeng, Richong Zhang, Feng Zhu, and Rui Zhao. Shikra: Unleashing multimodal Ilm's referential dialogue magic. arXiv preprint arXiv:2306.15199 2023.
[7] Wei-Lin Chiang, Zhuohan Li, Zi Lin, Ying Sheng, Zhanghao Wu, Hao Zhang, Lianmin Zheng, Siyuan Zhuang, Yonghao Zhuang, Joseph E Gonzalez, et al. Vicuna: An open-source chatbot impressing gpt-4 with 90%* chatgpt quality. See https://vicua. lmsys. org (accessed 14 April 2023), 2023.
[8] Yung-Sung Chuang, Yujia Xie, Hongyin Luo, Yoon Kim, James Glass, and Pengcheng He. Dola: Decoding by contrasting layers improves factuality in large language models. arXiv preprint arXiv:2309.0388 03883, 2023.
[9] Wenliang Dai, Junnan Li, Dongxu Li, Anthony Meng Huat Tiong, Junqi Zhao, Weisheng Wang, Boyang Li, Pascale Fung, and Steven Hoi. Instructblip: Towards general-purpose vision-language models with instruction tuning, 2023.
[10] Timothée Darcet, Maxime Oquab, Julien Mairal, and Piotr Bojanowski. Vision transformers need registers. The Twelfth International Conference on Learning Representations, 2023.
[11] Henghui Ding, Chang Liu, Suchen Wang, and Xudong Jiang. Vision-language transformer and query generation for referring segmentation. In ICCV, 2021.
[12] Xiaoyi Dong, Pan Zhang, Yuhang Zang, Yuhang Cao, Bin Wang, Linke Ouyang, Xilin Wei, Songyang Zhang, Haodong Duan, Maosong Cao, et al. Internlm-xcomposer2 Mastering free-form text-image composition and comprehension in vasion-language large model. arXiv preprint 20X1v22401.16420, 2024.
[13] Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by a two time-scale update rule converge to a local nash equilibrium. Advances in neural information processing systems, 30, 2017.
[14] Qidong Huang, Xiaoyi Dong, Pan Zhang, Bin Wang, Conghui He, Jiaqi Wang, Dahua Lin, Weim- ing Zhang, and Nenghai Yu. Opera: Alleviating hallucination in multi-modal large language models via over-trust penalty and citrospection-allocation allocation. arXiv preprint 171.913311.19911. 2023.
[15] Chaoya Jiang, Haiyang Xu, Mengfan Dong, Jiaxing Chen, Wei Ye, Ming Yan, Qinghao Ye, Ji Zhang, Fei Huang, and Shikun Zhang. Hallucination augmented contrastive learning for multimodal large language model. In Proceedings of the IEEF/CVF Conference on Computer Vision and Pattern Recognition, pages 27036-27046, 2024.

---

[16] Mandar Joshi, Eunsol Choi, Daniel S Weld, and Luke Zettlemoyer. Triviaqa: A large scale distantly supervised challenge dataset for reading comprehension. arXiv preprint arXiv: /tap//005.03555 2017.
[17] Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer Whitehead, Alexander Berg, Wan-Yen Yen Lo, et al. Segment anything. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 4015 4026, 2023.
[18) Xin Lai, Zhuotao Tian, Yukang Chen, Yanwei Li, Yuhui Yuan, Shu Liu, and Jiaya Jia. Lisa: Reasoning segmentation via large language model. arXiv preprint arXiv:2308.0000 2023.
[19] Sicong Leng, Hang Zhang, Guanzheng Chen, Xin Li, Shijian Lu, Chunyan Miao, and Lidong Bing. Mitigating object hallucinations in large vision-language models through visual contrastive decoding. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 13872-13882, 2024.
[20] Kenneth Li, Oam Patel, Fernanda Viégas, Hanspeter Pfister, and Martin Wattenberg. Inference- time intervention: Eliciting truthful answers from a language model. Advances in Neural Information Processing Systems, 36, 2024.
[21] Shuang Li, Tong Xiao, Hongsheng Li, Wei Yang, and Xiaogang Wang, Identity-aware textual- visual matching with latent co-attention. In ICCV, 2017.
[22] Xiang Lisa Li, Ari Holtzman, Daniel Fried, Percy Liang, Jason Eisner, Tatsunori Hashimoto, Luke Zettlemoyer, and Mike Lewis. Contrastive decoding: Open-ended text generation as optimization. arXiv preprint arXiv:22
2022.
[23] Yifan Li, Yifan Du, Kun Zhou, Jinpeng Wang. Wayne Xin Zhao, and Ji- Ji-Rong Wen. Evaluating object hallucination in large vision- laiion-l models. arXiv preprint arXiv:2305. 2023.
[24] Yifan Li, Yifan Du, Kun Zhou, Jinpeng Wang, Wayne Xin Zhao, and Ji-Rong Wen, Evaluating object hallucination in large vision-language models. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 292-305, 2023.
[25] Tsung-Y Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick. Microsoft coco: Common objects in context. In Computer Vision-ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part V 13, pages 740-755. Springer, 2014.
[26] Fuxiao Liu, Kevin Lin, Linjie Li, Jianfeng Wang, Yaser Yacoob, and Lijuan Wang. Mitigat- ing hallucination in large multi-modal models via robust instruction tuning. In The Twelfth International Conference on Learning Representations, 2023.
[27] Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning. arXiv preprint arXiv:2304. 08485, 2023.
[28) Jiazhen Liu, Yuhan Fu, Ruobing Xie, Runquan Xie, Xingwu Sun, Fengzong Lian, Zhanhui Kang, and Xirong Li. Phd: A prompted visual hallucination evaluation dataset. arXiv preprint 2403v:2403.11116 2024.
[29] Gen Luo. Yiyi Zhou, Xiaoshuai Sun, Liujuan Cao, Chenglin Wu, Cheng Deng, and Rongrong Ji. tast collaborative network for joint referring expression comprehension and segmentation. In CVPR, 2020.
[30] Valentino Maiorca, Luca Moschella, Antonio Norelli, Marco Fumero, Francesco Locatello, and Emanuele Rodolà. Latent space translation via semantic alignment. Advances in Neural Information Processing Systems, 36, 2024.
[31] Junhua Mao, Jonathan Huang, Alexander Toshev, Oana Camburu, Alan Yuille, and Kevin Murphy. Generation and comprehension of unambiguous object descriptions. In CVPR, 2016.
[32] Luca Moschella, Valentino Maiorca, Marco Fumero, Antonio Norelli, Francesco Locatello, and Emanuele Rodolà. Relative representations enable zero-shot latent space communication. arXiv preprint arXiv: 15430, 2022.
12

---

[33] Varun K. Nagaraja, Vlad I. Morariu, and Larry S. Davis. Modeling context between objects for referring expression understanding, In ECCV, 2016.
[34] Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. Training language models to follow instructions with human feedback. Advances in neural information processing systems, 35:27730-27744, 2022.
[35] Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International conference on machine learning. pages 8748 8763. PMLR, 2021.
[36] Pranav Rajpurkar, Robin Jia, and Percy Liang. Know what you don't know: Unanswerable questions for squad. arXiv preprint arXiv: 03822, 2018.
[37] Hanoona Rasheed, Muhammad Maaz, Sahal Shaji, Abdelrahman Shaker, Salman Khan, Hisham Cholakkal, Rao M Anwer, Erix Xing, Ming-Hsuan Yang, and Fahad S Khan. Glamm: Pixel grounding large multimodal model. arXiv preprint arXiv:2311.055: 1.03356, 2023.
[38] Anna Rohrbach, Lisa Anne Hendricks, Kaylee Burns, Trevor Darrell, and Kate Saenko. Object hallucination in image captioning. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 4035-4045, 2018.
[39] Jing Shi, Ning Xu, Yihang Xu, Trung Bui, Franck Dernoncourt, and Chenliang Xu. Learning by planning: Language-guidee global image editing. In CVPR, 2021.
[40] Nisan Stiennon, Long Ouyang, Jeffrey Wu, Daniel Ziegler, Ryan Lowe, Chelsea Voss. Alec Radford, Dario Amodei, and Paul F Christiano. Learning to summarize with human feedback. Advances in Neural Information Processing Systems, 33:3008-3000 10000, 2020.
[41] Stefanie Tellex, Nakul Gopalan, Hadas Kress-Gazit, and Cynthia Matuszek. Robots that use language. In Annual Review of Control, Robotics, and Autonomous Systems, 2020.
[42] Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timo- thée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13977 2023.
[43] George Tsatsaronis, Georgios Balikas, Prodromos Malakasiotis. Ioannis Partalas, Matthias Zschunke, Michael R Alvers, Dirk Weissenborn, Anastasia Krithara, Sergios Petridis, Dimitris Polychronopoulos, et al. An overview of the bioasq large-scale biomedical semantic indexing and question answering competition. BMC bioinformatics, 16:1-28, 2015.
[44] Leonid Nisonovich Vaserstein. Markov processes over denumerable products of spaces, de- scribing large systems of automata. Problemy Peredachi Informatsii, 5(3):64-72, 1969.
[45] Cédric Villani. Optimal transport: Old and new. 2008
[46] Bin Wang, Fan Wu, Xiao Han, Jiahui Peng, Huaping Zhong, Pan Zhang, Xiaoyi Dong, Weijia Li, Wei Li, Jiaqi Wang, et al. Vige: Visual instruction generation and correction. arXiv preprint arXiv:2308. 12714, 2023.
[47] Miao Xiong. Zhiyuan Hu, Xinyang Lu, Yifei Li, Jie Fu, Junxian He, and Bryan Hooi. Can Ilms express their uncertainty? an empirical evaluation of confidence elicitation in llms. arXiv preprint arXiv:2306. 13063, 2023.
[48] Zhao Yang, Jiaqi Wang, Yansong Tang, Kai Chen, Hengshuang Zhao, and Philip HS Torr. Lavt: Language-aware vision transformer for referring image segmentation. arXiv, 2021.
[49] Shukang Yin, Chaoyou Fu, Sirui Zhao, Ke Li, Xing Sun, Tong Xu, and Enhong Chen. A survey on multimodal large language models. arXiv preprint arXiv:2306.1335 2023.
[50] Shukang Yin, Chaoyou Fu, Sirui Zhao, Tong Xu, Hao Wang, Dianbo Sui, Yunhang Shen, Ke Li, Xing Sun, and Enhong Chen. Woodpecker: Hallucination correction for multimodal large language models. arXiv preprint arXiv:2310.1600 2023.
13

---

[51] Zhangyue Yin, Qiushi Sun. Qipeng Guo, Jiawen Wu, Xipeng Qiu, and Xuanjing Huang. Do large language models know what they don't know? arXiv preprint arXiv:2305.18155 2023.
[52] Haoxuan You, Haotian Zhang, Zhe Gan, Xianzhi Du, Bowen Zhang, Zirui Wang, Liangliang Cao, Shih-Fu Chang, and Yinfei Yang. Ferret: Refer and ground anything anywhere at any granularity. arXiv preprint arXiv:2310.07704, 2023.
[53] Licheng Yu, Patrick Poirson, Shan Yang, Alexander C Berg, and Tamara L Berg. Modeling context in referring expressions. In ECCV, 2016.
[54] Licheng Yu, Hao Tan, Mohit Bansal, and Tamara Berg. A joint speaker-liseene--reinoorrr model for referring expressions. In CVPR, 2017.
[55] Yuqian Yuan, Wentong Li, Jian Liu, Dongqi Tang, Xinjie Luo, Chi Qin, Lei Zhang, and Jianke Zhu. Osprey: Pixel understanding with visual instruction tuning. arXiv preprint arXiv:2312. 10032, 2023.
[56] Pan Zhang, Xiaoyi Dong Bin Wang, Yuhang Cao, Chao Xu, Linke Ouyang. Zhiyuan Zhao, Shuangrui Ding. Songyang Zhang. Haodong Duan, Hang Yan, et al. Internlm vision-language large model for advanced text-image comprchension and composition. arXiv preprint 2rXiv:2339..5112 2023.
Intemnl-xxcoposeer xcomposer: A
[57] Shilong Zhang, Peize Sun, Shoufa Chen, Min Xiao, Wenqi Shao, Wenwei Zhang, Kai Chen, and Ping Luo. Gpt4roi: Instruction tuning large language model region-of-interest. arXiv preprint arXiv:2307.03600, 2023.
[58] Yi-Fan Zhang, Weichen Yu, Qingsong Wen, Xue Wang, Zhang Zhang, Liang Wang, Rong Jin, and Tieniu Tan. Debiasing large visual language models. arXiv preprint arXiv:2403.05260 2024.
[59] Yue Zhang, Yafu Li, Leyang Cui, Deng Cai, Lemao Liu, Tingchen Fu, Xinting Huang, Enbo Zhao, Zhang, Yulong Chen, et al. Siren's 's song the ocean: survey on hallucination in large language models. arXiv preprint arXhv:3309.012219 2023.
[60] Yiyang Zhou, Chenhang Cui, Jaehong Yoon, Linjun Zhang, Zhun Deng, Chelsea Finn, Mohit Bansal, and Huaxiu Yao. Analyzing and mitigating object hallucination in large vision-language models, arXiv preprint arXiv:2310.00754, 2023.
[61] Deyao Zhu, Jun Chen, Xiaoqian Shen, Xiang Li, and Mohamed Elhoseiny. Minigpt-4: En- hancing laaguue understanding with advanced large language models. arXiv preprint arXiv:2304. 10592, 2023.
14

---

Appendix
In this appendix, we present more ablation studies on our proposed decoding strategy, including the impact of different MLLM-based RES models and the influence of subset size in probing-basee importance estimation. We also elaborate on the calculation of adopted hallucination metrics. At last, we provide visualization that uncovers the multi-moda alignment process in the intermediate layers.
A More Ablation Studies
In this section, we provide more ablation studies, including the impact of the different assisting RES models in cycle-consistencybaseee quality ranking, and the subset size in probing-based estimation. Different scoring RES model. In T we ablate the impact of different RES models utilized in the proposed wycee-oosistenny-bbae
quality ranking (CCR). We adopt another MLLM-bul based RES model LISA (18) to score the sentence quality of intermediate layers. From the table, observe that the choice the RES model affects the CCR. The RES task itself involves transforming language query into pixel-level visual representation of an object. Given that GLaMM (37 performs slightly better than LISA [8] in RES, we can therefore infer that the more robust this transformation is completed, the better the performance of the CCR in quality ranking.
Impact of probing subset quality and size. We also conducted ablation experiments on different subset sizes and the randomness of sampling, with the results listed in Table In the first row, we report the average result of 10 different random seeds 1/8 subset sampling, which is also the result shown in Table [ of the main paper. The second row reports the best results for the 1/8 subset. The results indicate that the randomness the sampled subsets affects the performance our proposed decoding strategy in reducing hallucinations. In other words, the quality of the subset has a certain impact on the decoding outcome.
However, assessing the quality of subsets during probing is also an intractable problem. One possible solution to use an unsupervised clustering method (e.g.. g.. K-means) first cluster the multi-modal features (extracted by CLIP (35 /embedding layers of MLLM (55) of the entire dataset, and divide the subsets based on different centroids, then calculate and store the inter-layer weights in an "importance weight bank." During inference, we can compute the distance between the new query feature and these centroids, selecting the set weights from the closest centroid for inter-layer combination during decoding. This strategy needs careful designing, and we consider it a future extension. Besides quality, by comparing the second and third rows, we also find that compared to the 1/8 subset, the 1/16 subset shows less stable de- de-hallucination and generation performance.
B Hallucination Metrics
The hallucination of objects MILM 14 generated by the model do match the appearance of the object (attributes, relation etc.) in the original image. We provide an example in Figure that demonstrates the hallucinations produced by an MLLM-based REG when there demand increase the granularity of generation. In study, we utilized two approaches to quantify the severity of the hallucination:
23 38
refers to the situations where the descriptions
The first approach is based widely adopted metric CHAIR [14 (38, which directly counts the number of hallucinatory descriptions generated by the model. relies on a reference expert table, providing the scope of the explicit object, and quantifies the object hallucination by calculating the ratio of "the objects mentioned but not in the expert table" to "all objects mentioned in a
Table 4: The impact of different RES models on the performance of RefCOCOg dataset. "full-R" denotes the result of CCR syycle-cosiiteeny-.. based ranking) on full dataset.
RES Model
METEORT
CHAIR |
CHAIR, +
Recall
Len
nCHAIR
nCHAIR, J
LISA (1/8)
171.0
43.1
31.32
0.809
23.1
1.865
1,356
LISA (full-R)
172.0
42,60
31.40
0.811
22.9
1.860
1.371
GlaMM (1/8)
172.0
42.25
30.95
0.821
22.9
1.840
1.348
GlaMM (ull--)
173.0
42.40
31.20
0.823
23.16
1.830
1.347
15

---

Table 5: Different sizes of subsets used in Probing- based estimation. The range of candidate layers is [0, 7]. "full-D" denotes calculate the layer importance weights the full dataset, and then integrate these weights into decoding. "full-R" denotes eycle- cew---ooonnssssssaaaeeeee based ranking on the full dataset. "top" and "avg" denotes the best/average results we tested on different sampled 1/8 subset.
Size
METEOR
CHAIRs
CHAIR; J
Recall
Len
nCHAIRs / nCHAIR, J
1/8 (avg)
172.0
42.25
30.95
0.821
22.9
1.840
1.348
1/8 (top)
172.0
41.90
30.90
0.819
22.9
1.829
1.349
1/16
171.0
42.80
31.72
0.812
22.5
1.902
1.409
full-D
172.0
41.60
30.70
0.818
22.8
1,824
1.346
full-R
173.0
42.40
31.20
0.823
23.16
1.830
1.347
Prompt type: Neutral & Sentiment
Prompt type: Misleuding Sentiment The people in the image are undoubtedly feeling nervous. Based the image, the people feeling nervous in the situstion?
the person feeling anxious
No, he seems happy,
No, they not nervous,
Prompt type: Neutral & Attribute
Prompt type: Misleuding & Attribute
bowl has smooth, lightweight
the table made of wood?
texture like a disposable container. Based image,
metal?
No, there is glass table.
Yes, It's stainless steel howl,
Prompt type: Neutral Position
Prompt type: Misleading & Position
In the image, is motorcycle situated front of the truck?
The object to the right of the TV is used carrying items. Based the image, the toilet to the right of the tv?
No, na motorcycle.
Yes, the toilet the right the TV.
Figure 6: Examples from PHD benchmark. Each prompt formed by different tasks (e.g. senti- ment/atriiute/positionn query modes (e.g. neutral/misleaddnng)
description". CHAIR divides the hallucination into two levels. The first level is sentence level, namely 5HAIRs, which the ratio of hallucinatory descriptions to all descriptions, and the second level is the deeper object level, namely CHAIR,. That is, the average ratio of the number of objects caused by ballucinations and all objects in each description. We refer to the detailed calculation methods from the original paper (38) and summarize them as the following formulas:
(hallucinated objects}|
CHAIR,
(all objects mentioned]]
||
(sentences with hallucinated object)
CHAIRs
|all sentences})
In this case, the CHAIR metric will get high scores as long as the model does not generate much but only the absolutely correct central object, but this goes against our goal of looking for high-granular information. Taking the second sample in Figure 4 as example, the ground-truth description "a sandwich", and the Osprey-7b model generates "A sandwich roll", which is completely right but short, while our generation is "A piece of bread visible on the right side of a plate. This bread appears to be a bun and is positioned in front of some tomatoes." Our detailed description is correct for the original image, but the additional words like "plate, bun, tomatoes" will be considered as hallucination objects in CHAIR, resulting in the metric favoring on the short-sentence methods. Therefore, to neutralize this preference, we add a variation the CHAIR metric by dividing it by the average number of tokens per description, which results in nCHAIR and nCHAIR, accordingly.
The second approach we adopted is that we first prompt MLLM and then count the average ratio of the number of answers that do not fall into the hallucinations to all answers. We utilize the PHD benchmark (28) to achieve this. This benchmark sets ten different types of questions, which are composed of five different tasks, with each task featuring two modes of questioning. The five tasks encompass: Object Recognition, which identifies the nature of objects; Attribute Recognition, which details the attributes of these objects; Sentiment Understanding, which interprets the emotional connotations associated with the objects; Positional Reasoning, which locates the objects space; and Counting, which quantifies the number of objects. The two modes of questioning include Neutral
16

---

mode and Misleading mode. The prompts of the former only include the original question, while the prompts the latter are accompanied with misleading descriptions. Since these questions are all interrogative sentences in PHD, it can be directly concluded whether the description is hallucinatory or not just from calculating the accurate "yes" "no" answers generated by the model. The prompt and response examples are listed in Figure
Birtaayy
2005 8 28
dog sitting on side the table. wearing black collar green tag. dog is looking at cake on table.
Figure 7: A case of hallucination in MLLM-bas base- REG, which mistakenly includes the attribute of the other dog the target.
C Visualization of intermediate multi-modal alignment
In this subsection, delve deeper into showcasing the transition of mudti-modal alignment across different layers of a well-trained MLLM, as well as the potential impact this transition process on the region-level understanding capabilities of intermediate layers.
Similar to Section 3.2 we considered 2000 triplets M, Y) based on a pair of RES and REG models. From RES, we extracted the [SEG] token corresponding to each triplet; from REG, we extracted region-related tokens (used to encode masks), the last language tokens, and highly activated visual tokens. To filter highly activated visual tokens,
calculate the activation norms of the CLIP output. After removing outlier tokens 4 [0 the top 20 most activated visual tokens are selected" We display the Wasserstein distances between these tokens in the middle layers in Figure & where we can observe the following phenomena: (a) The degree of multimodal alignment varies across different layers. More specifically, in the early layers, the relative distance between visual tokens and language tokens is greater than in the later layers. (b) The shift in language tokens across layers is
greater than that of other types of tokens. (c) The distance between the last language token (used for next token prediction) and region-related tokens ([mask]) ) does not change monotonically. Our observations suggest that the multi-modal alignment of intermediate layers of a well-trained MILLM undergoes a transitional phase, where potentially provides better region understanding compared to the final layer.
D Limitations
Our proposed training-free free decoding strategy has the following limitations. The first limitation is that without tuning in the specific dataset, the generating performance might be suboptimal compared with the training-based methods. Secondly, the RES model, it has performance requirements for the RES model and also results in additional computational load. Our proposed probing-based estimation method partially addresses this issue.
method directly inferences and scores based on
We followed Bondarenko et al. 4 that consider outliers the norm that larger than 6 deviations from the mean of corresponding activation tensor.
17

---

Figure 8: The intermediate alignments between risual-linguistcc and region-related tokens of well-trained MLLM (Osprey-7b (55)). "seg" means representation of [SEG] token from RES model. "mask" and "pos" denote encoded region prompts REG model (Osprey-7b). We also consider the and the third from last language tokens, denoted as "last" and "last3". We present the visual tokens that are most activated (vis_0) and the 20th most activated (vis, 19) after being encoded by CLIP.
18

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification:
The writing is strictly based on the actual contributions.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results be expected to generalize other settings.
is fine include aspirational goals as motivation as long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification:
We provide some discussions on the limitations and future directions.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification. asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low images are taken in low lighting. Or speech-to-text system might be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach address problems of privacy and faimess.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory Assumptions and Proofs
Question:
For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
19

---

Justification:
No theoretical results.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless whether the code and data are provided or not)?
Answer: [Yes]
Justification:
We have provided the details of experiments and will open-source the codes.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make clear how to reproduce that algorithm. (b) the contribution is primarily a new model architecture, the paper should describe the architccture clearly and fully.
(c) If the contribution is new model (e.g., a large language model), then there should either way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reprodueibility. the case of closed-source models, may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
20

---

Answer:
[Yes]
Justification:
reproducible, they We use the public datasets and will source the codes.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips. cc/
public/ Boolicc/guiees/Cdeeebbbb.ssess
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https for more details.
/ //nip. httpcc/cpplllgggiuueee getggdddde//eeubbmmiiimm
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: We provide the description of experimental setting and all details will be available in the codes.
Guidelines:
The answer NA means that the paper does include experiments.
The experimental setting should be presented the core the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, supplemental material.
7. Experiment Statistical Significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
Justification: Since our framework is training-free and only make use of the testing data for layer importance estimation, when the full dataset is used the results should be deterministic and when only subset is used, the results are also provided for comparison.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results accompanied by emror bars, confi- dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
21

---

The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative emrr rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
Justification:
We provide the version of GPUs for experiments.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code Of Ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics
Answer: [Yes]
Justification:
The research related contents conform the code of ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g.. if there is a special consid- eration due to laws or regulations their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer:
[Yes]
Justification: discuss potential usage and risk of our research outcomes.
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
22

---

The conference expects that many papers will be foundational research and not tied particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms monitor how system learns from feedback over time, improving the efficiency and accessibility of MI..
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: Not Applicable.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Lieenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: We credit the original codes used in research.
Guidelines:
The answer NA means that the paper does not existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (c.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
23

---

If this information is not available online, the authors are encouraged to reach out to the asset's 's creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification: No new assets.
Guidelines:
The answer NA means that the paper does not release new assets,
Researchers should communicate the details of the dataset/code/moodl as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: No such experiments.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According the NeurIPS Code of Ethies, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [No]
Justification: Not applicable.
Guidelines:
The answer NA means that the paper does involve crowdsourcing research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
24


Reviews:
[{"review_id": "tjqDwDpdKi", "reviewer": "Reviewer_adaN", "summary": "The paper addresses the Referring Expression Generation (REG) task using Multi-modal Large Language Models (MLLMs), identifying the key challenge as the trade-off between generating detailed descriptions and accurately targeting the referring objects, which often leads to hallucinations—the inclusion of incorrect or spurious information in the generated text. To address this issue, the authors propose a training-free \"unleash-then-eliminate\" method that leverages the intermediate layers of MLLMs and employs a cycle-consistency-based decoding strategy to mitigate hallucinations. The proposed approach is validated through extensive experiments on the RefCOCOg and PHD benchmarks, demonstrating superior performance compared to existing techniques.", "questions": "N/A", "limitations": "refer to weaknesses", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The paper is in well-written, which makes it easy to understand.\n\n2. The proposed \"unleash - then - eliminate\" method is training-free, which can avoid the need for additional data and training, reducing the complexity and cost of the model.\n\n3. I like the idea of utilizing the latent information in the intermediate layers of the current region-level MLLMs, which is often overlooked but contains more descriptive regional information.\n\n4. The cycle-consistency-based decoding method helps to alleviate the production of hallucinations in the generated sentences, improving the accuracy and reliability of the model's output. The hybrid layer importance measurement strategy not only increases the decoding speed but also maintains the ability to mitigate hallucinations, achieving a good balance between efficiency and performance.\n\n5. The method shows superior performance compared to existing methods on both semantic and hallucination - related metrics in the experiments, demonstrating its effectiveness.", "weaknesses": "1. Without tuning in the specific dataset, the generating performance of the method might be suboptimal compared to the training - based methods.\n\n2. The methods used in the paper, such as cycle ranking, may introduce additional computational load, which could affect the per-sample decoding speed. Although the proposed strategy helps to alleviate this issue, it may not completely solve it.\n\n3. The method assumes that the RES model can accurately estimate the region-aware descriptive performance of the captions generated by the candidate layers. However, the RES model may also have its own limitations and errors, which could affect the final evaluation results.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your positive acknowledgment of our work. We are glad to notice your interest in the latent information of the intermediate layers. We hope our responses below will partially address your concerns.\n\n**1. Suboptimal compared to the training-based methods.** \n\nYes, we proposed an inference-time decoding strategy that utilizes existing knowledge in trained models through representational operations to enhance the authenticity of the output. If the training is allowed, there are two potential areas within our framework where performance could be improved through training:\n1. If more descriptive datasets are available, the selection among intermediate layers could depend not only on the RES model but also on the ground truth sentence.\n2. A linear layer could be designed to learn the mapping from different scores across layers to the importance weights of layers.\nWe still need to emphasize that although training-based methods might yield better results, as you mentioned, the proposed \"unleash-then-eliminate\" method is training-free, which also avoids the need for additional data and training.\n\n**2. Computation load.** \n\nThanks for the feedback. We agree that the computational intensity of the cycle ranking could potentially affect the decoding speed. This is an inherent challenge when integrating more complex algorithms to improve the accuracy of the generation. We would like to highlight that in probing-based estimation, we compress the cycle process into a set of layer importance weights which could directly be merged into the original decoding process, thus enhancing the practicality of our method.\n\n**3. The RES model is imperfect.** \n\nThanks for the precise comments. We acknowledge that our method depends on the performance of the RES model. We hope that more powerful RES models in the future will help alleviate this limitation. In the table below, we have ablated the impact of the existing MLLM-based RES on our method. Given that GLaMM performs slightly better than LISA in RES, we can therefore infer that the more robust the RES model is, the better the performance of cycle-consistency-based in quality ranking. “full-R” denotes cycle-consistency-based ranking on the full dataset.\n\n| RES Model   | METEOR | CHAIR_S | CHAIR_I | Recall | Len   | nCHAIR_S | nCHAIR_I |\n|-------------|--------|---------|---------|--------|-------|----------|----------|\n| LISA (1/8)  | 171.0  | 43.1    | 31.32   | 0.809  | 23.1  | 1.865    | 1.356    |\n| LISA (full-R) | 172.0 | 42.60   | 31.40   | 0.811  | 22.9  | 1.860    | 1.371    |\n| GlaMM (1/8) | 172.0  | **42.25**   | **30.95**   | 0.821  | 22.9  | 1.840    | 1.348    |\n| GlaMM (full-R) | **173.0**| 42.40   | 31.20   | **0.823**  | 23.16 | **1.830**    | **1.347**    |", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "KJpUTdGLDC", "reviewer": "Reviewer_MMWB", "summary": "The paper aims to strike a balance between detailed description and precise captioning when using multimodal large language models (MLLM) in the task of referring expression generation (REG). A key observation is that the output of a Referring Expression Segmentation (RES) model should be consistent with the input of a REG model.\n\nBased on this observation, a training-free method has been proposed, called 'unleash-then-eliminate,'  which adopts an 'elicit-then-eliminate decoding' strategy. Captions are generated using contrastive decoding (Li et al., arXiv:2210.15097) and then fed into a RES model to produce corresponding masks. The Intersection over Union (IoU) between the masks tagged by the RES model and the input of the REG model is leveraged to select the candidate layer along with the generated caption. Additionally, a Probing-based Importance Estimation method is proposed to accelerate the decoding process.\n\nThe generation quality of the model is evaluated using the METEOR score on the RefCOCOg dataset. Additionally, the model's performance in avoiding hallucinations is assessed with the CHAIR and PHD metrics. The model proposed in this paper outperforms both Osprey and Dora not only in terms of generation quality but also in terms of adequacy.", "questions": "1. What is the final architecture of the proposed model with `Probing-based Importance Estimation` ?\n\n2. What are `the number of parameters` of the proposed models (with and without Probing-based Importance Estimation)?\n\n3. What are the metrics corresponding to those types (i.e. Object Recognition...) in table 3? What is the difference between `neutral-question mode` and `misleading mode`?", "limitations": "Yes.\nThe authors claim that since the model has not been tuned on a specific dataset, the generating performance is suboptimal compared to training-based methods.", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 1, "contribution": 1, "strengths": "1. The problem definition and motivation are clear. The paper indicates that a balance between detailed description and accurate targeting is necessary when using MLLM.\n\n2. The model architecture is clear. The proposed model utilizes contrastive decoding to unleash the information in intermediate layers for generating captions. It indirectly evaluates the quality of these layers by assessing the masks generated by a RES model in relation to the captions. Additionally, a Probing-based Importance Estimation method is proposed to expedite the decoding process.", "weaknesses": "1. The writing is disorganized and lacks clarity. For example, in the first sentence \"Referring expression generation (REG) Yu et al. [2016], Mao et al. [2016], Hu et al. [2016], Yu et al. [2017], Luo et al. [2020], Ding et al. [2021], Tanaka et al. [2019] is a task to\", the reference is mixed with words, making it difficult to follow.\n\n2. The proposed approach is technically simple. The architecture seems simply a combination of REG and RES models.\n\n3. The motivation is to achieve a trade-off between the granularity and accuracy of captioning by selecting different intermediate layers. However, the experiments only show that the proposed model outperforms other models, as indicated in Table 1. Furthermore, Table 2 reveals that the first bucket of layers achieves the best scores in both METEOR, which relates to generation quality, and nCHAIR_I, which relates to the avoidance of hallucinations. This finding contrasts with the initial proposal.\n\n4. The answer to `Open access to data and code` is [Yes], but there's only a `placeholder` found in the anonymous repo,  noted as footprint in the first page of paper.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your detailed review and constructive comments. The code is accessible through the link indicated in the abstract. We noticed that there might have been some misunderstandings regarding to our proposed \"Probing-based importance estimation\" method, and we hope this response will better convey our decoding strategy to you. \nWe will address your concerns below.\n\n**1. Reference format.** Thanks for the suggestion. We have changed it to numerical style in the revision. Additionally, we would like to summarize some related works on inference-time decoding strategies for mitigating hallucination in large language models (LLMs), hoping this can better help you position our work. Specifically, our work is in line with CCS [C. Burns et al., ICLR2022], ITI [K. Li et al., NeurIPS2023], and DoLa [Y.-S. Chuang et al., ICLR2024], which are all aiming to utilize the existing knowledge in trained models through representational operations to enhance the authenticity of the output. Compared to these works, we uncover the intermediate layers of multimodal LLMs and, as far as we know, are the first to propose using a combination of intermediate layers for region-level caption generation. There are more details in the code link.\n\n**2. The architecture seems simply a combination of REG and RES models.** Structurally, the cycle-consistency-based ranking method we proposed indeed relies on the combination of REG and RES. RES serves as an auxiliary tool we utilize to filter the intermediate layer information unleashed afterward.\nAdditionally, we would like to highlight some key points beyond the architecture:\n1. We designed an inference time decoding strategy, which can unleash the latent knowledge inside the intermediate layers without additional training.\n2. The key observation is: compared with the output of the last layer, the sentences of intermediate layers can sometimes provide more discriminative descriptions for the referred object, which motivates us to design a cycle between REG and RES to localize the desired information.\n3. Besides the REG and RES cycle process, we also proposed a probing-based estimation method that can obviate the need for RES, directly blending estimated weights into the existing model to enhance its anti-hallucination efficacy.\n\n**3. The trade-off between granularity and accuracy.** Thank you for your detailed comment and we apologize for any confusion caused. We would like to clarify that the trade-off discussed is inherent to the REG task itself. The more extensively a model describes, the higher the likelihood of errors, as demonstrated by the CHAIR-related metrics in Table 1. As descriptions become lengthier, both $\\text{CHAIR}_I$ and $\\text{CHAIR}_S$ metrics deteriorate, indicating an increase in hallucinations during detailed description generation. To alleviate this issue, we unleash information from the intermediate latent knowledge to broaden the space of choice. Then a RES model as a listener is used to evaluate and select proper output. This \"unleash-then-eliminate\" approach helps us get rid of the intrinsic constraint of detailedness and accuracy. Table 1 demonstrates the efficacy of our proposed method. In Table 2, our goal is to investigate the effects of candidate layers from different zones using our method, and to illustrate the properties of various regions within a trained MLLM. We have not explored such a trade-off across different layers.\n\n**4. Open source.** Thank you for your interest in our implementation. The code has been uploaded.\n\n**5. Final architecture of Probing-based estimation, and the number of parameters.** We have noticed that there might be some misunderstandings about the approach we have proposed. We have introduced **a decoding strategy for inference-time**, not a new model that requires training, hence **no new parameters** have been added. \nThe Probing-based estimation method involves the following steps:\n1. First, sample a subset as the probing set.\n2. On this probing set, perform cycle-consistency-based ranking and calculate scores for each sample in each layer.\n3. Calculate an inter-layer importance weight on this subset, which is mainly based on frequency counting of each candidate layer being the optimal layer.\n4. Combine the importance weight prior with the J-S divergence to represent the probability of each layer being selected (the higher the importance, the more likely it is to be chosen).\n5. Sample from this hybrid importance distribution to decide which layer to use for decoding the next token, until the generation finishes.\nThis approach compresses the “cycling” process into a set of importance weights, thus facilitating convenient decoding. We would like to emphasize that the experiments in Table 3 have demonstrated that such a set of weights can be directly applied to MLLM to alleviate hallucinations, without the need for RES models and the cycling process, enhancing the practicality of our method. We also provide a visualization of importance weights in **Figure A** (listed in common response).\n\n**6. The metrics of Table 3.** These refer to the prompt types of PHD benchmark. For better illustration, we provide some prompt and generation examples in **Figure B** (in common response). This benchmark sets ten different types of questions, which are composed of five different tasks, with each task featuring two modes of questioning. We refer you to the 'Datasets and Metrics' section and the newly added Appendix C for details on the different tasks. The two modes of questioning include Neutral mode and Misleading mode. The prompts of the former only include the original question, while the prompts of the latter would accompanied by misleading descriptions. More details are available in the code link.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "93YoA2zSn4", "reviewer": "Reviewer_85oQ", "summary": "This paper presents an approach for improving the accuracy and richness of referring expression generation (REG) by leveraging the descriptive potential of intermediate layers in Multi-modal Large Language Models (MLLMs). The method employs a cycle-consistency-based decoding strategy to reduce hallucinations and improve descriptive quality. The proposed method is evaluated on the RefCOCOg and PHD benchmarks, demonstrating superior performance over existing methods.", "questions": "You use PCA for dimensionality reduction and Wasserstein distances for comparing features. Can you elaborate on why these specific techniques were chosen and how they contribute to the effectiveness of your method compared to other possible choices?\n\nYour method aims to reduce hallucinations in generated descriptions. Can you provide more details on how you quantify hallucinations and the specific improvements observed with your method compared to baselines? Are there any cases where your method still struggles with hallucinations?\n\nIs there any Transformer-based region encoder?", "limitations": "The authors have not adequately addressed the limitations and potential negative societal impacts of their work.", "rating": 5, "confidence": 3, "soundness": 2, "presentation": 1, "contribution": 2, "strengths": "The \"unleash-then-eliminate\" strategy and the use of intermediate layers for generating more detailed descriptions is innovative to me.", "weaknesses": "The proposed method introduces additional complexity, particularly in the decoding process. While effective, the cycle-consistency-based approach may increase computational overhead, which could limit its applicability in real-time or resource-constrained environments.\n\nThe related work section lacks depth in its analysis and could benefit from a more thorough review of recent advancements, particularly in hallucination mitigation techniques.\n\nSome terms and notations, such as Q, H and W in Section 3.1 , are not clearly defined in the context of the paper, which can lead to confusion. Providing explicit definitions and clarifications would improve readability.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for your feedback. We have made the code open source at the link attached to the abstract. We will address your concerns below.\n\n**1. Increase computational overhead.** Thanks for the comments.\nIn the cycle-consistency-based quality ranking, the RES model is incorporated as an auxiliary, leading to increased computational overhead. However, we need to emphasize that our proposed probing-based estimation method simplifies the cycle between REG and RES into a set of importance weights (refer to Figure A in common response). It removes the need for RES during decoding, and mitigates the MLLM hallucinations through combinations of intermediate layers. This result has been demonstrated in our experiments, on the Table 3 PHD benchmark.\n\n**2. More related work.** \nThank you for your suggestion. Based on your advice, we would like to summarize some related works on inference-time decoding strategies for mitigating hallucination in large language models (LLMs). Specifically, our work represents an inference-time decoding strategy to mitigate hallucinations, which utilizes the latent knowledge within the intermediate layers without additional training. There are more details in the code link. We hope this could help you better understand the positioning of our work. \n\n**3. Undefined terms.** Thanks for your feedback. HW denotes the flattened encoded visual token numbers. Q denotes the number of tokens that the model used to encode the region prompt.\n\n**4. The choices used in visualization.**\nThanks for the question. We first observed through the early-exit method that outputs from different intermediate layers tend to vary, and some may even contain more descriptive content than the final layer. Subsequently, we attempted to quantify how much potential different intermediate layers have for better region understanding. PCA and Wasserstein distances are two common analytical methods used. PCA reduces the dimensionality of high-dimensional latent features while preserving the principal components, which not only speeds up the visualization process but also allows features from different dimensions to be reduced to the same dimension for distance calculation. We use Wasserstein distances as a measure of distance because it evaluates the distance between two distributions from the perspective of transportation cost. Compared to other methods used to measure distances between distributions, it requires fewer samples [M. Arjovsky et al., ICML2017].\n\n**5. How to quantify hallucinations.** \nThanks for the question. According to your suggestion, we would like to provide a detailed elucidation of the hallucination metric calculation and summarize it as follows:\nIn our study, we utilized two approaches to quantify the severity of the hallucination:\n1. The first approach is based on a widely adopted metric CHAIR, which directly counts the number of hallucinatory descriptions generated by the model. We use it for the sentences of REG task. It relies on a reference expert table, providing the scope of the explicit object, and quantifies the object hallucination by calculating the ratio of \"the objects mentioned but not in the expert table\" to \"all objects mentioned in a description\". The result is reported in Table 1 & 2.\n2. The second approach we adopted is prompting MLLM and counting the average ratio of the number of answers that do not fall into the hallucinations to all answers. We utilize the PHD benchmark (a challenging extension of POPE) to achieve this. Similar to POPE, since prompted questions are all interrogative sentences in this benchmark, it can be directly concluded whether the description is hallucinatory or not just from calculating the accurate \"yes\" or \"no\" answers generated by the model. The result is reported in Table 3. For more details, please refer to the code link.\n\n**6. The specific improvements and failure case.**\n1. **specific improvements:** \nBy extracting information from the intermediate layers of MLLMs, our method generates sentences that are more descriptive while reducing hallucinations. Specifically, Table 1 shows an increase in the METEOR metric, indicating higher quality sentence generation, and an improvement in the CHAIR series metrics, showing a reduction in hallucinations. Table 3's improvements on the PHD benchmark further demonstrate that our decoding strategy effectively mitigates hallucination. \n\n2. **failure case:** \nThank you for the question, and we provide a failure case as Figure C-(6) in the common response. In this case, the referred object is the black dog on the left. It can be seen that during the generation process, the model initially describes the referred object in detail, but eventually a hallucination occurs, mistaking the state of another dog (looking at the cake) for the state of the referred dog. This also illustrates the unique challenges of region-level understanding.\n\n**7. Is there any Transformer-based region encoder?** Thank you for the question. In the context of our paper, the region encoder is used to encode region prompts for the MLLM. To our knowledge, MLLMs that perform REG have not yet employed transformer blocks for encoding region prompts. Shikra [K. Chen et al., arXiv:2306.15195] converts regions into coordinates, represented using natural language numbers. Ferret [H. You et al., ICLR2024] uses CNN blocks and point sampling for encoding. GPT4ROI [S. Zhang et al., arXiv:2307.03601] and GLAMM [H. Rasheed et al., CVPR2024] employ ROI pooling methods, and Osprey [Y. Yuan et al., CVPR2024] utilize CNN blocks for multi-scale mask encoding. We speculate that such designs may aim to reduce computational complexity, as transformer modules require a larger amount of data to fit. Given the vast knowledge underlying pre-trained visual encoders (CLIP) and the MLLM base (LLAVA), the region prompts need to align with the existing knowledge. A lightweight region encoder could reduce the optimization difficulty.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "A1H6Qm9bW7", "reviewer": "Reviewer_ikvW", "summary": "This paper explores the Multi-modal Large Language Model (MLLM) based Referring Expression Generation (REG) task, which aims to generate unambiguous text descriptions for specific objects or regions in images. MLLM-based REG models tend to suffer from hallucination issues, and there is a trade-off between detailed descriptions and accurate targeting. To address this, a training-free method called \"unleash-then-eliminate\" is proposed, which elicits latent information in intermediate layers and uses cycle-consistency decoding to reduce hallucinations. Extensive experiments on the RefCOCOg and PHD benchmarks show that this method outperforms existing approaches in both semantic and hallucination-related metrics.", "questions": "1. What is the effect of \\alpha defined in Line 172\n2. \\mathcal{V} is not defined in Line 171.\n3.  The candidate sentence set A use n to represent  the size of candidate layers, while use N-1 in Line 169. So are n and N-1 the same, or do they represent different things?", "limitations": "The authors adequately addressed the limitations.", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The observation that the intermediate layers of the current region-level MLLMs sometimes hold more descriptive regional information than the final layer is interesting.\n2. The writing throughout the paper is clear and easy to follow. The authors have done a good job in presenting their ideas and methodologies in a manner that is both logical and comprehensible.\n3. The experimental results are compelling and demonstrate that the proposed method outperforms existing methods on the newly introduced evaluation metric.", "weaknesses": "1. Since the proposed method requires multiple layers for inference to be described, both REG and RES models are required, resulting in a very low efficiency of the method and requiring a lot of additional calculations and memory.\n2.  The ablation study is not sufficient. For example, subset used to select the optimal layer in Layer Importance Measurement. What is the Impact of subset size and quality on performance? What is the performance of using  cycle-consistency-based candidate ranking process for whole dataset, i.e., not using Layer Importance Measurement.\n3.  What is effect of using different RES models for the proposed method.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your attention to our work and your positive acknowledgment of our idea. We have open-sourced the code. We will address your concerns below.\n\n**1. low efficiency.** We would like to hightlight more about probing-based estimation that is designed to allivate this issue. While the cycle-consistency-based quality ranking incorporates the RES model, adding to the computational load, our probing-based estimation method simplifies this process. By transforming the REG-RES cycle into a straightforward set of importance weights (please refer to Figure A in our common response), we eliminate the necessity for RES in decoding. The proposed method and its resulting importance prior weights can effectively reduces MLLM hallucinations by using combinations of intermediate layers. We have validated this approach in our experiments, as shown in Table 3 of the PHD benchmark. We hope this could partially address your concern.\n\n**2. Ablation study.** Thanks for your precise and valuable feedback. We have reported some subset-related ablation experiments in the table below. The range of candidate layers is [0, 7]. As for the “full-D”, we calculate the layer importance weights on the full dataset, and then integrate these weights into decoding. “full-R” denotes cycle-consistency-based ranking on the full dataset. “top” and “avg” denotes the best/average results we tested on different sampled 1/8 subset.\n\n| Size   | METEOR | CHAIR_S | CHAIR_I | Recall | Len  | nCHAIR_S | nCHAIR_I |\n|--------|--------|---------|---------|--------|------|----------|----------|\n| 1/8 (avg) | 172.0  | 42.25   | 30.95   | 0.821  | 22.9 | 1.840    | 1.348    |\n| 1/8 (top) | 172.0  | 41.90   | 30.90   | 0.819  | 22.9 | 1.829    | 1.349    |\n| 1/16     | 171.0  | 42.80   | 31.72   | 0.812  | 22.5 | 1.902    | 1.409    |\n| full-D   | 172.0  | **41.60**   | **30.70**   | 0.818  | 22.8 | **1.824**    | **1.346**    |\n| full-R   | **173.0**  | 42.40   | 31.20   | **0.823**  | 23.16| 1.830    | 1.347    |\n\nIn the first row, we report the average result of 10 different random seeds for 1/8 subset sampling, which is also the result shown in Table 1 of the main paper. The second row reports the best results for the 1/8 subset. The results indicate that the randomness of the sampled subsets affects the performance of our proposed decoding strategy in reducing hallucinations. In other words, the quality of the subset has a certain impact on the decoding outcome. \n\nHowever, assessing the quality of subsets during probing is also an intractable problem. One possible solution is to use an unsupervised clustering method (e.g. K-means) to first cluster the multimodal features (extracted by CLIP or embedding layers of MLLM) of the entire dataset, and divide the subsets based on different centroids, then calculate and store the inter-layer weights in an \"importance weight bank.\" During inference, we can compute the distance between the new query and these centroids, selecting the set of weights from the closest centroid for inter-layer combination during decoding. This strategy needs careful designing, and we consider it a future extension. Besides quality, by comparing the second and third rows, we also find that compared to the 1/8 subset, the 1/16 subset shows less stable de-hallucination and generation performance.\n\n**3. Different RES model.**\nWe appreciate for the detailed comments. In the following Table , we ablate the impact of different RES models utilized in the proposed cycle-consistency-based quality ranking (CCR). We adopt another MLLM-based RES model LISA to score the sentence quality of intermediate layers. \n\n| RES Model   | METEOR | CHAIR_S | CHAIR_I | Recall | Len   | nCHAIR_S | nCHAIR_I |\n|-------------|--------|---------|---------|--------|-------|----------|----------|\n| LISA (1/8)  | 171.0  | 43.1    | 31.32   | 0.809  | 23.1  | 1.865    | 1.356    |\n| LISA (full-R) | 172.0 | 42.60   | 31.40   | 0.811  | 22.9  | 1.860    | 1.371    |\n| GlaMM (1/8) | 172.0  | **42.25**   | **30.95**   | 0.821  | 22.9  | 1.840    | 1.348    |\n| GlaMM (full-R) | **173.0**| 42.40   | 31.20   | **0.823**  | 23.16 | **1.830**    | **1.347**    |\n\nFrom the table, we can observe that the choice of the RES model affects the CCR. The RES task itself involves transforming a language query into a pixel-level visual representation of an object. Given that GLaMM performs slightly better than LISA in RES, we can therefore infer that the more robust this transformation is completed, the better the performance of the CCR in quality ranking.\n\n**4. Explanation of symbols.**\n Thanks for the question and we apologize for the confusion. $\\alpha$ is a hyperparameter in [0, 1] that truncates the next token distribution of $p(y_t|y_{<t})$. It helps split out the tokens whose score is lower than a proportion of the highest score. Larger $\\alpha$ entails more aggressive truncation, keeping only high-probability tokens, whereas smaller $\\alpha$ allows tokens of lower probabilities to be generated [Li et al., arXiv:2210.15097]. \n\n$\\mathcal{V}$ in Line 171 denotes the vocabulary set. \n\n$n$ and $N-1$ are different. $N$ denotes the number of layers of MLLM model, 32 in our case, and $N-1$ represents the number of remaining layers except the final layer. The lowercase $n$ is dynamic and refers to the number of candidate layers in experiments, which we chose as 8 in our implementation.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
We appreciate the time and effort of all reviewers in reviewing our manuscript. Your insightful feedback has been essential in enhancing our work’s quality. 

We have released our code, and it is available at the link listed in the abstract.

In this common response, we would like to explain the figures newly uploaded:

**1. Figure A:** We display the weights of different layers after probing-based importance estimation. These weights can be seen as a "compression" of the cycle-consistency-based ranking process, directly integrated into the decoding process without the need for a RES model, thus enhancing the practicality of our method. The effectiveness of the important weights is validated in Table 3 of our paper.

**2. Figure B:** We have listed some prompt-response examples from the PHD benchmark to illustrate the quantification of hallucinations. We hope this can help answer the questions of reviewers MMWB and 85oQ.

**3. Figure C (1)-(5):** Visualization of multi-modal alignment in intermediate layers . We delve deeper into showcasing the transition of multi-modal alignment across different layers of a well-trained MLLM, as well as the potential impact of this transition process on the region-level understanding capabilities of intermediate layers. We can observe the following phenomena: 
- The degree of multi-modal alignment varies across different layers. More specifically, in the early layers, the relative distance between visual tokens and language tokens is bigger than what in the later layers.
- The shift in language tokens across layers is greater than that of other types of tokens. 
- The distance between the last language token (used for next token prediction) and region-related tokens does not change monotonically.

   Despite the limitations of the specific distance in measuring token alignment comprehensively, our observations suggest that the multi-modal alignment of intermediate layers of a well-trained MLLM undergoes a transitional phase, with these varied transitional states potentially providing better region understanding compared to the final layer. Reviewer MMWB and 85oQ might have interests in it.

**4. Figure C (6):** We provide a failure case according to the suggestion of reviewer 85oQ.

Meta-review: This paper receives mostly positive rates (1 strong accept, 1 weak accept, 1 borderline accept and 1 borderline reject) thanks to the efficacy of the proposed model and convincing experimental results. Although overall reviews are positive, reviews still have some concerns, which should be addressed in the final version.
Decision: Accept (poster)