Unleashing Region Understanding in Intermediate Layers for MLLM-based Referring Expression Generation

Authors: yaoyuan  liang
zhuojun  cai
jian  xu
guanbo  huang
yiran  wang
xiao  liang
jiahao  liu
ziran  li
jingang  wang
shao-lun  huang
reg  mllm
res  mllm
Venue: NeurIPS_cc_2024_Conference
Published: 2024-05-15 19:34:50

Abstract: The Multi-modal Large Language Model (MLLM) based Referring Expression Generation (REG) task has gained increasing popularity, which aims to generate an unambiguous text description that applies to exactly one object or region in the image by leveraging foundation models. We empirically found that there exists a potential trade-off between the detailedness and the correctness of the descriptions for the referring objects. On the one hand, generating sentences with more details is usually required in order to provide more precise object descriptions. On the other hand, complicated sentences could easily increase the probability of hallucinations. To address this issue, we propose a training-free framework, named ``unleash-then-eliminate'', which first elicits the latent information in the intermediate layers, and then adopts a cycle-consistency-based decoding method to alleviate the production of hallucinations. Furthermore, to reduce the computational load of cycle-consistency-based decoding, we devise a Probing-based Importance Estimation method to statistically estimate the importance weights of intermediate layers within a subset. These importance weights are then incorporated into the decoding process over the entire dataset, intervening in the next token prediction from intermediate layers.
Extensive experiments conducted on the RefCOCOg and PHD benchmarks show that our proposed framework could outperform existing methods on both semantic and hallucination-related metrics. Code will be made available in https://github.com/Glupayy/unleash-eliminate.

Main paper: 
 introduction: 
 Referring expression generation (REG) [11, 29, 31, 53, 54] is a task to generate an unambiguous text description that applies to exactly one appointed object or region in the image. A good expression should be distinguishable enough to ensure that the listener can identify the unique target among various objects within the same image. With the great success achieved by large language models (LLMs), multi-modal large language models (MLLMs) [1, 9, 12, 27, 56, 61] have been introduced to perform this task and become increasingly popular in the research community. Some representative works [6, 37, 52, 55, 57] conduct visual instruction tuning on specialized region-involved multi-modal corpus and successfully empower MLLMs with the ability of region-level understanding.Though certain progress has been made, MLLMs themselves suffer from object hallucination [14, 23, 46, 50, 60] . Research suggests that the mechanism behind MLLMs' hallucinations could be related to the over-reliance on prior knowledge of LLMs rather than the multi-modal context provided by Figure 1 : Illustration of our method. (a) Given an image and a mask appointed to the target object, we first unleash the descriptive regional information of the middle layers and gain various candidate captions. These outputs are sent to a RES model that serves as a "listener" and the "listener" eliminates the inaccurate candidates. (b) Our proposed strategy to diminish the computational load of RES. First estimate the layer prior importance on a probe set with RES, then leverage it for the RES-free next token prediction.the input [14, 19, 50] . Therefore, MLLM-based REG models naturally inherit the aforementioned hallucination issues. Additionally, a further complication arises in the context of region-level understanding, where models are required to generate precise and identifiable descriptions for specific regions. To achieve that, the MLLMs sometimes have to make use of the surroundings for reference while effectively avoiding out-of-region information distorting the description of the target object, e.g. incorrectly attributing characteristics of other objects to the targeted region (Figure 7 in appendix). This requirement exacerbates the issue of attribute-level hallucinations [3, 50] . In this paper, we aim to explore this potential trade-off between the detailed description and accurate targeting of referring objects in MLLM-based REG task. To be specific, providing precise object descriptions necessitates generating sentences with more details, which results in longer sentences. Meanwhile, the text with increased granularity (length) is more likely to contain inaccurate or spurious information, commonly defined as "hallucinations." Table 1 shows an example of quantitative analysis.To address the trade-off between information richness and reliability, we propose a novel approach called "unleash-then-eliminate," as depicted in Figure 1 . We observed that the alignment of regionlevel multi-modal information does not maintain a monotonic progression during inter-layer transitions. Moreover, when multi-modal hidden states are projected into the language space using the language head, the intermediate layers sometimes hold more descriptive region information than the final layer. These observations (Section 3.2) imply that the most suitable layer for each referred region should not be solely confined to the final layer. Accordingly, we adopt contrastive decoding [8, 22] to unleash the object information contained in the intermediate layers. To eliminate unsuitable candidate outputs, we leverage the dual task of REG: Referring expression segmentation (RES) [37, 48] , which aims to segment a target object mask from the entire image given a sentence describing the object. Ideally, for the same object, the region input (formed as a mask) for the REG task and the mask output from the RES task should exhibit cycle consistency. Based on this insight, we propose a cycle-consistency-based decoding method, which enables us to choose among multiple outputs based on their descriptive quality thus reducing hallucinations while maintaining the richness of the output sentences. Furthermore, considering the need to diminish the computational load of RES, we develop a hybrid layer importance measurement strategy to select the best layer for each token during the next word prediction. This strategy leverages both the layer-wise prior importance estimated over probing subset, and the Jensen-Shannon divergence [8] between the logits of each candidate layer and the last layer. With the layer-wise prior importance, the MLLM-based REG model maintains promising performance in reducing hallucinations and enhancing the granularity of the generated sentences, even without the assistance of RES model. Extensive experiments conducted on the RefCOCOg [33] and PHD [28] benchmark demonstrate that our proposed framework surpasses existing methods on both semantic and hallucination-related metrics.Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?Answer: [Yes] Justification: The writing is strictly based on the actual contributions.Guidelines:• The answer NA means that the abstract and introduction do not include the claims made in the paper. • The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers. • The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings. • It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper. 
related works: 
 Region-level understanding in Multimodal Large Language Models. Significant progress has been made in unleashing the region-level understanding ability in MLLMs [6, 18, 37, 52, 55, 57] . To incorporate region-level information into sequence generation of MLLMs, some approaches [6, 52] integrate bounding box coordinates into the language input in the form of natural language prompts. Ferret [52] proposes a spatial-aware visual sampler that enables the arbitrary shapes of visual prompts. The current proposed Osprey [55] unlocked the capability of pixel understanding, alleviating the influence of irrelevant information in the visual prompt inputs. These methods have propelled the REG task into the era of MLLMs. Considering that MLLMs incorporate extensive knowledge from unimodal and multimodal pre-trained corpora, there is a potential for the REG task to leverage this inherent knowledge within the models to generate more specific and detailed expressions without additional training, thus more effectively addressing real-world applications [21, 39, 41] .Decoding strategies to mitigate hallucination in Large Language Models. Large language models are pre-trained on unlabeled corpora to acquire extensive world knowledge and subsequently undergo post-training to learn to follow instructions [34] and align with human preferences [2] . This systematic pre-and post-training pipeline makes them powerful at solving a wide range of NLP tasks [16, 36, 40, 43] . However, some studies indicate that they may fail to accurately assess their own knowledge [51] and often exhibit overconfidence in their responses [47] , which results in hallucinations [59] . To mitigate these issues, recent research [5, 8, 20] proposes inference-time decoding strategies for trained LLMs to find latent knowledge inside the internal activations without additional training. C. Burns et al. [5] introduce a Contrast-Consistent Search (CCS) algorithm to identify a direction of truth in the activation space of LLMs that remains consistent across negations, thereby reducing generated errors. Based on the discovery of CCS, ITI [20] dives deep into attention heads and suggests shifting model activations alongside factuality-related heads during inference to help reduce hallucinations. Besides "finding the direction of truth," DoLa [8] proposes contrastive decoding by comparing the differences in logits between the projections of later and earlier layers to better surface factual knowledge and reduce the generation of incorrect facts. In line with the motivations of [5, 20] , our investigation uncovers that well-trained MLLMs' intermediate layers differ in multi-modal alignment and region-level understanding capabilities. These observations inspire us to devise an inference-time decoding strategy that combines latent knowledge from multiple layers (with prior importance) to alleviate hallucinations.Hallucination in Multi-modal Large Language Models. In the realm of MLLMs, "hallucination" typically refers to "object hallucination," where the models generate plausible outputs containing objects that are either absent from or mismatched with the images [15, 19, 24, 38, 50] , and is commonly categorized into three types: category, attribute and relation hallucinations [3, 49] . Some efforts based on instruction tuning have been made to mitigate this issue in MLLMs. LRV-Instruction [26] introduces a dataset with positive instructions and unique negative prompts with different semantic levels to better align responses with image content. HACL [15] explores the vision-language embedding space, using contrastive learning to separate non-hallucinated from hallucinated texts. Without instruction tuning, Woodpecker [50] offers a training-free pipeline for hallucination correction, using expert models to enrich image context and ensuring each phase is interpretable by a step-by-step correction process. In another line of work, some efforts have sought new decoding strategies to avoid relying on extensive additional data and training: Opera [14] tackles the partial over-trust issue in decoding by applying a penalty to the model logits during beam-search decoding. VCD [19] links object hallucinations to biases and language priors, contrasting outputs from distorted and original visuals to ensure consistent generation. Compared to existing studies, our method initially enriches region-level context from intermediate layers rather than external expert models or knowledge base and provides a multi-layer ensembling solution to mitigate hallucinations.Question: Does the paper discuss the limitations of the work performed by the authors?Answer: [Yes] Justification: We provide some discussions on the limitations and future directions.Guidelines:• The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper. • The authors are encouraged to create a separate "Limitations" section in their paper.• The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be. • The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated. • The authors should reflect on the factors that influence the performance of the approach.For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon. • The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size. • If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness. • While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations. 
preliminary: 
 Leveraging the nuanced language representation capabilities inherent in multi-modal large language models, recent investigations have unlocked the REG [6, 37, 52, 55] and RES [18, 37] capabilities of MLLMs. To maintain clarity and conciseness, we have omitted the discussion on instruction tuning and assume that the described tasks are well-instructed by specific prompts. In this work, we utilize a "mask" as the region prompt.The MLLM-based REG model is typically structured around three principal components: a visual encoder (e.g., CLIP [35] with a linear adapter), a region-encoder, and an LLM base (e.g., LLaMA [42] or Vicuna [7] ). Given an image I, the visual encoder first extracts the visual information, then the linear adapter projects it into I f ∈ R HW ×C , where HW denotes the flattened visual token length and C denotes the hidden state dimension of LLM base. A referred region M (presented as a mask) is encoded as M f ∈ R Q×C by region encoder (e.g., a CNN-based extractor [55] or an ROI pooling layer [37, 57] ), where Q denotes the length of encoded region prompt tokens. Taking the visual feature I f , regional feature M f , and an embedded instruction text X f ∈ R L×C as input, where L represents the length of the instruction in tokens, the model concatenates them and forms a multi-modal input sequence S ∈ R (HW +Q+L)×C .Regarding the core architecture of the LLM base, the multi-modal sequence S is successively processed by the N stacked transformer layers. Eventually, an affine layer ϕ(•) serves as a language model head to predict the probability of the next token y t over the vocabulary set V. The logits for the token y t , given the sequence S and all preceding tokens y <t , are computed as follows:EQUATIONwhere h(N ) tdenotes the hidden state of the last transformer layer. The probability of the next token y t is then given by: p(y t |S, y <t ) = softmax logit N (y t |S, y <t ) , y t ∈ V.(2) Through this process, the model autoregressively generates the output text Y as the region description. For simplicity, we use p(y t |y <t ) to represent p(y t |S, y <t ), and logit N (y t |y <t ) to represent logit N (y t |S, y <t ) in the following.Referring expression segmentation. Recent advances in MLLM-based RES models [18, 37] largely employ a similar MLLM architecture (e.g., LLaVA [27] ) as used in the REG models, with the distinction that its input contains a description targeted at a specific region, while its output is a mask that can cover the described object.A widely adopted strategy [17, 18, 37] incorporates a [SEG] token, enabling the model to identify the [SEG] token in the output sequence as a cue for the presence of a segmentation target. A specialized MLP head ψ processes the output embedding of the [SEG] token h seg ∈ R 1×C , mapping it into the prompt space of the segmentation fundamental model (e.g., SAM [17] ), represented as hseg = ψ(h seg ). The segmentation model then decodes the target mask M s from the query token hseg and provides its confidence score CF .This section focuses on our observations of intermediate layers, attempting to uncover the latent descriptive information. We adopted a region understanding model Osprey-7b [55] with N = 32 layers as REG model, and GLaMM [37] as RES model by default.Each layer has different generation tendency. To reveal the latent information, we adopt the early-exit strategy on the REG model to generate a series of output sequences by applying the language model head (an affine layer) to the hidden states of each layer. As illustrated in Figure 2 , the sequences from the early layers (layers 1 to 10) manifest as nonsensical strings of characters, indicating the suboptimal alignment between the hidden representation of shallow layers and the ultimate vocabulary space of MLLM. Interestingly, among the middle layers (about layers 20 to 25), the model begins to output sentences with semantic meaning and gradually some region-specific expressions appear. Compared with the output of the last layer, these sentences with richer semantic information can sometimes provide more discriminative descriptions for the referred object, e.g. indicating the related position between the target object and others, and/or containing high granularity of attributes. The uniqueness and descriptiveness of expressions given by the middle layers are strongly related to the objective of REG, thus the latent features of intermediate layers have notable potential to enhance the unambiguity in MLLM-based REG. hidden states of the last token of each layer {h i } N i=1 are obtained. Inspired by FID metric [13] and recent investigations on the latent space communication [30, 32] , we first performed PCA dimensionality reduction to project both features into the same dimension, then treat the dimension-reduced [SEG] features as the anchor to calculate the Wasserstein-2 distances [13, 44, 45] between [SEG] and each intermediate hidden states. This allowed us to estimate the relative region understanding ability of each layer for the given multi-modal context of a certain object. We then calculate the frequency with which each layer had the smallest Wasserstein-2 distance to the [SEG] token. As depicted in the right of Figure 2 (orange), except for the final layer, the intermediate layers also have the potential to contain better latent information for a more precise region-related description. (b) Solely within the REG model: we investigated the multi-modal alignment process in region-level context across intermediate layers by calculating the Wasserstein-2 distance between each layer's region-encoded token [mask] and the last language token. Results are reported in Figure 2 (blue). These analyses within a well-trained MLLM show that the tokens of different modalities do not approach monotonically during inter-layer transitions. Hence we should give more chances to the intermediate layers for better region-level understanding. Detailed information can be found in the Appendix C.To enhance the granularity of region-level descriptions without introducing excessive hallucinations, we propose a method that integrates contrastive decoding with cycle consistency-based ranking to screen out appropriate descriptions for the interested regions (Section 3.3.1 and 3.3.2). This approach enables us to specifically leverage the commonly overlooked latent information contained in intermediate layers and ensures the identifiable description through caption quality estimation. Under the concerns of computation efficiency, we further develop a decoding strategy to reduce the operations of cycle ranking through hybrid layer importance measurement (Section 3.3.3). It involves two kinds of layer importance calculation to influence the selection probability of candidate layers during each word prediction step. The harvest prior importance weights can be directly applied to the decoding process of the original MLLM, mitigating the hallucinations. The following subsections provide further details.As depicted in Figure 2 , the manifestation of region-aware information differs across layers. To underscore descriptions related to specific regions-evident in the intermediate layers but faded in the final layer, we adopt a contrastive decoding approach [8, 22] by subtracting the log probabilities of the next token produced by the intermediate layer from those of the final layer. The resulting distribution is defined as the contrastive decoding logits of specific subtractor layers. These logits instead of originally the logits from the final layer are used for generating the subsequent token. Concretely, given a set of candidate layer indices J = {1, . . . , n}, the probability of token y t for layer j ∈ J is: where logit j (y t |y <t ) = ϕ(hEQUATION(j) t ), V head (y t ) = {y i ∈ V : p(y i |y <t ) ≥ α max w∈V p(w|y <t )}, α ∈ [0, 1]is a cutoff hyperparameter that truncates the next token distribution p(y t |y <t ) of the final layer. Following the previous works [8, 22] , we set α = 0.1 in the implementation. After contrastive decoding, each of the intermediate layers in the candidate set suggests a probability for predicting the next token y t . Finally, we harvest a sentence set A = {a 1 , a 2 , ..., a n } whose size equals to the size of candidate layers.Consider the formed triplet (I, M, Y) in Section 3.2, where I serves as the global contextual background of a region, M and Y respectively represent two modalities of the same object. For a pair of ideal REG and RES models, it is anticipated that M and Y can be interconverted losslessly during a cyclic operation of these two models. This implies that, due to the cycle consistency between the two modalities of the same object, feeding the output of the REG model into the RES model should yield a mask consistent with the input to the REG. If the output generated from the candidate layer is overly ambiguous or polluted by hallucinations, the RES model may struggle with accurately locating the target object against the background. Based on these assumptions and observations, we utilize RES model [37] to estimate the region understanding performance of the captions generated by the candidate layers, allowing us to rank these candidates effectively.Figure 3 illustrates the pipeline of cycle-consistency-based quality ranking. The input image I and the referred region (represented as a mask) M are processed by the REG model, which continuously extracts and aligns features across successive layers for the multi-modal context. After the information elicitation of intermediate layers, we harvest a set of sequence A = {a 1 , a 2 , ..., a n }. Each sequence in A, paired with the image I, forms an input pair (I, a j ) to feed into the RES model. The RES model then segments out the corresponding mask M ′ j for each input pair. We evaluate the quality of each layer's sentences by calculating the Intersection over Union (IoU) between M and M ′ j :EQUATIONwhere CF j refers to the output of IoU score head within the segmentation foundation model (e.g. SAM [17] ), which can be interpreted as a confidence score of the generated mask. Subsequently, the candidate sentences from each layer are ranked by the Score j . For each sample, the a j with the highest Score j is selected as the final sentence, and its layer is deemed the best candidate layer. Output: q ∈ R n , n j=1 q j = 1 ▷ Importance weight of each candidate layer 3: Initialize count ← 0 ∈ R n▷ counts of times each layer has the maximum score 4: for j ← 1 to n do 5:count j = m i=1 1 {j = arg max j∈J Score i,j } 6: q j ← countj m7: return qAlthough our cycle-consistency-based candidate ranking process improves the generation quality, it introduces additional computational load from the RES model compared to the original decoding method, significantly affecting the per-sample decoding speed. To alleviate this issue, we propose a simple yet effective strategy called Probing-based Importance Estimation to speed up the decoding process. This strategy involves frequency counting of each candidate layer being the optimal layer based on their performance within a probing subset. With a subset containing m samples, the specifics of this calculation are outlined at Algorithm 1. The estimated weight q is then served as prior knowledge that reflects the candidate layers' importance and is then utilized over the entire dataset in the decoding process, intervening in the next-word prediction.Furthermore, inspired by the success of distance-guided layer selection in LLMs [8] , we also apply this metric as a second guidance. Concretely, we first calculate the distance between the next-token probability of the final layer and each candidate layer at the current decoding step, then the calculated values are normalized across candidate layers as follows,EQUATIONwhere D j = JSD(p||p conj ) denotes the J-S divergence between the next-token probability of the final layer p and the layer p conj . Finally, the hybrid layer importance is obtained by adding up the probing-based prior and sample-wise distance followed by a softmax normalization:EQUATIONwhere d is a vector of normalized divergence values across the candidate layers. We then sample from the probability distribution q to select one layer among candidates to predict the next token at each decoding step, till the generation finishes.Overall, this decoding approach first estimates a prior layer-wise importance weight from a small subset and then applies this distribution to contrastive decoding, effectively improving decoding efficiency while preserving the ability to mitigate hallucinations. In addition, our experiments (Table 3 ) also show that the prior importance q calculated from one (probing) dataset could be directly transferred to another dataset with similar image domain for reducing the hallucinations. This strong transferability further illuminates a new promising application scenario where the historic estimate could serve as a cold-start for inference in new environments when the probing dataset is not available.Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?Answer Justification: We provide the description of experimental setting and all details will be available in the codes. Guidelines:• The answer NA means that the paper does not include experiments.• The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them. • The full details can be provided either with the code, in appendix, or as supplemental material. 
datasets and metrics: 
 RefCOCOg. The RefCOCOg [33] dataset is a classical benchmark for referring expression generation, which contains 85,474 expressions for 54,822 objects in 26,711 images. Expressions in RefCOCOg are annotated on Amazon Mechanical Turk in a non-competitive way and tend to be longer (8.43 words per sentence on average) and more expressive than RefCOCO [53] and Ref-COCO+ [53] . In the MLLM-empowered REG task, the relatively short ground truth cannot fully cover the expression space of the sentences generated by the MLLM at evaluation. Therefore, our experiments focus on the METEOR metric as it is more comprehensive and flexible, allowing for a more nuanced recognition of linguistic variations. Prompted Visual Hallucination Evaluation Benchmark (PHD). This benchmark [28] focuses on the four major types of hallucination faced by MLLMs, namely Object hallucination, Attribute hallucination, Multi-modal conflicting hallucination, and Counter-common-sense hallucination. It evaluates and explores the hallucinations through comprehensive prompt-based tasks, which also helps identify the causes of these hallucinations. While this benchmark does not explicitly evaluate regionlevel hallucinations, its detailed evaluating strategies for object attribute and position are closely related to region-level understanding, allowing it to effectively indicate regional-level hallucinations.Baseline region-level MLLM model and decoding method. Table 1 presents the performance comparison of semantic quality and hallucination evaluation on RefCOCOg dataset, with METEOR and CHAIR metrics. Recent research [58] has revealed that the temperature parameter t has a notable effect on the hallucination of generated sentences. In light of this, we included an analysis of the baseline region-level MLLM model, Osprey-7b [55] , performing at both lower (t = 0.2) and higher (t = 0.9) temperature settings. Meanwhile, we also compared a baseline decoding method, DoLa [8] , which demonstrated that leveraging contrastive decoding in vanilla LLMs can enhance the authenticity of the generated results. In comparison to the Osprey-7b1 , the sentences generated by DoLa [8] demonstrated better performance on METEOR and nCHAIR metrics.Performance of cycle-consistency-based sentence quality ranking. As introduced in Section 3.3.2, to balance the trade-off between information granularity and accuracy, we first unleashed the region description of intermediate layers by contrastive decoding, then filtered out the inaccurate sentences by cycle-consistency-based quality ranking. For each sample in RefCOCOg, we evaluated the best sentence generated by the best candidate layer. The result is reported in the fifth row of Table 1 . We observed that our method not only gains more descriptive sentences but also demonstrates a reduction in the hallucination metric (measured in terms of nCHAIR) compared to Osprey-7b and DoLa. The examples of generation results are listed in Figure 4 .Text generation via hybrid layer importance measurement. In Section 3.3.3, we proposed a method that effectively reduces the computational overhead of the RES scoring model. In our experiment, we divided all 32 intermediate layers (including the embedding layer) into four consecutive groups (detailed in Section 4.3) and calculated the relative importance of the layers within each group Ours: A piece of bread is visible on the right side of a plate. This bread appears to be a bun and is positioned in front of some tomatoes.The right side of the sandwich contains the deli meat, cheese, lettuce, tomato, and mustard.Ours: A large brown wooden dining room table dominates the scene. The table is littered with various objects including a bowl with chips, a bottle, a spoon and a cup.A pizza with spinach and tomatoes is being made on a tray.One man is holding a slice of the pizza while the other adds toppingsThe woman on the left is wearing a plaid shirt and a fringe skirt. She has a large belt buckle and a necklace.A pizza with white cheese on it.that holds a baby and a man at it.in the foreground of the photo, with a baby sitting on it.Ours: There is a pizza on the left, being picked up. The pizza appears to have spinach on it, and is topped with tomatoesThe woman on the left has long dark hair and is wearing a plaid shirt. She's standing next to the motorcycle and appears to be a posed model.The lady in a checked blouse.The visualization comparison of generations between Osprey-7b [55] , DoLa [8] and ours. Osprey-7b's outputs are quite brief and omit visual details of the referred objects. The generations from DoLa are more extensive but are accompanied by hallucinations. In contrast, our method increases descriptive information and curtails the generation of hallucinations.separately. Figure 5 depicts the resulting layer importance weights of four different ranges. The result reveals that layer importance is not uniform across candidate layers and also varies among different groups. We applied the importance weights estimated from 1/8 subset to the decoding process and listed the first group's result in the fourth row of Table 1 . Notably, in comparison to full-set sentence-by-sentence quality ranking, this probing-based decoding method offers comparable performance with improved efficiency, demonstrating the effectiveness of our statistical layer importance estimation.As aforementioned in Section 3.2, it was observed that different layers have distinct generation preferences for the same sample, inspiring our further exploration in quantitative experiments. The first 32 layers (where layer 0 is the embedding layer) of the Osprey-7b model were organized into four groups: [0, 7], [8, 15] , [16, 23] , and [24, 31] . In each experiment, one group was selected as the candidate layer set and contrasted with the last layer to generate the candidate sentences, followed by cycle-consistency-based ranking to choose the best. Table 2 indicates that the first group exhibits the best performance in METEOR and nCHAIR I metrics. It is also noticeable that there are considerable performance discrepancies between the different groups, with a general trend of semantic quality declining as the depth of the layer increases (after contrastive decoding). However, we found that although the fourth group had the lowest METEOR score, its nCHAIR S was slightly better than that of the first group. This observation suggests that a well-trained MLLM exhibits variations in latent knowledge across its zones, leading to differences in generation performance. From the preceding experiments, we discovered that using layer importance in the next token prediction enhances the generated output's quality of the same dataset. Additionally, we also noticed that the determined layer prior importance can be smoothly transferred to a different dataset (different prompts) sharing a similar image domain to reduce the hallucinatory outputs. Specifically, we applied the layer importance ([0, 7] group) calculated on the RefCOCOg dataset directly to the PHD dataset, where the image inputs also originate from MSCOCO [25] , and decoded the output using the method described in Section 3.3.3. Given the absence of any region prompt in the PHD dataset, we applied a zero mask to the region encoder. The comparative results are shown in Table 3 . We observe that our approach enhances the model's understanding across most tasks, which indicates that without the additional training, the estimated layer prior importance could serve as a cold-start for inference in new environments when the probing dataset is not available, helping reduce the occurrence of hallucinations during the decoding. 
conclusion: 
 Our research on the MLLM-based Referring Expression Generation (REG) task explored a potential trade-off between information richness and reliability of the intermediate generation results. We introduced a "unleash-then-eliminate" approach that utilizes latent information from intermediate layers and employs a cycle-consistency-based decoding method to reduce hallucinations. Our method outperforms existing techniques, confirming its efficacy for enhancing REG task performance. 
experiment statistical significance: 
 Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments? Answer: [Yes] Justification: Since our framework is training-free and only make use of the testing data for layer importance estimation, when the full dataset is used the results should be deterministic and when only a subset is used, the results are also provided for comparison. Guidelines:• The answer NA means that the paper does not include experiments.• The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper. • The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions). • The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)• The assumptions made should be given (e.g., Normally distributed errors).• It should be clear whether the error bar is the standard deviation or the standard error of the mean. • It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified. • For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates). • If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text. 
experiments compute resources: 
 Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?Answer: [Yes]Justification: We provide the version of GPUs for experiments.Guidelines:• The answer NA means that the paper does not include experiments.• The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage. • The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute. • The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper). 
code of ethics: 
 Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines? 
broader impacts: 
 Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?Answer: [Yes] Justification: We discuss the potential usage and risk of our research outcomes.Guidelines:• The answer NA means that there is no societal impact of the work performed.• If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact. • Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.• The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster. • The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology. • If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML). 
safeguards: 
 Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?Answer: [NA]Justification: Not Applicable.Guidelines:• The answer NA means that the paper poses no such risks.• Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters. • Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images. • We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.12. Licenses for existing assets Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?Answer: [Yes] Justification: We credit the original codes used in our research.Guidelines:• The answer NA means that the paper does not use existing assets.• The authors should cite the original paper that produced the code package or dataset.• The authors should state which version of the asset is used and, if possible, include a URL. • The name of the license (e.g., CC-BY 4.0) should be included for each asset.• For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided. • If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset. • For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided. 
 
 Appendix: In this appendix, we present more ablation studies on our proposed decoding strategy, including the impact of different MLLM-based RES models and the influence of subset size in probing-based importance estimation. We also elaborate on the calculation of adopted hallucination metrics. At last, we provide a visualization that uncovers the multi-modal alignment process in the intermediate layers.In this section, we provide more ablation studies, including the impact of the different assisting RES models in cycle-consistency-based quality ranking, and the subset size in probing-based estimation.Different scoring RES model. In Table 4 , we ablate the impact of different RES models utilized in the proposed cycle-consistency-based quality ranking (CCR). We adopt another MLLM-based RES model LISA [18] to score the sentence quality of intermediate layers. From the table, we can observe that the choice of the RES model affects the CCR. The RES task itself involves transforming a language query into a pixel-level visual representation of an object. Given that GLaMM [37] performs slightly better than LISA [18] in RES, we can therefore infer that the more robust this transformation is completed, the better the performance of the CCR in quality ranking.Impact of probing subset quality and size. We also conducted ablation experiments on different subset sizes and the randomness of sampling, with the results listed in Table 5 . In the first row, we report the average result of 10 different random seeds for 1/8 subset sampling, which is also the result shown in Table 1 of the main paper. The second row reports the best results for the 1/8 subset. The results indicate that the randomness of the sampled subsets affects the performance of our proposed decoding strategy in reducing hallucinations. In other words, the quality of the subset has a certain impact on the decoding outcome.However, assessing the quality of subsets during probing is also an intractable problem. One possible solution is to use an unsupervised clustering method (e.g., K-means) to first cluster the multi-modal features (extracted by CLIP [35] /embedding layers of MLLM [55] ) of the entire dataset, and divide the subsets based on different centroids, then calculate and store the inter-layer weights in an "importance weight bank." During inference, we can compute the distance between the new query feature and these centroids, selecting the set of weights from the closest centroid for inter-layer combination during decoding. This strategy needs careful designing, and we consider it a future extension. Besides quality, by comparing the second and third rows, we also find that compared to the 1/8 subset, the 1/16 subset shows less stable de-hallucination and generation performance.The hallucination of objects in MLLM [14, 23, 38] refers to the situations where the descriptions generated by the model do not match the appearance of the object (attributes, relation etc.) in the original image. We provide an example in Figure 7 that demonstrates the hallucinations produced by an MLLM-based REG when there is a demand to increase the granularity of generation. In our study, we utilized two approaches to quantify the severity of the hallucination:The first approach is based on a widely adopted metric CHAIR [14, 38] , which directly counts the number of hallucinatory descriptions generated by the model. It relies on a reference expert table, providing the scope of the explicit object, and quantifies the object hallucination by calculating the ratio of "the objects mentioned but not in the expert table" to "all objects mentioned in a Is the person feeling anxious in the situation?No, he seems happy.Is the table made of wood?No, there is a glass table.In the image, is a motorcycle situated in front of the truck?The people in the image are undoubtedly feeling nervous. Based on the image, are the people feeling nervous in the situation?No, they are not nervous.No, there is no motorcycle.The bowl has a smooth, lightweight texture like a disposable container. Based on the image, is the bowl made of metal?Yes, it's a stainless steel bowl.The object to the right of the TV is used for carrying items. Based on the image, is the toilet to the right of the tv?Yes, the toilet is to the right of the TV. description". CHAIR divides the hallucination into two levels. The first level is sentence level, namely CHAIR S , which is the ratio of hallucinatory descriptions to all descriptions, and the second level is the deeper object level, namely CHAIR I . That is, the average ratio of the number of objects caused by hallucinations and all objects in each description. We refer to the detailed calculation methods from the original paper [38] and summarize them as the following formulas:EQUATIONCHAIR S = |{sentences with hallucinated object}| |{all sentences}|In this case, the CHAIR metric will get high scores as long as the model does not generate much but only the absolutely correct central object, but this goes against our goal of looking for high-granular information. Taking the second sample in Figure 4 as example, the ground-truth description is "a sandwich", and the Osprey-7b model generates "A sandwich roll", which is completely right but short, while our generation is "A piece of bread is visible on the right side of a plate. This bread appears to be a bun and is positioned in front of some tomatoes." Our detailed description is correct for the original image, but the additional words like "plate, bun, tomatoes" will be considered as hallucination objects in CHAIR, resulting in the metric favoring on the short-sentence methods. Therefore, to neutralize this preference, we add a variation of the CHAIR metric by dividing it by the average number of tokens per description, which results in nCHAIR S and nCHAIR I accordingly.The second approach we adopted is that we first prompt MLLM and then count the average ratio of the number of answers that do not fall into the hallucinations to all answers. We utilize the PHD benchmark [28] to achieve this. This benchmark sets ten different types of questions, which are composed of five different tasks, with each task featuring two modes of questioning. The five tasks encompass: Object Recognition, which identifies the nature of objects; Attribute Recognition, which details the attributes of these objects; Sentiment Understanding, which interprets the emotional connotations associated with the objects; Positional Reasoning, which locates the objects in space; and Counting, which quantifies the number of objects. The two modes of questioning include Neutral mode and Misleading mode. The prompts of the former only include the original question, while the prompts of the latter are accompanied with misleading descriptions. Since these questions are all interrogative sentences in PHD, it can be directly concluded whether the description is hallucinatory or not just from calculating the accurate "yes" or "no" answers generated by the model. The prompt and response examples are listed in Figure 6 .A dog is sitting on the left side of the table. It is wearing a black collar and a green tag. The dog is looking at the cake on the table. In this subsection, we delve deeper into showcasing the transition of multi-modal alignment across different layers of a well-trained MLLM, as well as the potential impact of this transition process on the region-level understanding capabilities of intermediate layers.Similar to Section 3.2, we considered 2000 triplets (I, M, Y) based on a pair of RES and REG models. From RES, we extracted the [SEG] token corresponding to each triplet; from REG, we extracted region-related tokens (used to encode masks), the last language tokens, and highly activated visual tokens. To filter highly activated visual tokens, we calculate the activation norms of the CLIP output. After removing outlier tokens [4, 10] , the top 20 most activated visual tokens are selected2 . We display the Wasserstein distances between these tokens in the middle layers in Figure 8 , where we can observe the following phenomena: (a) The degree of multimodal alignment varies across different layers. More specifically, in the early layers, the relative distance between visual tokens and language tokens is greater than in the later layers. (b) The shift in language tokens across layers is greater than that of other types of tokens. (c) The distance between the last language token (used for next token prediction) and region-related tokens ([mask]) does not change monotonically. Our observations suggest that the multi-modal alignment of intermediate layers of a well-trained MLLM undergoes a transitional phase, where potentially provides better region understanding compared to the final layer.Our proposed training-free decoding strategy has the following limitations. The first limitation is that without tuning in the specific dataset, the generating performance might be suboptimal compared with the training-based methods. Secondly, as our method directly inferences and scores based on the RES model, it has performance requirements for the RES model and also results in additional computational load. Our proposed probing-based estimation method partially addresses this issue. ). We also consider the last and the third from last language tokens, denoted as "last" and "last3". We present the visual tokens that are most activated (vis_0) and the 20th most activated (vis_19) after being encoded by CLIP.NeurIPS Paper ChecklistJustification: The research related contents conform the code of ethics.Guidelines:• The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.• If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics. • The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction).The version without RefCOCOg fine-tuning was used. https://huggingface.co/sunshine-lwt/ Osprey-7b/tree/main.We followed Bondarenko et al.[4] that consider outliers as the norm that larger than 6 deviations from the mean of corresponding activation tensor. 


Decision: This paper receives mostly positive rates (1 strong accept, 1 weak accept, 1 borderline accept and 1 borderline reject) thanks to the efficacy of the proposed model and convincing experimental results. Although overall reviews are positive, reviews still have some concerns, which should be addressed in the final version.

Reviews: review: paper_summary: paper_summary: The paper addresses the Referring Expression Generation (REG) task using Multi-modal Large Language Models (MLLMs), identifying the key challenge as the trade-off between generating detailed descriptions and accurately targeting the referring objects, which often leads to hallucinations—the inclusion of incorrect or spurious information in the generated text. To address this issue, the authors propose a training-free "unleash-then-eliminate" method that leverages the intermediate layers of MLLMs and employs a cycle-consistency-based decoding strategy to mitigate hallucinations. The proposed approach is validated through extensive experiments on the RefCOCOg and PHD benchmarks, demonstrating superior performance compared to existing techniques.
strength_weakness: strength_weakness: 1. Without tuning in the specific dataset, the generating performance of the method might be suboptimal compared to the training - based methods.

2. The methods used in the paper, such as cycle ranking, may introduce additional computational load, which could affect the per-sample decoding speed. Although the proposed strategy helps to alleviate this issue, it may not completely solve it.

3. The method assumes that the RES model can accurately estimate the region-aware descriptive performance of the captions generated by the candidate layers. However, the RES model may also have its own limitations and errors, which could affect the final evaluation results.
questions: questions: N/A
limitations: limitations: refer to weaknesses
paper_summary: paper_summary: The paper aims to strike a balance between detailed description and precise captioning when using multimodal large language models (MLLM) in the task of referring expression generation (REG). A key observation is that the output of a Referring Expression Segmentation (RES) model should be consistent with the input of a REG model.

Based on this observation, a training-free method has been proposed, called 'unleash-then-eliminate,'  which adopts an 'elicit-then-eliminate decoding' strategy. Captions are generated using contrastive decoding (Li et al., arXiv:2210.15097) and then fed into a RES model to produce corresponding masks. The Intersection over Union (IoU) between the masks tagged by the RES model and the input of the REG model is leveraged to select the candidate layer along with the generated caption. Additionally, a Probing-based Importance Estimation method is proposed to accelerate the decoding process.

The generation quality of the model is evaluated using the METEOR score on the RefCOCOg dataset. Additionally, the model's performance in avoiding hallucinations is assessed with the CHAIR and PHD metrics. The model proposed in this paper outperforms both Osprey and Dora not only in terms of generation quality but also in terms of adequacy.
strength_weakness: strength_weakness: 1. The writing is disorganized and lacks clarity. For example, in the first sentence "Referring expression generation (REG) Yu et al. [2016], Mao et al. [2016], Hu et al. [2016], Yu et al. [2017], Luo et al. [2020], Ding et al. [2021], Tanaka et al. [2019] is a task to", the reference is mixed with words, making it difficult to follow.

2. The proposed approach is technically simple. The architecture seems simply a combination of REG and RES models.

3. The motivation is to achieve a trade-off between the granularity and accuracy of captioning by selecting different intermediate layers. However, the experiments only show that the proposed model outperforms other models, as indicated in Table 1. Furthermore, Table 2 reveals that the first bucket of layers achieves the best scores in both METEOR, which relates to generation quality, and nCHAIR_I, which relates to the avoidance of hallucinations. This finding contrasts with the initial proposal.

4. The answer to `Open access to data and code` is [Yes], but there's only a `placeholder` found in the anonymous repo,  noted as footprint in the first page of paper.
questions: questions: 1. What is the final architecture of the proposed model with `Probing-based Importance Estimation` ?

2. What are `the number of parameters` of the proposed models (with and without Probing-based Importance Estimation)?

3. What are the metrics corresponding to those types (i.e. Object Recognition...) in table 3? What is the difference between `neutral-question mode` and `misleading mode`?
limitations: limitations: Yes.
The authors claim that since the model has not been tuned on a specific dataset, the generating performance is suboptimal compared to training-based methods.
paper_summary: paper_summary: This paper presents an approach for improving the accuracy and richness of referring expression generation (REG) by leveraging the descriptive potential of intermediate layers in Multi-modal Large Language Models (MLLMs). The method employs a cycle-consistency-based decoding strategy to reduce hallucinations and improve descriptive quality. The proposed method is evaluated on the RefCOCOg and PHD benchmarks, demonstrating superior performance over existing methods.
strength_weakness: strength_weakness: The proposed method introduces additional complexity, particularly in the decoding process. While effective, the cycle-consistency-based approach may increase computational overhead, which could limit its applicability in real-time or resource-constrained environments.

The related work section lacks depth in its analysis and could benefit from a more thorough review of recent advancements, particularly in hallucination mitigation techniques.

Some terms and notations, such as Q, H and W in Section 3.1 , are not clearly defined in the context of the paper, which can lead to confusion. Providing explicit definitions and clarifications would improve readability.
questions: questions: You use PCA for dimensionality reduction and Wasserstein distances for comparing features. Can you elaborate on why these specific techniques were chosen and how they contribute to the effectiveness of your method compared to other possible choices?

Your method aims to reduce hallucinations in generated descriptions. Can you provide more details on how you quantify hallucinations and the specific improvements observed with your method compared to baselines? Are there any cases where your method still struggles with hallucinations?

Is there any Transformer-based region encoder?
limitations: limitations: The authors have not adequately addressed the limitations and potential negative societal impacts of their work.
paper_summary: paper_summary: This paper explores the Multi-modal Large Language Model (MLLM) based Referring Expression Generation (REG) task, which aims to generate unambiguous text descriptions for specific objects or regions in images. MLLM-based REG models tend to suffer from hallucination issues, and there is a trade-off between detailed descriptions and accurate targeting. To address this, a training-free method called "unleash-then-eliminate" is proposed, which elicits latent information in intermediate layers and uses cycle-consistency decoding to reduce hallucinations. Extensive experiments on the RefCOCOg and PHD benchmarks show that this method outperforms existing approaches in both semantic and hallucination-related metrics.
strength_weakness: strength_weakness: 1. Since the proposed method requires multiple layers for inference to be described, both REG and RES models are required, resulting in a very low efficiency of the method and requiring a lot of additional calculations and memory.
2.  The ablation study is not sufficient. For example, subset used to select the optimal layer in Layer Importance Measurement. What is the Impact of subset size and quality on performance? What is the performance of using  cycle-consistency-based candidate ranking process for whole dataset, i.e., not using Layer Importance Measurement.
3.  What is effect of using different RES models for the proposed method.
questions: questions: 1. What is the effect of \alpha defined in Line 172
2. \mathcal{V} is not defined in Line 171.
3.  The candidate sentence set A use n to represent  the size of candidate layers, while use N-1 in Line 169. So are n and N-1 the same, or do they represent different things?
limitations: limitations: The authors adequately addressed the limitations.

Comments: comment: We really appreciate your acknowledgment of our responses to the initial comments, and the adjustment of the rating. We would be grateful for the opportunity to discuss any remaining concerns that could be addressed in our manuscript further. Thank you very much for your time!
Thank you so much for your encouraging feedback and for upgrading your score to a "strong accept." We are truly grateful for your recognition of the efforts made in our rebuttal. It's great to hear that our responses have successfully addressed your concerns. We're committed to continuously improving our work, taking into account your insights and those from other reviewers. Thanks again for your supportive words!
The author did address some of my concerns, and I keep my initial rating.
Thanks for the response. Authors made a good rebuttal. After carefully reading it, I find all my concerns have been properly solved. I also would like to say,  every research paper may have some imperfections, but I am more concerned with whether the paper offers new insights that can drive further exploration in the field. The presence of minor flaws does not diminish the value of a work that makes a meaningful contribution. From my perspective, the quality of this work has satisfactorily met the established requirements for acceptance. Thus, I would like to give STRONG ACCEPT as my final score.
The author did address some of my concerns and I'll raise my score from 3 (Reject) to 4 (Borderline reject).
Thank you once again for your insightful comments. We deeply appreciate your guidance and found great resonance with your perspectives. With the discussion period nearing completion in less than two days, please feel free to share any final comments at your convenience.
It is great to know that our responses are helpful, and thank you for your positive feedback and support for this work! Your suggestions have helped improve this work and we will incorporate them in the revised manuscript.
Thank you once again for your valuable comments and thorough review. We have tried our best to clarify the concerns on the paper. Your detailed feedback has enhanced the clarity of our explanations. With the discussion period nearing completion in less than two days, we would appreciate it if you could let us know if any aspects remain unclear. We truly appreciate this opportunity to improve our work and shall be most grateful for any feedback you could give us.
Thank you once again for your insightful comments! We deeply appreciate your precise feedback and truly resonate with your perspectives, which have been instrumental in enhancing our work. With the discussion period nearing completion in less than two days, please feel free to share any final remarks at your convenience.