Title: Zooming from Context to Cue: Hierarchical Preference Optimization for Multi-Image MLLMs
Venue: NeurIPS 2025
Accepted: yes
Keywords: Multi-modal Large Language Models (MLLMs)

Direct Preference Optimization

Hallucination Mitigation
Abstract: Multi-modal Large Language Models (MLLMs) excel at single-image tasks but struggle with multi-image understanding due to cross-modal misalignment, leading to hallucinations (context omission, conflation, and misinterpretation). Existing methods using Direct Preference Optimization (DPO) constrain optimization to a solitary image reference within the input sequence, neglecting holistic context modeling. To address this, we propose Context-to-Cue Direct Preference Optimization (CcDPO), a multi-level preference optimization framework that enhances per-image perception in multi-image settings by zooming into visual clues—from sequential context to local details. Our approach features two sequentially dependent components: (i) Context-Level Optimization: By introducing low-cost sequence preference pairs, we optimize the model to distinguish between complete and disrupted multi-image contexts, thereby correcting cognitive biases in MLLMs’  multi-image understanding. (ii) Needle-Level Optimization: By integrating region-specific visual prompts with multimodal preference supervision, we direct the model’s attention to critical visual details, effectively suppressing perceptual biases toward fine-grained visual information. To support scalable optimization, we also construct MultiScope-42k, an automatically generated multi-image dataset with hierarchical preference pairs. Experiments show that CcDPO significantly reduces hallucinations and yields consistent performance gains across general single- and multi-image tasks. Codes are available at https://github.com/LXDxmu/CcDPO.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper aims to address common hallucinations in MLLMs when processing multi-image inputs, specifically context omission, conflation, and detail misinterpretation. To this end, the authors propose a hierarchical preference optimization framework named CCDPO. This framework is divided into two levels: the Context-Level, which optimizes the model's understanding of the global sequence through perturbation strategies like sequence truncation and content swapping , and the Needle-Level, which enhances the model's perception of local, key details through a combination of visual prompts and vision-contrastive learning. Additionally, the authors constructed an automatically generated preference dataset, MultiScope-42k, to support the training of this framework. Experimental results indicate that this method achieves performance gains across multiple multi-image and single-image benchmarks.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n1. The paper provides a valuable and clear categorization of hallucinations in multi-image understanding, identifying them as context omission, context conflation, and detail misinterpretation. This classification offers distinct targets and a focused direction for the methodological design that follows.\n2. The model is evaluated extensively on a wide range of multi-image and single-image benchmarks, with performance compared against relevant baselines like SFT and MIA-DPO. Additionally, the paper includes detailed ablation studies to validate the effectiveness of the framework's different components.\n\nWeaknesses:\n1. The paper's core contribution lies in its novel combination of existing techniques rather than a fundamental methodological breakthrough. DPO and its applications in multimodal contexts are previously established, with the paper itself citing MIA-DPO as a baseline for multi-image tasks. The vision-contrastive learning at the needle-level also explicitly draws from prior work. Therefore, the work presents itself more as a sophisticated engineering and integration effort, with its methodological novelty being somewhat limited.\n\n\n2. The ablation study does not analyze the individual effects of the Needle-Level and Context-Level optimizations. A more direct comparison could have provided better insight into the contribution of each method to downstream task performance.\n\n3. In the ablation study tables, shorthand terms like Needle-Levi-TDPO and Needle-Levi-VDPO are used without being clearly defined in the main text of the paper. This lack of explicit definition can cause confusion for the reader.\n\nQUESTIONS:\nSee weaknesses.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 2, "DATE": "2025-06-29"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes Context-to-Cue Direct Preference Optimization (CcDPO), a hierarchical framework to improve multi-image understanding in Multi-modal Large Language Models (MLLMs). The method tackles three common hallucination issues - context omission, context conflation, and detail misinterpretation - by introducing two levels of preference alignment: (1) Context-Level Optimization, which promotes accurate and complete multi-image comprehension through structured per-image captions and perturbation-based training; and (2) Needle-Level Optimization, which enhances fine-grained perception using region-level visual prompts and contrastive supervision. To support training, the authors construct MultiScope-42k, a large-scale dataset with automatically generated multi-level preference pairs. Experiments on a wide range of multi-image and single-image benchmarks demonstrate that CcDPO consistently improves performance and reduces hallucination compared to prior methods like MIA-DPO.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\nThe paper presents a well-motivated and clearly described approach to improving multi-image understanding in MLLMs, addressing common hallucination issues. The proposed CcDPO framework is conceptually sound, combining structured context-level and fine-grained needle-level preference optimization. The methodology is supported by a newly constructed dataset, MultiScope-42k, which enables scalable training. The empirical results are thorough, covering a wide range of benchmarks, and show consistent improvements over strong baselines. The writing is clear and the paper is easy to follow.\n\nWeaknesses:\nThe overall design is solid, but the method mainly combines existing techniques such as DPO and visual prompting in a hierarchical way, rather than introducing fundamentally new mechanisms. In addition, as the authors acknowledge, the framework does not explicitly handle temporal reasoning, which may limit its applicability to video-related tasks.\n\nQUESTIONS:\nThe paper presents an impressive contribution with the CcDPO framework, supported by a robust dataset (MultiScope-42k) and thorough evaluations. My only minor concern is in Table 1, where the bolded result for Qwen2-VL on the MIRB task does not correspond to the highest score, which may confuse readers. Please clarify if this is intentional or correct the bolding, and consider verifying other tables for consistency. This small revision would further polish an already strong submission.\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 4, "IMPACT": 4, "ORIGINALITY": 3, "DATE": "2025-07-01"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces CcDPO (Context-to-Cue Direct Preference Optimization) as a hierarchical preference optimization framework for improving multi-image understanding problems of Multimodal Large Language Models (MLLMs). The authors identify three prevalent types of hallucinations—context omission, context conflation, and detail misinterpretation—and propose a two-level optimization strategy to address them: (i) Context-Level Optimization improves holistic sequence comprehension via structured per-image captions and perturbation-based contrastive training; (ii) Needle-Level Optimization enhances fine-grained perception using region-level visual prompts and vision-language contrastive objectives. The proposed method is trained on a newly constructed dataset, MultiScope-42k, and achieves improvements across multiple multi-image benchmarks (e.g., MUIRBench, MIBench, MIRB). \n\nHowever, the authors do not include any ablation or comparative experiments where CcDPO is trained using existing public datasets (e.g., LLaVA-23K, MDVP, MVC) to isolate the contribution of the optimization method itself versus the data quality and scale. Also, I do not find any supplementary materials or documentation that would help reproduce the reported results using the released code. The provided README appears to be identical to that of LLaVA-NeXT: Open Large Multimodal Models, without any specific instructions for training or evaluating CcDPO.\n\nSTRENGTHS AND WEAKNESSES:\nPros:\n1. The paper addresses a critical challenge in multi-image understanding. The authors clearly articulate the importance of this issue and provide a thoughtful taxonomy of error types, which helps ground the methodological design.\n\n2. The authors present a carefully constructed dataset (MultiScope-42k) tailored for hierarchical preference optimization, which effectively supports both context-level and fine-grained training objectives.\n\n3. The proposed two-level optimization framework (CcDPO) is methodologically sound and conceptually intuitive. \n\nCons: \n1. The released code lacks any specific README or documentation related to CcDPO. It appears to be identical to the LLaVA-NeXT repository, with no visible modifications or guidance for reproducing the results. This makes the authors’ claim of code release less convincing and significantly limits reproducibility.\n\n2. There is no ablation study evaluating CcDPO without the proposed MultiScope-42k dataset. As a result, it is unclear whether the performance gains stem from the optimization method itself or simply from the quality and scale of the new dataset. This is unfair for the baseline models that cannot access the MultiScope-42k datasets. \n\n3. The two proposed optimization stages—context-level and needle-level—are designed independently and operate in parallel. There is no clear logical or learning dependency between them, which weakens the hierarchical narrative implied in the method’s name and motivation.\n\nQUESTIONS:\n1. Could the authors clarify whether the released codebase includes any implementation of CcDPO beyond what is already available in LLaVA-NeXT? A readme of CcDPO is preferred in response.\n\n2. Could the authors provide an ablation study where CcDPO is trained without MultiScope-42k, for example using only existing datasets such as LLaVA-23K or MDVP? Also, the baselines trained with MultiScope-42k is also necessary for verification. \n\n3. The two optimization components—context-level and needle-level—appear to be trained independently and applied in parallel. Could the authors clarify whether there is any interaction or dependency between them during training or inference?\n\nThe major limitations are stated in the weakness part. \nI encourage the authors to respond the ablation study of MultiScope-42k, as well as the readme of codes for reproduction.\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 2, "REVIEWER_CONFIDENCE": 4, "CLARITY": 2, "IMPACT": 1, "ORIGINALITY": 1, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces a method for mitigating bias and hallucination in multi-image question answering (QA) with multimodal LLMs (MLLMs). The authors start by identifying the potential root causes of these errors: 1) context omission, 2) context conflation, and 3) misinterpretation. The authors constructed a preference optimization dataset to guide the MLLMs to avoid these issues. The authors also introduce the needle level preference optimization, which adds visual prompts to further guide the model towards preventing errors.  \n\nThe proposed method is applied to open-sourced foundation LLMs and tested on multi-image QA benchmarks. Improvement in response quality is observed. The authors also provided ablation studies to justify the effectiveness of the proposed method over SFT and previous baselines.\n\nSTRENGTHS AND WEAKNESSES:\n+ The problem of multi-image MLLMs is both important and interesting. It is great that the authors identify existing methods' problems with detailed analysis and empirical results. The analysis is technically sound to me. \n+ Using multi-image captioning as a proxy to gauge the context understanding of multi-image MLLMs is an interesting take. \n+ The proposed method directly addresses the issues identified with the existing methods. It uses light-weight DPO approaches so practitioners can easily adopt it. \n+ Improvements in benchmark results are observed, and clear ablation studies are presented.\n\nWeaknesses\n- Details of how to construct visual prompts are missing.\n\nQUESTIONS:\nUsing DPO to guide the model towards better spatial and sequential understanding seems to be an interesting direction. I am wondering in terms of spatial reasoning, whether the format of the added visual prompt has an impact on the quality. In the current text it is roughly described as \"bounding boxes and key points\".  Can the authors give more details on how the visual prompts are injected and what are the impact of different types of visual prompts?\n\nLIMITATIONS:\nYes.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-03"}